Robot collision avoidance method, device and equipment based on wheel detection and recognition

Abstract

The present invention is applicable to the field of mechanical control technology and provides a robot collision avoidance method, device, and equipment based on wheel detection and identification. The method comprises: collecting image data of a vehicle to be tested; detecting wheels in the image data; obtaining wheel point cloud data including distance information to determine the number and position of wheels; screening wheel groups based on the number and position of wheels; calculating the deflection angle of the robot relative to the vehicle to be tested based on the wheel groups; adjusting the robot's operating angle based on the deflection angle; and adjusting the offset distance of the robot after the angle adjustment based on the midpoint position of the wheel group. The robot's driving state is adjusted so that it travels along the central axis to avoid collisions that lead to detection failures. The accuracy of the collected data is determined, the success rate of underbody detection is improved, the risk of collision is avoided, and the detection efficiency and accuracy of the detection results are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of mechanical control, and in particular relates to a robot collision avoidance method, device and equipment based on wheel detection and identification. Background Art

[0002] With the development of robotics technology, robots are widely used in various fields. Among them, robots, as intelligent terminals for automobile underbody inspection, have strong practicality. However, due to the variability of the application environment, it is inevitable that the robot will accidentally drive out from the middle of the vehicle body or directly hit the wheel when operating under the vehicle, thereby interrupting the inspection task.

[0003] In actual application scenarios, the robot's wheels slip, the ground is uneven, and other factors may cause the robot's driving direction to deviate, making it unable to move along the central axis, resulting in collisions with the wheels and driving out of the bottom of the vehicle. As a result, the path planning route may be curved, and the collected bottom vehicle data may be distorted, affecting the detection results. Summary of the Invention

[0004] In order to solve the technical problems existing in the existing technology in actual application scenarios, the robot's wheel slips, the uneven ground, etc. cause the robot's driving direction to deviate, and it cannot move along the central axis, resulting in collisions with wheels and driving out of the bottom of the vehicle midway, which in turn causes the path planning line to be curved and the collected bottom vehicle data to be distorted, affecting the detection results.

[0005] First aspect

[0006] The present invention provides a robot collision avoidance method based on wheel detection and recognition, which is applied to a robot. A camera is set in front of the robot. The method includes:

[0007] Collect image data of the vehicle to be tested;

[0008] detecting wheels in image data;

[0009] Obtain wheel point cloud data including distance information to determine the number and position of wheels;

[0010] Filter wheel sets based on wheel quantity and wheel position;

[0011] Calculate the deflection angle of the robot relative to the vehicle under test based on the wheel set;

[0012] Adjust the robot's operating angle according to the deflection angle;

[0013] According to the midpoint position of the wheel group, the offset distance of the robot after angle adjustment is adjusted.

[0014] Second aspect

[0015] The present invention provides a robot collision avoidance device based on wheel detection and identification, comprising:

[0016] Acquisition module: used to collect image data of the vehicle to be tested;

[0017] Detection module: used to detect wheels in image data;

[0018] Acquisition module: used to obtain wheel point cloud data including distance information, and determine the number and position of wheels;

[0019] Screening module: used to screen wheel sets according to wheel quantity and wheel position;

[0020] Calculation module: used to calculate the deflection angle of the robot relative to the vehicle under test based on the wheel set;

[0021] The first adjustment module is used to adjust the robot's operating angle according to the deflection angle;

[0022] The second adjustment module is used to adjust the offset distance of the robot after the angle adjustment according to the midpoint position of the wheel group.

[0023] The third aspect

[0024] An embodiment of the present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the robot collision avoidance method based on wheel detection and recognition in the first aspect are implemented.

[0025] The present invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the steps of the robot collision avoidance method based on wheel detection and identification in the first aspect are implemented.

[0026] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0027] When adjusting the robot, the present invention selects a stable characteristic wheel position. Based on wheel detection and positioning and corresponding wheel point cloud data, the left and right wheel groups are obtained from the identified wheels in different situations, and then calculations and processing are performed according to the situation. This is suitable for handling a variety of complex situations, improving the robot's adaptability and applicability to different operating scenarios. Wheel position information is used to determine whether the robot has deflected and whether it is on the central axis. This allows the robot's movement direction to be adjusted in a timely manner, ensuring that the robot passes under the vehicle to be inspected along the central axis. This ensures the accuracy of the collected data, improves the success rate of under-vehicle inspection, avoids the risk of collision, and improves inspection efficiency and the accuracy of the inspection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0029] Figure 1 This is a flow chart of a robot collision avoidance method based on wheel detection and identification provided by the present invention;

[0030] Figure 2 This is a schematic structural diagram of a robot provided by the present invention;

[0031] Figure 3 This is a schematic diagram of the deflection angle of a robot provided by the present invention;

[0032] Figure 4 Schematic diagram of a robot provided by the present invention in different states;

[0033] Figure 5 This is a schematic diagram of a correction of a robot provided by the present invention when it is rotated to align with the midpoint position in a right deflection state;

[0034] Figure 6 This is a schematic structural diagram of a robot collision avoidance device based on wheel detection and identification provided by the present invention;

[0035] Figure 7 It is a structural diagram of a terminal device provided by the present invention. DETAILED DESCRIPTION

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the specific embodiments of the present invention will be described below with reference to the accompanying drawings. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings and other embodiments can be obtained based on these drawings without inventive work.

[0037] To simplify the drawings, only portions relevant to the invention are schematically depicted in each figure; they do not represent the actual structure of the product. Furthermore, to simplify the drawings and facilitate understanding, in some figures, only one component with the same structure or function is schematically depicted or labeled. In this document, "one" not only means "only one" but also "more than one."

[0038] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0039] It should be noted that, unless otherwise specified or limited, the terms "mounted," "connected," and "connected" should be understood broadly. For example, they can refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediary; and internal communication between two components. Those skilled in the art will understand the specific meanings of these terms in the present invention based on the specific circumstances.

[0040] In addition, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0041] First, the present invention provides a robot collision avoidance method based on wheel detection and recognition, which is applied to a robot, and a camera is set in front of the robot.

[0042] In one embodiment, the reference Figure 1 , a flow chart of a method for walking and avoiding collision provided by the present invention. Figure 2 , a schematic structural diagram of a robot provided by the present invention.

[0043] like Figure 2 As shown, the robot has a camera capable of collecting true color (RGB) image data and depth image data. The camera is installed in front of the robot and tilted upward.

[0044] like Figure 1 As shown, the robot collision avoidance method based on wheel detection and recognition includes the following steps:

[0045] Collect image data of the vehicle to be tested.

[0046] The image data includes true color image data and depth image data.

[0047] RGB image data is collected by an RGB-D camera, and can also be captured by a regular camera. Depth image data, also known as depth image data, can be obtained by a depth camera. Depth cameras use special technologies such as structured light, time of flight, or binocular vision to measure the distance from objects in the scene to the sensor and generate corresponding depth images. RGB image data and depth image data can be acquired simultaneously by the same camera or separately by different cameras.

[0048] The collected RGB image data of the vehicle under test and the depth image data corresponding to all pixels in the RGB image data are aligned with each other. If the true color image data and depth image data are acquired simultaneously through the same camera, the true color image data and depth image data can be aligned using intrinsic parameter alignment. Specifically, the true color image and depth image are aligned in the camera coordinate system using intrinsic parameters, which include the camera's focal length, optical center (image center), distortion parameters, etc. If the true color image data and depth image data are acquired separately through different cameras, the true color image and depth image can be aligned in the world coordinate system. First, the pixels in the depth image are converted to three-dimensional points in the world coordinate system using the depth value and extrinsic parameter information. Then, using the pixel coordinates in the true color image and the camera's extrinsic parameter information, the points in the true color image are converted to three-dimensional points in the world coordinate system. Finally, based on the coordinate correspondence, the true color image and depth image can be aligned.

[0049] By aligning the true color image data and the depth image data, it is possible to ensure that the collected wheel data completely and accurately reflects the distance information between the wheel and the camera in the corresponding RGB image data.

[0050] Detect wheels in image data.

[0051] Specifically, the wheel detection and positioning model is used to detect whether there is a wheel in the image. If no wheel is detected, the operation is terminated. The wheel detection and positioning model can be selected according to actual needs. The existing image recognition technology is sufficient to obtain wheel features from true color image data.

[0052] Obtain wheel point cloud data including distance information to determine the number and position of wheels.

[0053] The distance information is the distance between the camera and each wheel.

[0054] It should be noted that a loop traversal is performed within the acquired wheel detection frame to obtain the distance information between the wheel and the camera, and then the depth pixel points are selected or discarded based on the distance information, and those that meet the conditions are retained. Finally, the wheel point cloud data of each wheel is obtained, the feature data of the wheel is obtained, and then the robot is positioned based on the wheel point cloud data. The wheel detection frame is determined first, rather than traversing directly. This greatly reduces the resource consumption required for traversal and the data acquisition speed, thereby improving the robot's work efficiency.

[0055] In one possible implementation, the image data includes true color image data and depth image data, and the distance information is the distance between the camera and each wheel. Acquiring wheel point cloud data including the distance information and determining the number and position of the wheels specifically includes:

[0056] The true color image data is input into the wheel alignment model to obtain the detection frames of all wheels. The number of detection frames is the number of wheels.

[0057] Among them, since the robot may deviate during operation, the data collected by the camera may include multiple wheels, which may be one or more wheels.

[0058] Loop through the depth image data corresponding to the detection frame and obtain the depth pixel points corresponding to the detection frame.

[0059] Set the initial distance threshold and initial grayscale threshold.

[0060] In actual use, most tires are dark in color. Therefore, the grayscale of the pixels in the wheel area is judged. Those with a grayscale less than the grayscale threshold are considered to be tires, thereby eliminating non-wheel points. In addition, the depth value of the wheel area is screened, and those with a depth value less than the distance threshold are considered to be valid data. In this way, noise points are eliminated, making the generated wheel point cloud more accurate.

[0061] It should be noted that those skilled in the art can set the distance threshold and the grayscale threshold according to actual needs, and the present invention does not limit this.

[0062] According to the initial distance threshold and initial grayscale threshold, the noise points in the depth pixels are removed to create the wheel point cloud data including distance information:

[0063]

[0064] Among them, cx, cy, fx, and fy represent internal parameters in true color image data, th1 represents the initial distance threshold, th2 represents the initial grayscale threshold, gray represents grayscale image data, rgb represents true color image data, depth represents depth image data, and i and j represent the column coordinates and row coordinates of the pixel, respectively.

[0065] Count the number of points in each wheel point cloud data.

[0066] The number of points refers to the number of points in the point cloud data. A point cloud is a dataset consisting of a series of three-dimensional points. Each point has coordinate information indicating its position in space. The number of points refers to the number of these points and is used to indicate the density or density of the point cloud data. A larger number of points usually indicates that the point cloud data is more detailed and complete. The point cloud centroid (or center of mass) refers to the average position of all points in the point cloud. In three-dimensional space, each point has a three-dimensional coordinate representing its position. By calculating the average value of the coordinates of all points in the point cloud, the coordinates of the point cloud centroid can be obtained. The point cloud centroid can be used as an approximation of the center position of the point cloud data to represent the overall position of the point cloud.

[0067] When the number of points in the wheel point cloud data is greater than or equal to the preset number, determine the wheel position:

[0068]

[0069] Among them, pose c (x, y, z) represents the point cloud centroid, that is, the wheel position, and pointCloud represents the wheel point cloud data.

[0070] In one possible implementation, obtaining wheel point cloud data including distance information and determining the number and positions of wheels further includes:

[0071] When the number of points in the wheel point cloud data is less than a preset number, the wheel point cloud data is deleted.

[0072] It's understandable that the number of points reflects the density of the wheel point cloud, while the point cloud centroid can be used to calculate the center of the wheel point cloud, which is used to determine the wheel's position and posture. Analysis of the number of points and the point cloud centroid allows for wheel positioning, quality assessment, and subsequent processing. Calculating the point cloud centroid requires that the number of points meet certain requirements. If the number of points is too small to calculate, the point cloud centroid is discarded, thereby reducing the influence of irrelevant factors.

[0073] Filter wheel sets based on number of wheels and wheel position.

[0074] The wheel set includes two matching wheels.

[0075] It is understandable that during the movement of the robot, the number of wheels collected by the robot may be multiple wheels due to its different positions, and the need to obtain matching left and right wheel groups for different numbers of wheels is different. If there are two wheels, it is easy to determine the matching relationship, but for multiple wheels, other methods need to be used for matching, thereby improving the robot's work adaptability to scenarios and improving the robot's efficiency in handling complex situations.

[0076] In a possible implementation, screening the wheel set according to the number of wheels and wheel positions specifically includes:

[0077] According to the general standards for automobiles, a first distance threshold, a second distance threshold, and a third distance threshold are set, wherein the first distance threshold and the second distance threshold are used to limit the upper and lower limits of the vehicle width, respectively, and the third distance threshold is used to limit the degree of difference between the distance information of the left and right wheels.

[0078] It should be noted that the distance and width between the left and right wheels of a conventional vehicle are generally fixed within a certain range, and the positions of the two wheels relative to the vehicle body are the same. The depth difference between the wheels in different coordinate systems is also fixed within a certain range. Based on this, a threshold value can be set to determine whether the two wheels are matching left and right wheels. Based on this, the first distance threshold, the second preset distance, and the third distance threshold are set to locate the specific position of the robot and make this determination, thereby more accurately controlling the robot's driving direction. It is understood that those skilled in the art can set the values ​​of the first distance threshold, the second preset distance, and the third distance threshold according to actual needs, and the present invention is not limited here.

[0079] When the number of wheels is 2, the first distance threshold, the second distance threshold, and the third distance threshold are combined to determine whether the two wheels form a left and right wheel group. The specific determination method is:

[0080]

[0081] Among them, d th0 represents the first distance threshold, d th1 represents the second distance threshold, d th2 represents the third distance threshold, pose1.x represents the x-coordinate of the first target wheel, pose2.x represents the x-coordinate of the second target wheel, pose1.z represents the distance information of the first wheel, and pose2.z represents the distance information of the second wheel.

[0082] When the number of wheels is 3, the three wheels are arranged in ascending order according to the distance information to obtain the first target wheel, the second target wheel and the third target wheel.

[0083] When the absolute value of the distance information difference between the first target wheel and the second target wheel is smaller than the absolute value of the distance information difference between the second target wheel and the third target wheel, the first target wheel and the second target wheel are set as wheels to be matched; otherwise, the second target wheel and the third target wheel are set as wheels to be matched.

[0084] Determine whether the wheel to be matched is a wheel set. The specific determination method is as follows:

[0085]

[0086]

[0087] Where th represents the detection box area ratio threshold of the wheels to be matched, rect_area represents the detection box area of ​​the wheels to be matched, pose1, pose2, and pose3 represent the first target wheel, the second target wheel, and the third target wheel, respectively.

[0088] It should be noted that the distance between the left and right wheels of the same group and the camera is not particularly different, so the area ratio of the left and right wheel detection frames is constant. However, the distance between the wheels of different groups and the camera varies greatly, and the area ratio of the detection frames is smaller. Therefore, the detection frame area rect_area is used as a basis here to improve the matching accuracy.

[0089] When the number of wheels is 4, the distance between each wheel is traversed and the left and right wheel groups are selected based on the distance. The specific selection method is as follows:

[0090]

[0091] Among them, d ij Indicates the separation distance, d th3 represents the fourth distance threshold, d th4 Indicates the fifth distance threshold.

[0092] When the screened wheel groups are one group, the left wheel and the right wheel in the wheel group are determined by the following method:

[0093]

[0094] In the case that there are multiple wheel groups screened out, the wheel group closest to the camera is screened out from the multiple left and right wheel groups using distance information.

[0095] Reference Figure 3 , which shows a schematic diagram of the deflection angle of a robot provided by the present invention.

[0096] Figure 3 The situations in which the robot deviates to the left and to the right are given in FIG.

[0097] Reference Figure 4 , showing a schematic diagram of a robot provided by the present invention in different states.

[0098] Figure 4 The following diagram shows the robot in normal, displacement, and angular deviation conditions. The robot is moving directly toward the midpoint between its two wheels, with its direction of motion perpendicular to the line connecting the two wheels. This indicates a normal robot condition. Displacement deviation occurs when the robot's direction of motion deviates from the midpoint between the two wheels. Angular deviation occurs when the robot's direction of motion is not perpendicular to the line connecting the two wheels.

[0099] Reference Figure 5 , showing a schematic diagram of the correction of the rotation alignment midpoint of a robot provided by the present invention in a right deflection state.

[0100] Figure 5The correction method for the robot in the right offset state is given in Figure 2. In the right offset state, the robot first rotates to adjust the movement direction toward the midpoint between the two wheels to correct the robot's angular deviation.

[0101] Based on the wheel set, the deflection angle of the robot relative to the vehicle under test is calculated.

[0102] It should be noted that the ideal state of the robot should be moving along the central axis at the position of the left and right wheel groups, but in reality displacement or angle deviation often occurs, resulting in wheel collision. To achieve the ideal state, it is necessary to adjust the robot direction according to the deflection angle calculated by the wheel, and then adjust the robot position according to the displacement of the central axis. Since the wheel feature is the easiest to obtain and process during the movement of the robot, by determining the left and right wheels in the left and right wheel groups, and then calculating the corresponding geometric data based on the symmetrical left and right wheels, the deflection angle of the robot from the central axis can be obtained. This process consumes less computing resources and is highly efficient, which improves the real-time processing capability and processing efficiency of the robot.

[0103] In one possible implementation, calculating the deflection angle of the robot relative to the vehicle to be tested based on the wheel set specifically includes:

[0104] A coordinate system is established with the line connecting the left and right wheels of the wheel group as the x-axis and the direction perpendicular to the x-axis and consistent with the movement direction of the robot as the y-axis.

[0105] Get the x-coordinate values ​​of the wheels in the wheel set, and use the wheel with the larger x-coordinate value as the right wheel, and the other wheel as the left wheel.

[0106] Calculate the deflection angle based on the distance between the right wheel and the left wheel:

[0107]

[0108] Among them, pose R Indicates the right wheel, pose L represents the left wheel, and ang represents the deflection angle.

[0109] Adjust the robot's operating angle according to the deflection angle.

[0110] In a possible implementation, the operation angle of the robot is adjusted according to the deflection angle as follows:

[0111] When the distance information of the left wheel is greater than the distance information of the right wheel, the robot is rotated to the right; otherwise, the robot is rotated to the left.

[0112] It should be noted that during the detection process, a large amount of calculation data should be placed in the detection algorithm rather than adjusting the posture. In order to avoid excessive adjustment of the robot's moving posture, a preset deflection angle needs to be set before the angle adjustment, allowing the robot to perform direct detection while being able to perform under-vehicle detection normally, avoiding excessive adjustment. This not only reduces the resource consumption of posture correction, but also improves the robot's fault tolerance and improves detection efficiency. It should be noted that those skilled in the art can set the size of the preset deflection angle according to actual needs, and the present invention does not limit this.

[0113] According to the midpoint position of the wheel group, the offset distance of the robot after angle adjustment is adjusted.

[0114] It should be noted that the purpose of adjusting the offset distance of the robot after the angle adjustment is to make the robot run to the central axis, wherein the central axis is perpendicular to the line connecting the two wheels in the wheel set.

[0115] In actual use, as long as the robot has a deflection angle, the robot may have a certain offset distance. This offset distance is the distance the robot deviates from the central axis. During the operation of the robot, the distance between the robot and the midpoint between the left and right wheels can be determined based on the midpoint between the left and right wheels. The midpoint between the left and right wheels must be on the central axis. After selecting the midpoint between the left and right wheels as the target, the robot can be controlled to move towards the target. During the movement, it is affected by the moving speed. This moving speed can be a pre-set speed known to the robot. Based on this, the running time of the robot can be calculated, and then the timer is set to control the robot to move to the midpoint between the left and right wheels. After reaching the position, the robot reaches the central axis position, thereby completing the robot's control of the travel route, avoiding deviating too far from the central axis and causing wheel collisions, reducing the failure rate of the robot, and reducing human intervention.

[0116] In a possible implementation, adjusting the offset distance of the robot after the angle adjustment according to the midpoint position of the wheel assembly specifically includes:

[0117] Determine the midpoint between the left and right wheels based on the left and right wheel coordinates:

[0118] x m =(pose L.x +pose R.x ) / 2

[0119] Among them, x m Indicates the midpoint between the left and right wheels in a wheel set;

[0120] If the robot's x-axis coordinate is less than the midpoint between the left and right wheels, it is determined that the robot has excessively large right offset. Calculate the robot's offset distance and correction time:

[0121] d=0.5*(pose R .x-pose L .x)-pose R .x

[0122] v h =V*tan(d / pose R .z)

[0123] t=d / v h

[0124] Where d is the offset distance, V is the preset linear speed of the robot, and v h represents the horizontal speed of the robot, and t represents the correction time;

[0125] Run the robot with an offset distance and a correction duration.

[0126] It can be understood that when the x-axis coordinate of the robot is greater than the midpoint between the left and right wheels, it is determined that the robot has an excessively large left offset, and the offset distance and correction time can be calculated in the same way.

[0127] During actual use, after adjusting the robot, if there is still an angle offset, it is necessary to re-call the steps of calculating the deflection angle of the robot relative to the vehicle to be tested based on the wheel group and adjusting the robot's running angle based on the deflection angle, calculate the deflection angle of the robot after the distance adjustment, adjust the robot's running angle for the second time, and run the robot with the adjusted posture.

[0128] It can be understood that when the robot moves to the midpoint between the left and right wheels, the robot's posture has a deflection angle relative to the central axis position. At this time, the same method is used to calculate the deflection angle at this time, so that the robot's posture is adjusted to be parallel to the central axis to complete the control of the robot.

[0129] It should be noted that after the robot completes the detection, it also includes determining whether it has driven out from under the vehicle. If so, the detection is terminated. Otherwise, the above steps are repeated until the robot completely drives out from under the vehicle.

[0130] It should be noted that if the robot cannot detect the wheels during its driving process, it may have driven out from under the vehicle. At this time, the robot's detection is terminated, thereby avoiding useless detection and achieving more precise control.

[0131] Compared with the prior art, the present invention has at least the following beneficial effects:

[0132] When adjusting the robot, the present invention selects a stable characteristic wheel position. Based on wheel detection and positioning and corresponding wheel point cloud data, the left and right wheel groups are obtained from the identified wheels in different situations, and then calculations and processing are performed according to the situation. This is suitable for handling a variety of complex situations, improving the robot's adaptability and applicability to different operating scenarios. Wheel position information is used to determine whether the robot has deflected and whether it is on the central axis. This allows the robot's movement direction to be adjusted in a timely manner, ensuring that the robot passes under the vehicle to be inspected along the central axis. This ensures the accuracy of the collected data, improves the success rate of under-vehicle inspection, avoids the risk of collision, and improves inspection efficiency and the accuracy of the inspection results.

[0133] like Figure 6 The present invention provides a robot collision avoidance device 20 based on wheel detection and identification, which is applied to a robot.

[0134] See also Figure 6 , showing a structural schematic diagram of a robot collision avoidance device based on wheel detection and identification provided by the present invention.

[0135] like Figure 6 As shown, a robot collision avoidance device 20 based on wheel detection and recognition includes:

[0136] Acquisition module 201: used to collect image data of the vehicle to be tested;

[0137] Detection module 202: used to detect wheels in image data;

[0138] Acquisition module 203: used to acquire wheel point cloud data including distance information, and determine the number and position of wheels;

[0139] Screening module 204: used to screen the wheel set according to the number of wheels and wheel positions;

[0140] Calculation module 205: used to calculate the deflection angle of the robot relative to the vehicle to be tested based on the wheel set;

[0141] The first adjustment module 206 is used to adjust the operating angle of the robot according to the deflection angle;

[0142] The second adjustment module 207 is used to adjust the offset distance of the robot after the angle adjustment according to the midpoint position of the wheel group.

[0143] In one possible implementation, the image data includes true color image data and depth image data, and the distance information is the distance between the camera and each wheel. The determination module 203 specifically includes:

[0144] Input submodule: used to input true color image data into the wheel alignment model to obtain the detection frames of all wheels. The number of detection frames is the number of wheels.

[0145] Traversal sub-step: used to loop through the depth image data corresponding to the detection frame and obtain the depth pixel points corresponding to the detection frame;

[0146] The first setting sub-step is used to set the initial distance threshold and the initial grayscale threshold;

[0147] The first establishment submodule is used to remove noise points in depth pixels based on the initial distance threshold and initial grayscale threshold, and establish wheel point cloud data including distance information:

[0148]

[0149] Where cx, cy, fx, and fy represent internal parameters in true color image data, th1 represents the initial distance threshold, th2 represents the initial grayscale threshold, gray represents grayscale image data, rgb represents true color image data, depth represents depth image data, and i and j represent the column and row coordinates of the pixel, respectively.

[0150] Statistics submodule: used to count the number of points in each wheel point cloud data;

[0151] The first determination submodule is used to determine the wheel position when the number of points in the wheel point cloud data is greater than or equal to a preset number:

[0152]

[0153] Among them, pose c (x, y, z) represents the point cloud centroid, that is, the wheel position, and pointCloud represents the wheel point cloud data.

[0154] In a possible implementation, the determining module 203 further includes:

[0155] Delete submodule: used to delete wheel point cloud data when the number of points in the wheel point cloud data is less than the preset number.

[0156] In one possible implementation, the wheel set includes two matching wheels, and the screening module 204 specifically includes:

[0157] A second setting submodule is used to set a first distance threshold, a second distance threshold, and a third distance threshold according to a general automobile standard, wherein the first distance threshold and the second distance threshold are used to limit the upper and lower limits of the vehicle width, respectively, and the third distance threshold is used to limit the degree of difference between the distance information of the left and right wheels;

[0158] The first judgment submodule is used to determine whether the two wheels are a left and right wheel group in combination with the first distance threshold, the second distance threshold, and the third distance threshold when the number of wheels is 2. The judgment method is as follows:

[0159]

[0160] Among them, d th0 represents the first distance threshold, d th1 represents the second distance threshold, d th2 represents the third distance threshold, pose1.x represents the x-coordinate of the first target wheel, pose2.x represents the x-coordinate of the second target wheel, pose1.z represents the distance information of the first wheel, and pose2.z represents the distance information of the second wheel;

[0161] Arrangement submodule: for arranging the three wheels in ascending order according to the distance information when the number of wheels is 3, to obtain the first target wheel, the second target wheel and the third target wheel;

[0162] A third setting submodule is configured to set the first target wheel and the second target wheel as wheels to be matched if the absolute value of the distance information difference between the first target wheel and the second target wheel is smaller than the absolute value of the distance information difference between the second target wheel and the third target wheel; otherwise, set the second target wheel and the third target wheel as wheels to be matched;

[0163] The second judgment submodule is used to judge whether the wheel to be matched is a wheel set. The judgment method is as follows:

[0164]

[0165]

[0166] Where th represents the detection box area ratio threshold of the wheels to be matched, rect_area represents the detection box area of ​​the wheels to be matched, pose1, pose2, and pose3 represent the first target wheel, the second target wheel, and the third target wheel, respectively.

[0167] The first screening submodule is used to traverse the distance between the wheels in pairs when the number of wheels is 4, and screen the left and right wheel groups based on the distance. The specific screening method is:

[0168]

[0169] Among them, d ij Indicates the separation distance, d th3 represents the fourth distance threshold, d th4 represents the fifth distance threshold;

[0170] The third judgment submodule is used to judge the left wheel and the right wheel in the wheel group when the screened wheel group is a group. The judgment method is as follows:

[0171]

[0172] The second screening submodule is used to screen out the wheel group closest to the camera from the multiple left and right wheel groups using distance information when there are multiple screened wheel groups.

[0173] In a possible implementation, the first calculation module 205 specifically includes:

[0174] The second establishment submodule is used to establish a coordinate system with the line connecting the left and right wheels of the wheel set as the x-axis and the direction perpendicular to the x-axis and consistent with the movement direction of the robot as the y-axis;

[0175] Acquisition submodule: used to obtain the x-coordinate value of the wheel in the wheel group, and the wheel with the larger x-coordinate value is regarded as the right wheel, and the other wheel is regarded as the left wheel;

[0176] The first calculation submodule is used to calculate the deflection angle based on the distance information between the right wheel and the left wheel:

[0177]

[0178] Among them, pose R Indicates the right wheel, pose L represents the left wheel, and ang represents the deflection angle.

[0179] In a possible implementation, the first adjustment module 206 specifically includes:

[0180] When the distance information of the left wheel is greater than the distance information of the right wheel, the robot is rotated to the right; otherwise, the robot is rotated to the left.

[0181] In a possible implementation, the second adjustment module specifically includes:

[0182] The second determination submodule is used to determine the midpoint position between the left and right wheels based on the left wheel coordinates and the right wheel coordinates:

[0183] x m =(pose L.x +pose R.x ) / 2

[0184] Among them, x m Indicates the midpoint between the left and right wheels in a wheel set;

[0185] The third determination submodule is used to determine that the robot has excessively large right offset when the robot's x-axis coordinate is less than the midpoint between the left and right wheels, and calculate the robot's offset distance and correction time.

[0186] d=0.5*(pose R .x-pose L .x)-pose R .x

[0187] v h =V*tan(d / pose R .z)

[0188] t=d / v h

[0189] Where d is the offset distance, V is the preset linear speed of the robot, and v h represents the horizontal speed of the robot, and t represents the correction time;

[0190] Run submodule: used to run the robot with offset distance and correction duration.

[0191] The robot collision avoidance device 20 based on wheel detection and identification provided by the present invention can implement each process implemented in the above method embodiment, and will not be described again here to avoid repetition.

[0192] The virtual system provided by the present invention may be a system, or a component, integrated circuit, or chip in a terminal.

[0193] Compared with the prior art, the present invention has at least the following beneficial effects:

[0194] When adjusting the robot, the present invention selects a stable characteristic wheel position. Based on wheel detection and positioning and corresponding wheel point cloud data, the left and right wheel groups are obtained from the identified wheels in different situations, and then calculations and processing are performed according to the situation. This is suitable for handling a variety of complex situations, improving the robot's adaptability and applicability to different operating scenarios. Wheel position information is used to determine whether the robot has deflected and whether it is on the central axis. This allows the robot's movement direction to be adjusted in a timely manner, ensuring that the robot passes under the vehicle to be inspected along the central axis. This ensures the accuracy of the collected data, improves the success rate of under-vehicle inspection, avoids the risk of collision, and improves inspection efficiency and the accuracy of the inspection results.

[0195] Figure 7 FIG. 1 is a schematic diagram of a terminal device provided by an embodiment of the present invention. Figure 7As shown, a terminal device 30 in this embodiment includes: a processor 300, a memory 301, and a computer program 302 stored in the memory 301 and executable on the processor 300, such as a program for a robot collision avoidance method based on wheel detection and recognition. When the processor 300 executes the computer program 302, the steps of the aforementioned robot collision avoidance method based on wheel detection and recognition are implemented. Alternatively, when the processor 300 executes the computer program 302, the functions of the various units in the aforementioned device embodiments are implemented.

[0196] Exemplarily, the computer program 302 may be divided into one or more units, one or more of which are stored in the memory 301 and executed by the processor 300 to implement the present invention. The one or more units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 302 in a terminal device 30. For example, the computer program 302 may be divided into modules with the following specific functions:

[0197] Acquisition module: used to collect image data of the vehicle to be tested;

[0198] Detection module: used to detect wheels in image data;

[0199] Acquisition module: used to obtain wheel point cloud data including distance information, and determine the number and position of wheels;

[0200] Screening module: used to screen wheel sets according to wheel quantity and wheel position;

[0201] Calculation module: used to calculate the deflection angle of the robot relative to the vehicle under test based on the wheel set;

[0202] The first adjustment module is used to adjust the robot's operating angle according to the deflection angle;

[0203] The second adjustment module is used to adjust the offset distance of the robot after the angle adjustment according to the midpoint position of the wheel group.

[0204] The terminal device includes but is not limited to a processor 300 and a memory 301. Those skilled in the art will understand that Figure 7 It is only an example of a terminal device 30 and does not constitute a limitation on the terminal device 30. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the robot may also include input and output devices, network access devices, buses, etc.

[0205] The processor 300 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0206] The memory 301 may be an internal storage unit of the terminal device 30, such as a hard disk or memory of the terminal device 30. The memory 301 may also be an external storage device of the terminal device 30, such as a plug-in hard disk, a smart memory card (SmartMediaCard, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the terminal device 30. Furthermore, the memory 301 may also include both an internal storage unit of the terminal device 30 and an external storage device. The memory 301 is used to store the computer program and other programs and data required by the roaming control device. The memory 301 may also be used to temporarily store data that has been output or is to be output.

[0207] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0208] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the embodiment of the method of the present invention. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0209] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0210] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned various method embodiments can be implemented.

[0211] An embodiment of the present invention provides a computer program product. When the computer program product is run on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0212] If the integrated unit is implemented as a software functional unit and sold or used as a standalone product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the process steps in the above-mentioned method embodiments by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium can at least include: any entity or device capable of carrying the computer program code to the camera / robot, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. Examples include USB flash drives, removable hard drives, magnetic disks, or optical disks. In some jurisdictions, based on legislation and patent practice, computer-readable media cannot be electric carrier signals or telecommunication signals.

[0213] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0214] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0215] In the embodiments provided by the present invention, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0216] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, and may be located in one place or distributed across multiple network units.

[0217] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0218] It will also be understood that the term "and / or" used in the present description and appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0219] As used in the present specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to monitoring," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is monitored" may be interpreted as meaning "upon determination" or "in response to determining" or "upon monitoring [described condition or event]" or "in response to monitoring [described condition or event]," depending on the context.

[0220] In addition, in the description of the present specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0221] References to "one embodiment" or "some embodiments" in the present specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present invention. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0222] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A robot collision avoidance method based on wheel detection and recognition, applied to a robot, characterized in that: The robot is provided with a camera in front of the robot, and the method includes: Collect image data of the vehicle to be tested; detecting a wheel in the image data; Obtain wheel point cloud data including distance information to determine the number and position of wheels; Filtering the wheel set according to the number of wheels and the wheel positions; Calculating a deflection angle of the robot relative to the vehicle to be tested based on the wheel set; Adjusting the operating angle of the robot according to the deflection angle; Adjusting the offset distance of the robot after the angle adjustment according to the midpoint position of the wheel group; The wheel set includes two matching wheels, the number of wheels and the wheel positions, and screening the wheel set specifically includes: According to the general automobile standard, a first distance threshold, a second distance threshold, and a third distance threshold are set, wherein the first distance threshold and the second distance threshold are used to limit the upper and lower limits of the vehicle width, respectively, and the third distance threshold is used to limit the degree of difference between the distance information of the left and right wheels; When the number of wheels is 2, the first distance threshold, the second distance threshold, and the third distance threshold are combined to determine whether the two wheels form a left and right wheel group. The determination method is specifically as follows: Among them, d th0 represents the first distance threshold, d th1 represents the second distance threshold, d th2 represents the third distance threshold, pose1.x represents the x-coordinate of the first target wheel, pose2.x represents the x-coordinate of the second target wheel, pose1.z represents the distance information of the first wheel, and pose2.z represents the distance information of the second wheel; When the number of wheels is 3, the three wheels are arranged in ascending order according to the distance information to obtain a first target wheel, a second target wheel, and a third target wheel; When the absolute value of the distance information difference between the first target wheel and the second target wheel is less than the absolute value of the distance information difference between the second target wheel and the third target wheel, the first target wheel and the second target wheel are set as the wheels to be matched; otherwise, the second target wheel and the third target wheel are set as the wheels to be matched; Determine whether the wheel to be matched is the wheel set, specifically by: Wherein, th represents the detection box area ratio threshold of the wheels to be matched, rect_area represents the detection box area of ​​the wheels to be matched, pose1, pose2, and pose3 represent the first target wheel, the second target wheel, and the third target wheel, respectively; When the number of wheels is 4, the intervals between the wheels are traversed two by two, and the left and right wheel groups are selected according to the intervals. The specific selection method is: Among them, d ij Denotes the separation distance, d th3 represents the fourth distance threshold, d th4 represents the fifth distance threshold; In the case where the screened wheel groups are one group, the left wheel and the right wheel in the wheel group are determined by the following method: In the case that there are multiple wheel groups screened out, the wheel group closest to the camera is screened out from the multiple left and right wheel groups using the distance information.

2. The robot collision avoidance method based on wheel detection and recognition according to claim 1, wherein the image data includes true color image data and depth image data, and the distance information is the distance between the camera and each of the wheels, The obtaining of wheel point cloud data including distance information and determining the number and position of wheels specifically includes: Inputting the true color image data into a wheel alignment model to obtain detection frames of all wheels, where the number of the detection frames is the number of wheels; Loop through the depth image data corresponding to the detection frame to obtain the depth pixel points corresponding to the detection frame; Set the initial distance threshold and initial grayscale threshold; According to the initial distance threshold and the initial grayscale threshold, noise points in the depth pixel points are removed to establish the wheel point cloud data including distance information: Wherein, cx, cy, fx, and fy represent internal parameters in the true color image data, th1 represents the initial distance threshold, th2 represents the initial grayscale threshold, gray represents grayscale image data, rgb represents the true color image data, depth represents the depth image data, and i and j represent the column coordinates and row coordinates of the pixel respectively; Counting the number of points in each wheel point cloud data; When the number of points in the wheel point cloud data is greater than or equal to a preset number, determining the wheel position: Among them, pose c (x, y, z) represents the centroid of the point cloud, that is, the wheel position, and pointCloud represents the wheel point cloud data.

3. The robot collision avoidance method based on wheel detection and recognition according to claim 2, characterized in that: The obtaining of wheel point cloud data including distance information and determining the number and position of wheels further comprises: When the number of points in the wheel point cloud data is less than the preset number, the wheel point cloud data is deleted.

4. The robot collision avoidance method based on wheel detection and recognition according to claim 1, characterized in that: Calculating the deflection angle of the robot relative to the vehicle to be tested according to the wheel set specifically includes: A coordinate system is established with a line connecting the left and right wheels of the wheel set as an x-axis and a direction perpendicular to the x-axis and consistent with the movement direction of the robot as a y-axis; Obtaining x-coordinate values ​​of the wheels in the wheel set, with the wheel with the larger x-coordinate value being the right wheel and the other wheel being the left wheel; The deflection angle is calculated according to the distance information between the right wheel and the left wheel: Among them, pose R Indicates the right wheel, pose L represents the left wheel, and ang represents the deflection angle.

5. The robot collision avoidance method based on wheel detection and recognition according to claim 4, characterized in that: According to the deflection angle, the operation angle of the robot is adjusted specifically as follows: When the distance information of the left wheel is greater than the distance information of the right wheel, the robot is rotated to the right; otherwise, the robot is rotated to the left.

6. The robot collision avoidance method based on wheel detection and recognition according to claim 1, characterized in that: Adjusting the offset distance of the vehicle bottom robot after the angle adjustment according to the midpoint position of the wheel group specifically includes: Determine the midpoint between the left and right wheels based on the left wheel coordinates and the right wheel coordinates: x m =(pose L.x +pose R.x ) / 2 Among them, x m Indicates the midpoint between the left and right wheels in the wheel set; When the x-axis coordinate of the robot is less than the midpoint between the left and right wheels, it is determined that the robot has excessively deviated to the right, and the robot's offset distance and correction time are calculated: d=0.5*(pose R .x-pose L .x)-pose R .x v h =V*tan(d / pose R .z) t=d / v h Wherein, d represents the offset distance, V represents the preset linear speed of the robot, and v h represents the horizontal speed of the robot, and t represents the correction time; The robot is operated with the offset distance and the correction duration.

7. A robot collision avoidance device based on wheel detection and recognition, adopting the robot collision avoidance method based on wheel detection and recognition according to any one of claims 1 to 6, characterized in that: include: Acquisition module: used to collect image data of the vehicle to be tested; Detection module: used for detecting wheels in the image data; Acquisition module: used to obtain wheel point cloud data including distance information, and determine the number and position of wheels; Screening module: used for screening the wheel set according to the number of wheels and the wheel positions; A calculation module is used to calculate the deflection angle of the robot relative to the vehicle to be tested based on the wheel set; A first adjustment module is used to adjust the operating angle of the robot according to the deflection angle; The second adjustment module is used to adjust the offset distance of the robot after the angle adjustment according to the midpoint position of the wheel group.

8. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the robot collision avoidance method based on wheel detection and identification according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the robot collision avoidance method based on wheel detection and identification according to any one of claims 1 to 6 are implemented.

Images