GAZEBO modeling method in ROS based on image recognition

Abstract

The invention discloses a GAZEBO modeling method in an ROS based on image recognition. Comprising the following steps of image preprocessing, denoising, detection, wall marking and file conversion. According to the obtained data, including recognized target object coordinates, linear body coordinates, center point coordinates, length and angle, generating an xml description file of GAZEBO, and automatically generating different simulation models according to different characteristics by using the data through a python script. According to the method, the image recognition accuracy is high, the adaptability is high, the wall model is automatically generated, the modeling process is simplified, and automatic conversion from image recognition to simulation model modeling is achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of autonomous driving, and specifically relates to a modeling method of GAZEBO in ROS based on image recognition. Background Art

[0002] With the rapid development of fields such as intelligent manufacturing and service robots, robots need to be able to operate more precisely in a simulated environment, especially in application scenarios such as unmanned vehicle testing and robot navigation. Currently, the environments required for modeling are mostly hand-drawn or designed using professional software. This method is not only time-consuming and laborious, but also prone to errors. At the same time, hand-drawn models are often insufficient to reflect the complexity of the real environment. Especially for complex parking lot environments, there are many difficulties in hand modeling. The main problems existing in the existing modeling methods are cumbersome processes, low accuracy, and difficulty in reflecting the complexity of the real environment. Summary of the Invention

[0004] The purpose of the present invention is to provide a modeling method of GAZEBO in ROS based on image recognition, which has high accuracy of image recognition, strong adaptability, can automatically generate a wall model, simplifies the modeling process, and realizes the automatic conversion and modeling from image recognition to a simulation model.

[0005] A modeling method of GAZEBO in ROS based on image recognition according to the present invention is characterized in that it includes the following steps: 1) Image preprocessing; Select the original picture required for modeling for image preprocessing, and convert the image into a grayscale image through the cv2.cvtColor() function to obtain a black-and-white original image; 2) Denoising; Adopt a window smoothing algorithm for the above-obtained original image, and remove the noise points in the image through the cv2.blur() mean filtering function, and select the target image from the obtained original image; 3) Detection; Perform sliding window detection on the target image over the original image. Set the sliding window step size as required. Calculate the grayscale histograms of the images intercepted by the sliding window and the target image using the cv2.calcHist() function, denoted as H1 and H2 respectively. Set the score as degree. If H1[i] is equal to H2[i], then degree is incremented by 1. If H1[i] is not equal to H2[i], then degree = degree + (1 - abs(H1[i] - H2[i]) / max(H1[i], H2[i])). After traversing the entire grayscale histogram, obtain the final score d = degree / len(H1). Set a threshold of 0.8. When the score d is greater than 0.8, it is considered that the image captured by the sliding window has more than 80% overlap with the target image, and it is regarded as detecting the target image, and record the coordinates of the target image; 4) Wall marking; Generate walls by manually selecting points or automatically identify the endpoints of walls through image recognition methods to generate walls; 5) File conversion; Based on the data obtained above, including the coordinates of the identified target objects, straight line coordinates, center point coordinates, length, and angle, generate the xml description file of GAZEBO, and automatically generate different simulation models according to different characteristics of the above data through a python script.

[0006] The method of generating walls by manually selecting points in step 4) is for straight lines. Manually select both ends of the straight line through the cv2.setMouseCallback() function to obtain the X value a and the Y value b. Calculate the length of the straight line dist = math.sqrt((a[i + 1] - a[i]) ** 2 + (b[i + 1] - b[i]) ** 2) * 0.094 and the angle ang = math.atan((a[i + 1] - a[i]) / b[i + 1] - b[i]) relative to the origin of the image, where 0.094 is the actual length corresponding to one pixel point in the image, which is 0.094 meters. Obtain the angle of the rectangle's position in the same way. Finally, calculate the center coordinates of the straight line and the rectangle, and save the above data.

[0007] Compared with the prior art, the present invention has obvious beneficial effects. From the above technical solutions, it can be seen that: the present invention first uses image processing technology to preprocess the two-dimensional plan of the parking lot, sets a threshold by using the window smoothing function to identify the rectangles in the figure, and these rectangles represent the load-bearing columns and the lane lines of the parking spaces. After the recognition is completed, the user can mark two points, generate by the manual selection method or automatically identify the wall endpoints through the machine learning method to generate the wall, so as to complete the recognition of the entire parking lot map. Finally, by writing a script to convert the recognition result into the description file format of GAZEBO and importing it into the GAZEBO environment, the high accuracy of image recognition, strong adaptability, automatic generation of wall models, simplification of the modeling process, and automatic conversion and modeling from image recognition to simulation models are realized. Detailed implementation mode

[0008] Example 1: A GAZEBO modeling method in ROS based on image recognition according to the present invention is characterized in that; it includes the following steps: 1) Image preprocessing; Select the original image to be modeled for image preprocessing, convert the image to a grayscale image through the cv2.cvtColor() function to obtain a black-and-white original image; 2) Denoising; Adopt the window smoothing algorithm for the above-mentioned original image, and remove the noise points in the image through the cv2.blur() mean filtering function, and select the target image from the obtained original image; 3) Detection; Perform sliding window detection on the target image on the original image. According to the required set sliding window step size, calculate the grayscale histograms of the images intercepted by the sliding window and the target image through the cv2.calcHist() function, which are recorded as H1 and H2 respectively. Set the score as degree. If H1[i] is equal to H2[i], then degree is incremented by 1. If H1[i] is not equal to H2[i], then degree = degree + (1 - abs(H1[i] - H2[i]) / max(H1[i], H2[i])). After traversing the entire grayscale histogram, the final score d = degree / len(H1) is obtained. Set a threshold of 0.8. When the score d is greater than 0.8, it is considered that the image taken by the sliding window coincides with the target image by more than 80%, which is regarded as detecting the target image, and record the coordinates of the target image.

[0009] 4) Wall marking; For a straight line object, manually select both ends of the straight line object through the cv2.setMouseCallback() function to obtain the X value a and the Y value b. Calculate the length of the straight line object dist = math.sqrt((a[i + 1]- a[i]) ** 2 + (b[i+ 1]- b[i]) ** 2) * 0.094 and the angle ang = math.atan((a[i + 1]- a[i]) / b[i+1]-b[i]) relative to the origin of the image. Here, 0.094 represents that one pixel in the image corresponds to an actual length of 0.094 meters. For the angle of the rectangle's position, obtain it in the same way. Finally, calculate the center coordinates of the straight line object and the rectangle object, and save the above data.

[0010] 5) File conversion; According to the data obtained above, including the coordinates of the recognized target object, the coordinates of the straight line object, the center point coordinates, the length, and the angle, generate the xml description file of GAZEBO, and automatically generate different simulation models according to different characteristics of the above data through a python script.

[0011] Example 2: A GAZEBO modeling method in ROS based on image recognition according to the present invention is characterized in that it includes the following steps: 1) Image preprocessing; Select the original image for image preprocessing, and convert the image into a grayscale image through the cv2.cvtColor() function to obtain a black and white original image; 2) Denoising; Apply the window smoothing algorithm to the obtained original image, and remove the noise points in the image through the cv2.blur() mean filtering function. Select the target image from the obtained original image; 3) Detection; Perform sliding window detection on the target image on the original image. According to the required set sliding window step size, calculate the grayscale histograms of the image intercepted by the sliding window and the target image through the cv2.calcHist() function, denoted as H1 and H2 respectively. Set the score as degree. If H1[i] is equal to H2[i], then degree is incremented by 1. If H1[i] is not equal to H2[i], then degree = degree + (1 - abs(H1[i]- H2[i]) / max(H1[i], H2[i])). After traversing the entire grayscale histogram, obtain the final score d = degree / len(H1). Set a threshold of 0.8. When the score d is greater than 0.8, it is considered that the image obtained by the sliding window has more than 80% overlap with the target image, and it is regarded as detecting the target image, and record the target image coordinates.

[0012] 4) Wall marking; For straight objects, the wall endpoints are automatically recognized through image recognition methods to generate wall data.

[0013] 5) File conversion; Based on the data obtained above, including the coordinates of the target object, straight object coordinates, center point coordinates, length, and angle, generate the xml description file of GAZEBO, and automatically generate different simulation models according to different characteristics through a python script, as follows: Column description part: f' <model name="pillar_{j}">'\n' f' <pose>{x} {y} 0 0 0 0< / pose> '\n' ' <include>'\n' ' ' <static>false< / static> '\n' ' ' <uri> / MyModel< / uri> '\n' '< / include> '\n' '< / model> \n' Wall description part: <link name="Walli_{i}"> \n' f' <pose>{x} {y} 2 0 -0 {ang}< / pose> \n' ' <self_collide>0< / self_collide>\n' ' <enable_wind>0< / enable_wind>\n' ' <kinematic> 0< / kinematic> \n' ' <inertial>\n' ' <mass> 0.01< / mass> \n' ' <inertia>\n' ' <ixx> 0.166667< / ixx> \n' ' <ixy> 0< / ixy> \n' ' <ixz> 0< / ixz> \n' ' <iyy> 0.166667< / iyy> \n' ' <iyz> 0< / iyz> \n' ' <izz> 0.166667< / izz> \n' '< / inertia> \n' ' <pose> 0 0 0 0 -0 0< / pose> \n' '< / inertial> \n' ' <gravity> 1< / gravity> \n' ' <visual name="Wall_0_Visual">'\n' ' <pose> 0 0 0 0 0 0< / pose> '\n' ' <geometry>'\n' ' ' <box>\n' f' <size>{dist} 0.4 4.0< / size> \n' '< / box> '\n' '< / geometry> '\n' ' <material>\n' ' <script>\n'' <uri>file: / / media / materials / scripts / gazebo.material< / uri>\n'' <name>Gazebo / Grey< / name>\n'' < / script> \n' ' <ambient> 1 1 1 1< / ambient> \n' ' <shader type="pixel" / > \n' '< / material> '\n' ' <meta> '\n' ' <layer> 0< / layer> '\n' ' \n' ' <transparency> 0< / transparency> '\n' ' <cast_shadows>1< / cast_shadows>\n' '< / visual> \n' ' <collision name="Wall_0_Collision">' ' <laser_retro>0< / laser_retro> ' ' <max_contacts>10< / max_contacts> ' ' <pose> 0 0 0 0 0 0< / pose> ' ' <geometry>'\n' ' <box>\n' f' <size>{dist} 0.4 4.0< / size> \n' '< / box> \n' '< / geometry> ' ' <surface>'\n' ' <friction>'\n' ' <ode>\n' ' <mu> 1< / mu> \n' ' <mu2> 1< / mu2> \n' ' <fdir1> 0 0 0< / fdir1> \n' ' <slip1> 0< / slip1> \n' ' <slip2> 0< / slip2> \n' '< / ode> '\n' ' <torsional>\n' ' <coefficient> 1< / coefficient> \n' ' <patch_radius>0< / patch_radius>\n' ' <surface_radius>0< / surface_radius>\n' ' <use_patch_radius>1< / use_patch_radius>\n' ' <ode>\n' ' <slip> 0< / slip> \n' '< / ode> \n' '< / torsional> '\n' '< / friction> '\n' ' <bounce>'\n' ' <restitution_coefficient>0< / restitution_coefficient>\n' ' <threshold> 1e+06< / threshold> \n' '< / bounce> '\n' ' <contact>' ' <collide_without_contact>0< / collide_without_contact> ' ' <collide_without_contact_bitmask>1< / collide_without_contact_bitmask> ' ' <collide_bitmask>1< / collide_bitmask> ' ' <ode>' ' <soft_cfm>0< / soft_cfm> ' ' <soft_erp>0.2< / soft_erp> ' ' <kp> 1e+13< / kp> ' ' <kd> 1< / kd> ' ' <max_vel>0.01< / max_vel> ' ' <min_depth>0.01< / min_depth> ' '< / ode> ' ' <bullet>'\n' ' <split_impulse>1< / split_impulse>\n' ' <split_impulse_penetration_threshold>-0.01< / split_impulse_penetration_threshold>\n' ' <soft_cfm>0< / soft_cfm>\n' ' <soft_erp>0.2< / soft_erp>\n' ' <kp> 1e+13< / kp> \n' ' <kd> 1< / kd> \n' '< / bullet> ' '< / contact> '\n' '< / surface> ' '< / collision> \n' '\n').

[0014] The above is only a preferred embodiment of the present invention, and it does not impose any form of limitation on the present invention. Any simple modification, equivalent change, and modification made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. A GAZEBO modeling method in ROS based on image recognition, characterized in that; It includes the following steps: 1) Image preprocessing; Select the required original image for modeling and perform image preprocessing. Convert the image to a grayscale image through the cv2.cvtColor() function to obtain a black-and-white original image; 2) Denoising; Apply the window smoothing algorithm to the above-mentioned original image and use the cv2.blur() mean filtering function to remove the noise points in the image. Select the target image from the obtained original image; 3) Detection; Perform sliding window detection on the target image on the original image. Set the sliding window step size as required. Calculate the grayscale histograms of the images intercepted by the sliding window and the target image through the cv2.calcHist() function, denoted as H1 and H2 respectively. Set the score as degree. If H1[i] is equal to H2[i], then degree is incremented by 1. If H1[i] is not equal to H2[i], then degree = degree + (1 - abs(H1[i] - H2[i]) / max(H1[i], H2[i])). After traversing the entire grayscale histogram, obtain the final score d = degree / len(H1). Set a threshold of 0.

8. When the score d is greater than 0.8, it is considered that the image captured by the sliding window has more than 80% overlap with the target image, which is regarded as detecting the target image, and record the target image coordinates; 4) Wall marking; Generate walls by manual selection or automatically identify wall endpoints through image recognition methods to generate walls; 5) File conversion; According to the data obtained above, including the coordinates of the target object, the coordinates of the straight line body, the center point coordinates, the length and the angle, generate the xml description file of GAZEBO, and automatically generate different simulation models according to different characteristics of the above data through a python script.

2. The modeling method of GAZEBO in ROS based on image recognition according to claim 1, characterized in that; The method of generating walls by manual selection in step 4) is for the straight line body. Manually select both ends of the straight line body through the cv2.setMouseCallback() function to obtain the X value a and the Y value b. Calculate the length of the straight line body dist = math.sqrt((a[i + 1] - a[i]) ** 2 + (b[i + 1] - b[i]) ** 2) * 0.094 and the angle ang = math.atan((a[i + 1] - a[i]) / b[i + 1] - b[i]) relative to the origin of the image, where 0.094 is the length corresponding to one pixel point in the image in reality, which is 0.094 meters. Obtain the angle of the rectangle in the same way. Finally, calculate the center coordinates of the straight line body and the rectangle body, and save the above data.