Teaching method and device of train detection robot and storage medium

Abstract

The invention discloses a train detection robot teaching method and device and a storage medium, and relates to the technical field of train detection. Comprising the following steps: step 101, acquiring first operation information of a user, and generating detection item information; step 102, acquiring a first image shot by a camera on the robot, and displaying the first image; 103, when the first image does not meet the first preset condition, executing the step 102 until the first image meets the first preset condition; otherwise, executing the step 104; step 104, acquiring second operation information of the user, and acquiring photographing point information of the robot; correspondingly storing the detection item information and the photographing point information to obtain detection item teaching information of the detection item; step 105, when the user inputs the next detection item, executing the steps 101-104; and step 106, acquiring third operation information of the user, and generating a teaching file. The method is used for solving the technical problems that an existing teaching method needs long time and is tedious in process.

Description

Technical Field

[0001] The present invention relates to the technical field of train detection, and in particular to a teaching method, a device and a storage medium of a train detection robot. Background Art

[0002] The current intelligent development trend is to use robots to replace people to inspect train defects in the rail transit industry. The robot is equipped with a video or photographic device, which takes pictures of the inspection parts, and then analyzes and processes the captured images through image processing software to obtain the inspection results. Robots can refer to drones, unmanned vehicles, etc. In this process, the robot must first be taught. The teaching process is the process of robot learning. The operator must teach the robot to do certain actions step by step. During this process, the control system memorizes the taught actions in the form of a program. After the teaching is completed, the program is input to the robot so that it can perform these actions according to the program memorized during the teaching.

[0003] In the prior art, teaching software and image viewing software are required for teaching. The teaching software is used to record the posture information of the robot when the robot operator controls the robot to pose and take pictures of the inspection parts of the train. The image viewing software is used to view the images captured by the robot. The image processing software is used to process the image and obtain the detection results. The specific process is: 1. The robot operator teaches the robot posture, that is, moves the robot to the inspection position, adjusts the robot's posture, the camera's shooting angle, and then takes the image. The teaching software records the robot's position information, posture information and camera's shooting angle when the image is taken. Then the robot operator moves the robot to the next inspection position and continues to shoot until all inspection positions are shot. 2. The image is packaged with the image algorithm processing personnel. The image algorithm processing personnel use the image viewing software to view the image, determine whether the image meets the image algorithm processing conditions, and organize the judgment results into a document and send it to the train inspection personnel for confirmation. 3. The train inspection personnel further confirm whether the image meets the requirements by observing whether the image contains detection points. If the image algorithm processor and train inspection personnel believe that all images meet the requirements, the teaching information recorded by the teaching software can be used. If some images do not meet the requirements, the robot operator needs to re-shoot and re-record the robot's position information, posture information and camera shooting angle. This can be repeated several times to complete the teaching.

[0004] In this case, firstly, the robot operator needs to be familiar with the train inspection procedures in order to know the best shooting position and shooting angle, which places high demands on the robot operator. Secondly, the teaching process often needs to be repeated several times to complete, and each teaching process requires the completion of the teaching shooting, image algorithm processing personnel confirmation, and train inspection personnel confirmation. The teaching process takes a long time and is cumbersome. Summary of the invention

[0005] The purpose of the present invention is to provide a teaching method, device and storage medium for a train inspection robot, which can enable robot operators, image algorithm personnel and train inspection personnel to carry out teaching work together without the process of repeated confirmation and modification, thereby improving the efficiency and accuracy of the teaching work.

[0006] In order to achieve the above object, the present invention adopts the following technical scheme:

[0007] The present invention provides a teaching method for a train detection robot, comprising:

[0008] Step 101, obtaining first operation information of a user for inputting a detection item, and generating detection item information;

[0009] Step 102, obtaining a first image taken by a camera on the robot, and displaying the first image; so that a user can determine whether the first image meets a first preset condition, wherein the first preset condition includes that the first image contains a detection item point corresponding to the detection item and that the first image meets an image algorithm processing condition;

[0010] Step 103, when the first image does not meet the first preset condition, the robot changes its position and / or the camera changes its posture to obtain a new first image, executing step 102 until the first image meets the first preset condition; when the first image meets the first preset condition, executing step 104;

[0011] Step 104, obtaining second operation information of the user for confirming and saving the photo point, obtaining the photo point information of the robot, the photo point information including the posture information of the robot and the posture information of the camera when the camera captures the first image that meets the first preset condition; and correspondingly saving the detection item information and the photo point information to obtain the detection item teaching information of the detection item;

[0012] Step 105, when the user inputs the next test item, execute steps 101-104;

[0013] Step 106 , obtaining third operation information input by the user for generating a teaching file, and generating a teaching file, wherein the teaching file includes detection item teaching information corresponding to each detection item.

[0014] Optionally, before step 104, step 107 is further included, obtaining fourth operation information of the user for marking on the first image, and generating marking information;

[0015] The step 104 of correspondingly saving the detection item information and the photographing point information to obtain the detection item teaching information of the detection item specifically includes:

[0016] The detection item information, the photographing point information and the marking information are correspondingly saved to obtain the detection item teaching information of the detection item.

[0017] Optionally, before step 106, the following steps may also be performed:

[0018] When the user moves the robot from one photo taking point to the next photo taking point, obtaining waypoint information of the robot, wherein the waypoint information includes position information of several position points of the robot;

[0019] The waypoint information and the photo taking point information are saved in correspondence;

[0020] The teaching file generated in step 106 includes the waypoint information.

[0021] Optionally, before step 101, step 108 is further included, obtaining a train inspection procedure, generating and displaying an inspection item library based on the inspection procedure, wherein the inspection item library includes inspection items for selection by a user;

[0022] The step 101 specifically includes: obtaining first operation information of a detection item selected by a user from the detection item library, and generating detection item information based on the detection item selected by the user.

[0023] Optionally, in step 102, obtaining a first image captured by a camera on the robot and displaying the first image specifically includes:

[0024] Acquire fifth operation information of the user for real-time display, acquire a framing image of the camera during the framing process of the camera on the robot, and display the framing image in real time;

[0025] Acquire sixth operation information of a shooting button clicked by a user, and control the camera to shoot to obtain the first image;

[0026] The first image is acquired and displayed.

[0027] Optionally, in step 102, obtaining a first image captured by a camera on the robot and displaying the first image specifically includes:

[0028] Acquire sixth operation information of a shooting button clicked by a user, and control the camera to shoot to obtain the first image;

[0029] The first image is acquired and displayed.

[0030] Optionally, before step 104, the following steps may also be included:

[0031] The seventh pre-processed operation information selected by the user is obtained, and the first image is pre-processed to obtain a pre-processed image; the pre-processed image is used to assist the user in determining whether the first image meets the first preset condition.

[0032] The present invention also provides a teaching method for a train detection robot, comprising:

[0033] Step 201, the teaching software operator opens the teaching software on the teaching device and establishes a communication connection with the robot;

[0034] Step 202, the teaching software operator inputs the test items on the teaching software;

[0035] Step 203, the robot operator moves the robot to a detection position and adjusts the position of the robot and the posture of the camera;

[0036] Step 204: the robot operator controls the camera to take a picture, or the teaching software operator controls the camera to take a picture, to obtain a first image;

[0037] Step 205, the teaching software acquires the first image and displays it;

[0038] Step 206, the teaching software operator controls the teaching software to pre-process the first image to obtain a pre-processed image;

[0039] Step 207, the teaching software operator determines whether the first image meets the requirements based on the first image and the pre-processed image;

[0040] Step 208, if the requirements are not met, the robot operator is notified to adjust the robot's posture and the camera's posture, and then steps 204-206 are executed repeatedly until the first image meets the requirements, and the shooting position of the robot taking the first image that meets the requirements is determined as the shooting point of the detection item;

[0041] Step 209: If the requirements are met, the teaching software operator annotates the first image and generates annotation information;

[0042] Step 210, the teaching software operator controls the teaching software to obtain the photographing point information corresponding to the first image, and saves the photographing point information, the detection item information and the marking information accordingly, so as to obtain the detection item teaching information of the detection item;

[0043] Step 211, the operator of the teaching software inputs the next test item, and then loops through steps 202-210;

[0044] Step 212, looping through step 211 until all detection items have been taught;

[0045] Step 213, the robot operator controls the robot to move along a preset route between the two photo taking points, selects a number of position points in the preset route as waypoints, and notifies the teaching personnel to obtain and record waypoint information of each waypoint, where the waypoint information includes the position information of the waypoint;

[0046] Step 214, the software operator controls the teaching software to record the photographing point information and the waypoint information in correspondence;

[0047] Step 215 , the software operator controls the teaching software to generate a teaching file, wherein the teaching file includes teaching information corresponding to each detection item and waypoint information.

[0048] The present invention also provides a computer device, comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of any one of the methods described above.

[0049] The present invention also provides a computer-readable storage medium having a computer program / instruction stored thereon, wherein the computer program / instruction, when executed by a processor, implements the steps of any one of the methods described above.

[0050] In the technical solution of the present invention, after the robot captures the first image, it can be sent to the teaching software for display in time for the image processing personnel and the train inspection personnel to confirm. When the first image does not meet the requirements, it can be adjusted with the robot operator in time to obtain an image that meets the first requirement and obtain a photo point that meets the requirements. This allows the robot operator, image processing personnel and train inspection personnel to teach at the same time. When teaching each detection item, the robot operator can be notified in time to make adjustments, so as to obtain an accurate and better photo point. It is no longer like before, first the robot operator takes photos of all detection items, and records the corresponding photo point information through the teaching software; then the photos are sent to the image algorithm processing personnel, the image algorithm processing personnel opens the image viewing software for viewing, and after judgment, they are sent to the train inspection personnel for judgment. The three steps cannot be performed at the same time, and one step needs to be completed before the next step can be performed. In this way, these three steps need to be repeated every time a modification is required. In this way, when using this method for teaching, it takes less time, is more efficient, and has a higher accuracy rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A schematic diagram of a flow chart of a teaching method for a train inspection robot;

[0052] Figure 2 A schematic diagram of the interface of the teaching software before acquiring the first image;

[0053] Figure 3 A schematic diagram of an interface of a teaching software displaying a first image and annotating the image;

[0054] Figure 4 A schematic diagram of an interface for displaying a preprocessed image after the teaching software preprocesses the first image;

[0055] Figure 5 It is a schematic diagram represented by EXCEL in the generated teaching file;

[0056] Figure 6 This is a schematic diagram of the annotation information in the generated teaching file represented in .jason format. DETAILED DESCRIPTION

[0057] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are only examples of devices or methods consistent with some aspects of the present invention.

[0058] To make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described in detail below. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other implementation methods obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0059] The following is an explanation of the embodiments with reference to the accompanying drawings. In addition, the embodiments shown below do not limit the invention described in the claims. In addition, the entire contents of the configurations shown in the following embodiments are not limited to the solutions required as the invention described in the claims.

[0060] See also Figure 1 The present invention provides a teaching method for a train inspection robot. From a program perspective, the execution subject of the process can be a program installed on a server or a teaching device; from a hardware perspective, the execution subject of the process can be a teaching device, specifically a computer, a control center, etc. Figure 1 As shown, the method may include the following steps:

[0061] Step 101: obtaining first operation information of a user for inputting a detection item, and generating detection item information.

[0062] Open the teaching software on the teaching device and establish a communication connection with the robot to be taught. The user enters the test items to be taught on the teaching software interface. There is no limitation on the input method of the test items, which can be manual typing input, voice input, etc. Alternatively, the teaching software interface may have a pre-generated test item library, see Figure 2 , one form of display of the detection item library can be Figure 2 The two columns of option lists on the right side of the diagram. The user selects the test item that needs to be taught in the option list. For example, if the test item is the bolts on the bogie, then click the bogie and bolt options. After obtaining the user's first operation information, the teaching software responds to the first operation information, generates the test item information corresponding to the test item, and records and saves it.

[0063] The user here refers to the software operator, who may be an image algorithm processor or a train detection operator, or other personnel with relevant knowledge.

[0064] Step 102, obtaining a first image taken by a camera on the robot, and displaying the first image; so that a user can determine whether the first image meets a first preset condition, wherein the first preset condition includes that the first image contains a detection item point corresponding to the detection item and that the first image satisfies an image algorithm processing condition.

[0065] The robot operator moves the robot to the shooting area corresponding to the detection item. For example, if the detection item is the bolts on the bogie on the left side of car 1, the robot is moved to the shooting area where the bolts on the bogie can be photographed. Then the robot position (which can be represented by three-dimensional coordinates) and posture (which can be represented by the robot's yaw angle) as well as the camera posture (camera angle) are adjusted more accurately, so that the camera can shoot a certain part of the train corresponding to the detection item (in this case, the bogie) to obtain the first image. The first image is transmitted to the teaching software. After the teaching software obtains the first image, it is displayed in the corresponding area of ​​the teaching software interface. See Figure 3 The first image is displayed in the image display area. Specifically, the first image can be displayed after the user clicks the photo point setting display button. When the photo point setting display button is clicked again, the newly acquired first image will be displayed.

[0066] At this time, the user judges whether the first image meets the first preset condition by viewing the first image. The first preset condition includes that the first image contains the detection item points corresponding to the detection item and the first image satisfies the image algorithm processing condition. Specifically, it can be: the image algorithm processing personnel judge whether the first image can be processed by the image processing software based on experience. The basis for the judgment can be: whether the first image is too dark or too bright, or whether the shooting angle of the first image is convenient for the image processing software to process, etc. The train inspection personnel observe whether all the detection item points corresponding to the detection item are contained in the first image. For example, if the detection item is the bolts on the bogie, then check whether the first image contains the bogie, whether it contains all the bolts on the bogie, etc., and whether the bolts are clearly visible, etc.

[0067] The users here can be image algorithm processing personnel and train detection personnel, and of course other personnel with relevant knowledge.

[0068] When taking photos, the operator may control the camera to take photos. After taking the first image, the first image is sent to the teaching software. After the software operator clicks the corresponding button for obtaining the first image, the teaching software obtains the first image and displays it. The teaching software may also control the camera to take photos. For example, after the teaching software obtains the user's operation information for taking photos, it issues a command to control the camera to take photos.

[0069] Step 103, when the first image does not meet the first preset condition, the robot changes its position and / or the camera changes its posture to obtain a new first image, execute step 102 until the first image meets the first preset condition; when the first image meets the first preset condition, execute step 104.

[0070] When the user (here it may refer to the image algorithm processing personnel and the train inspection personnel) believes that the first image does not meet the first preset condition, the robot operator will be notified in time to change the robot's position and / or the camera's posture, and then retake the first image. The retaken first image will be transmitted to the teaching software, and the teaching software will display the newly taken first image after obtaining it. The image algorithm processing personnel and the train inspection personnel continue to judge the new first image. And so on, until the first image taken by the robot meets the first preset condition. This shows that the first image taken by the robot in this position and the camera in this posture meets the first preset condition. During the actual inspection, the robot is controlled to reach the corresponding position, maintain the corresponding posture, and the camera is controlled to maintain the corresponding posture, so that the first image that meets the requirements can be taken. The position point where the robot takes the qualified first image is a qualified photo point. Through this step, the train inspection personnel can instantly check whether the first image contains the required inspection item points, and the image algorithm processing personnel can also instantly check whether the first image taken meets the image algorithm processing conditions. In this way, the inspection points of each inspection item can be instantly confirmed and adjusted in the entire inspection process, instead of waiting until the entire shooting process is completed, sending the taken photos to the image algorithm processing personnel, and the image algorithm processing personnel view them through the image viewing software, and then send them to the train inspection personnel for judgment, and then need to repeat several complete processes to complete the teaching. This improves work efficiency.

[0071] Step 104, obtaining second operation information used by the user to confirm and save the photo point, obtaining the photo point information of the robot, the photo point information including the posture information of the robot and the posture information of the camera when the camera captures the first image that meets the first preset condition; the third operation information is the operation performed by the user when determining that the first image meets the first preset condition; the detection item information, annotation information and photo point information are correspondingly saved to obtain the detection item teaching information.

[0072] When the robot is taking images, the teaching software will obtain the robot's photo point information. The photo point information includes the robot's posture information and the camera's posture information when taking images. The posture information refers to the position information and posture information. The position information can be represented by three-dimensional coordinates, and the posture information can be represented by the robot's yaw angle; the camera's posture information can be represented by the camera angle. When the image algorithm processing personnel and the train inspection personnel confirm that the first image meets the first preset condition, they will click to confirm and save the photo point option. This click operation can be called the second operation. After the teaching software obtains this second operation information, the teaching software will save the photo point information and the detection item information accordingly, thereby obtaining the detection item teaching information of the detection item. A detection item teaching information can be represented in the form of an EXCEL table, see Figure 5 , the information of a row in the table can represent a detection item teaching information, and the detection item teaching information can include the photo point information and the detection item information, wherein the photo point information can include the robot's X-axis coordinate, Y-axis coordinate, Z-axis coordinate, yaw angle YAW, and camera angle; the detection item information can include the car number, location, large component, small component, and small component task. The detection item teaching information can also include the name of the first image, the exposure time, the first image itself, etc. The current detection item teaching is completed. Then enter the teaching of the next detection item.

[0073] Step 105: When the user inputs the next detection item, steps 101-104 are executed.

[0074] After the teaching of one inspection item is completed, the teaching of the next inspection item is started. The user inputs a new inspection item. If the next inspection item is another inspection part of the inspection train, the robot operator moves the robot to a shooting area where the inspection part can be photographed, adjusts the robot's posture and the camera's posture, takes pictures, and then executes steps 101-104 until the teaching of the inspection item is completed.

[0075] Step 106 , obtaining third operation information input by the user for generating a teaching file, and generating a teaching file, wherein the teaching file includes detection item teaching information corresponding to each detection item.

[0076] Then, and so on, until all the detection items have been taught. The number of detection items is determined according to the actual situation, and can be one or more. The multiple here refers to more than two (including two). When all the detection items have been taught, the user will click the Save Teaching File button. This click operation of the user is referred to as the third operation here. After obtaining the third operation information, the teaching software generates a teaching file. The teaching file includes the detection item teaching information corresponding to each detection item. The specific format of the teaching file is not limited. It can be expressed in the form of an EXCEL table, in the form of a .json file, etc. See. Figure 5 The teaching file includes teaching information of detection items corresponding to multiple detection items.

[0077] After the teaching is completed, a program is written based on the teaching file and input into the robot. The robot will then take pictures at the photo points corresponding to each detection item. After the shooting is completed, the photos and related information are input into the image processing software for processing and analysis to obtain the detection results.

[0078] In this method, after the robot captures the first image, it can be sent to the teaching software for display in time for confirmation by the image processing personnel and the train inspection personnel. When the first image does not meet the requirements, it can be adjusted in time with the robot operator to obtain an image that meets the first requirement and obtain a photo point that meets the requirements. This allows the robot operator, image processing personnel and train inspection personnel to teach at the same time. When teaching each inspection item, the robot operator can be notified in time to make adjustments, so as to obtain accurate and better photo points. It is no longer like before, where the robot operator first takes photos of all inspection items and records the corresponding photo point information through the teaching software; then sends the photos to the image algorithm processing personnel, the image algorithm processing personnel opens the image viewing software for viewing, and sends the photos to the train inspection personnel for judgment after judgment. The three steps cannot be performed at the same time, and one step needs to be completed before the next step can be performed. In this way, these three steps need to be repeated every time a modification is required. In this way, when using this method for teaching, it takes less time, is more efficient, and has a higher accuracy rate.

[0079] Optionally, before step 104, step 107 is further included, obtaining fourth operation information of the user for marking on the first image, and generating marking information;

[0080] The step 104 of correspondingly saving the detection item information and the photographing point information to obtain the detection item teaching information of the detection item specifically includes:

[0081] The detection item information, the photographing point information and the marking information are correspondingly saved to obtain the detection item teaching information of the detection item.

[0082] If both the image algorithm processing personnel and the train inspection personnel believe that the first image meets the first preset condition, the user can make annotations on the first image. This annotation operation can be referred to as the fourth operation here. After obtaining the fourth operation information, the teaching software generates the corresponding annotation information. The annotation method can be to mark the annotation box on the first image, see Figure 3 Taking bolts as an example, a rectangular frame can be used to mark the first image to frame the bolts on the first image. The marking information can provide a reference for the subsequent image processing software when processing the first image, so that the image processing software can perform identification and detection more specifically, thereby obtaining more accurate detection results. The marking information, detection item information, and photo point information are saved in correspondence, and together constitute the detection item teaching information of the detection item. See Figure 6 , annotation information (such as annotation boxes) can be saved in the form of .json files.

[0083] Optionally, before step 106, the following steps may also be performed:

[0084] When the user moves the robot from one photo taking point to the next photo taking point, obtaining waypoint information of the robot, wherein the waypoint information includes position information of several position points of the robot;

[0085] The waypoint information and the photo taking point information are saved in correspondence;

[0086] The teaching file generated in step 106 includes the waypoint information.

[0087] When inspecting trains, some robots may have preset fixed tracks. For example, when it comes to unmanned vehicles, the fixed tracks can be set in advance so that the unmanned vehicle can travel along the fixed tracks. At this time, the photo points are determined. Since there is a fixed track between the two photo points, the driving route of the unmanned vehicle is also determined. The unmanned vehicle can move from one photo point to the next along the preset track according to its own movement strategy. In this case, waypoint information may not be involved.

[0088] When the robot does not have a fixed track, for example, when the robot is a drone, when it flies from one photo point to the next photo point according to its own mobile strategy, it may be hindered by the train and cannot travel along the best route (shortest distance or shortest time). Its flight path needs to be planned. When teaching the drone, the drone can be held and moved from one photo point to the next photo point along the planned route. In this process, when the robot moves to a certain position point, the robot operator can notify the teaching software operator to record the position information of the position point. The teaching software operator can click the waypoint button. The operation of clicking the waypoint button can be referred to as the eighth operation here. After obtaining the eighth operation information, the teaching software can obtain the waypoint information of the position point. The waypoint information includes the position information of the position point. In this way, several waypoint information can be obtained between two photo points. The waypoint information and the photo point information are saved accordingly. When generating the teaching file, the waypoint information is also saved in the teaching file. These several waypoints can form the driving route of the drone from one photo point to another. In this way, the best route can be planned in advance for the robot to travel.

[0089] Optionally, before step 101, step 108 is further included, obtaining a train inspection procedure, generating and displaying an inspection item library based on the inspection procedure, wherein the inspection item library includes inspection items for selection by a user;

[0090] The step 101 specifically includes: obtaining first operation information of a detection item selected by a user from the detection item library, and generating detection item information based on the detection item selected by the user.

[0091] Different trains may have different inspection requirements for some reason, and the inspection procedures will also be different, and the required inspection items will also be different. For example, different trains may have different structures, and the parts required to be inspected are also different. In this way, the inspection procedures of the train can be imported into the teaching software. The teaching software generates an inspection item library based on the inspection procedures. When the user enters the inspection items, he can just click them. The teaching software generates inspection item information based on the inspection items selected by the user. See Figure 2 , you can click on the two columns of test item libraries on the right side of the teaching software, see Figure 5 The corresponding generated detection item information may include the carriage number, location (left or right side of the vehicle), large parts, small parts, small parts tasks, etc.

[0092] Optionally, in step 102, obtaining a first image captured by a camera on the robot and displaying the first image specifically includes:

[0093] Acquire fifth operation information of the user for real-time display, acquire the camera's framing image in real time during the robot's camera framing process, and display the framing image in real time;

[0094] Acquire sixth operation information of a shooting button clicked by a user, and control the camera to shoot to obtain the first image;

[0095] The first image is acquired and displayed.

[0096] In the process of the robot operator controlling the robot to make the camera take pictures, the teaching software can obtain the camera's viewfinder image in real time (that is, the image captured in the camera when the camera is turned on but the shutter is not pressed to form a photo) and display it. There can be a real-time display image button on the teaching software. After the software operator clicks the real-time display image button, the operation of clicking the real-time display image button is referred to as the fifth operation information here. After obtaining the fifth operation information, the teaching software obtains the viewfinder image in real time and displays the viewfinder image in the image display area on the teaching software interface. In this way, when the robot's posture changes or the camera angle changes, the camera's viewfinder image can be transmitted to the teaching software in real time for the teaching software operator to view. When the photo taken by the camera does not meet the first preset condition, the robot operator can be guided to make fine adjustments according to the viewfinder image. When the teaching software operator thinks it is possible, the shooting button on the teaching software (named as the photo point in the figure) can be clicked. This click operation is called the sixth operation. After obtaining the sixth operation information, the teaching software controls the camera to shoot and obtain the first image. At this time, the image display area of ​​the teaching software no longer displays the real-time viewfinder image, but the first image obtained by shooting. In this way, the photo point information can be determined more conveniently. The real-time image display button and the shooting button can be two buttons or one button. When it is one button, the display state is switched once each time the button is clicked. For example, when the viewfinder image is displayed, clicking the button will display the first image taken. When the first image is displayed, clicking the button will display the viewfinder image in real time.

[0097] It should be noted that the camera angle can be controlled and adjusted by the robot operator or through the teaching software.

[0098] Optionally, in step 102, obtaining a first image captured by a camera on the robot and displaying the first image specifically includes:

[0099] Acquire sixth operation information of a shooting button clicked by a user, and control the camera to shoot to obtain the first image;

[0100] The first image is acquired and displayed.

[0101] Of course, the teaching software may not have the function of displaying the viewfinder image in real time, but only controls the camera to capture the first image after the user clicks the capture button, then acquires the first image, and displays the first image.

[0102] Optionally, before step 104, the following steps may also be included:

[0103] The seventh pre-processing operation information selected by the user is obtained, the first image is pre-processed, and a pre-processed image is output; the pre-processed image is used to assist the user in determining whether the first image meets the first preset condition.

[0104] After the teaching software displays the first image, the software operator can click the pre-processing button. The click operation here is called the seventh operation information. After obtaining the first operation information, the teaching software can pre-process the first image to obtain a pre-processed image and display it. Figure 4 , Figure 4 The image display area in the image display area is a pre-processed image, which can be used to reflect the result of the preliminary recognition and detection of the first image by the image processing software. Before the pre-processing function is added, whether the first image can be processed by the image processing software mainly depends on the experience of the image algorithm personnel. After the image pre-processing function is added, it can help the image algorithm processing personnel to more accurately judge whether the image processing software can process the first image.

[0105] The teaching software interface may also include buttons for adding and deleting waypoints and photo points, and display areas for waypoint counts, photo point counts, and camera angles.

[0106] The present invention also provides a teaching method for a train detection robot, comprising:

[0107] Step 201: The teaching software operator opens the teaching software on the teaching device and establishes a communication connection with the robot, which may be based on the IP address of the robot.

[0108] Step 202: the teaching software operator inputs the test items on the teaching software.

[0109] Step 203, the robot operator moves the robot to a detection position and adjusts the position of the robot and the posture of the camera;

[0110] Step 204: the robot operator controls the camera to take a picture, or the teaching software operator controls the camera to take a picture through the teaching software to obtain a first image;

[0111] Step 205, the teaching software acquires the first image and displays it;

[0112] Step 206, the teaching software operator controls the teaching software to pre-process the first image to obtain a pre-processed image;

[0113] Step 207, the teaching software operator determines whether the first image meets the requirements based on the first image and the pre-processed image;

[0114] Step 208, if the requirements are not met, the robot operator is notified to adjust the robot's posture and the camera's posture, and then steps 204-206 are executed repeatedly until the first image meets the requirements, and the shooting position of the robot taking the first image that meets the requirements is determined as the shooting point of the detection item;

[0115] Step 209: If the requirements are met, the teaching software operator annotates the first image and generates annotation information;

[0116] Step 210, the teaching software operator controls the teaching software to obtain the photographing point information corresponding to the first image, and saves the photographing point information, the detection item information and the marking information accordingly, so as to obtain the detection item teaching information of the detection item;

[0117] Step 211, the operator of the teaching software inputs the next test item, and then loops through steps 202-210;

[0118] Step 212, looping through step 211 until all detection items have been taught;

[0119] Step 213, the robot operator controls the robot to move along a preset route between the two photo taking points, selects a number of position points in the preset route as waypoints, and notifies the teaching personnel to obtain and record waypoint information of each waypoint, where the waypoint information includes the position information of the waypoint;

[0120] Step 214 , the software operator controls the teaching software to record the photographing point information and the waypoint information in correspondence.

[0121] Step 215 , the software operator controls the teaching software to generate a teaching file, wherein the teaching file includes teaching information corresponding to each detection item and waypoint information.

[0122] The present invention also provides a computer device, comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of any one of the methods described above.

[0123] The present invention also provides a computer-readable storage medium having a computer program / instruction stored thereon, wherein the computer program / instruction implements the steps of any one of the methods described above when executed by a processor.

Claims

1. A teaching method for a train inspection robot, characterized in that: include: Step 101, obtaining first operation information of a user for inputting a detection item, and generating detection item information; Step 102, obtaining a first image taken by a camera on the robot, and displaying the first image; so that the user can judge whether the first image meets the first preset condition, wherein the first preset condition includes that the first image contains the detection item point corresponding to the detection item and the first image satisfies the image algorithm processing condition; Step 103, when the first image does not meet the first preset condition, the robot changes its position and / or the camera changes its posture to obtain a new first image, executing step 102 until the first image meets the first preset condition; when the first image meets the first preset condition, executing step 104; Step 104, obtaining second operation information of the user for confirming and saving the photo point, and obtaining the photo point information of the robot, wherein the photo point information includes the position information of the robot and the posture information of the camera when the camera captures the first image that meets the first preset condition; The detection item information and the photographing point information are correspondingly saved to obtain detection item teaching information of the detection item; Step 105, when the user inputs the next test item, execute steps 101-104; Step 106 , obtaining third operation information input by the user for generating a teaching file, and generating a teaching file, wherein the teaching file includes detection item teaching information corresponding to each detection item.

2. The teaching method of the train inspection robot according to claim 1, characterized in that: Before step 104, the method further includes step 107, obtaining fourth operation information of the user for marking on the first image, and generating marking information; The step 104 of correspondingly saving the detection item information and the photographing point information to obtain the detection item teaching information of the detection item specifically includes: The detection item information, the photographing point information and the marking information are correspondingly saved to obtain the detection item teaching information of the detection item.

3. The teaching method of the train inspection robot according to claim 1, characterized in that: Before step 106, the following steps are also included: When the user moves the robot from one photo taking point to the next photo taking point, obtaining waypoint information of the robot, wherein the waypoint information includes position information of several position points of the robot; The waypoint information and the photo taking point information are saved in correspondence; The teaching file generated in step 106 includes the waypoint information.

4. The teaching method of the train inspection robot according to claim 1, characterized in that: The step 101 also includes step 108, obtaining a train inspection procedure, generating and displaying an inspection item library based on the inspection procedure, wherein the inspection item library includes inspection items for selection by a user; The step 101 specifically includes: obtaining first operation information of a detection item selected by a user from the detection item library, and generating detection item information based on the detection item selected by the user.

5. The teaching method of the train inspection robot according to claim 1, characterized in that: The step 102 of acquiring a first image captured by a camera on the robot and displaying the first image specifically includes: Acquire fifth operation information of the user for real-time display, acquire a framing image of the camera during the framing process of the camera on the robot, and display the framing image in real time; Acquire sixth operation information of a shooting button clicked by a user, and control the camera to shoot to obtain the first image; The first image is acquired and displayed.

6. The teaching method of the train inspection robot according to claim 1, characterized in that: The step 102 of acquiring a first image captured by a camera on the robot and displaying the first image specifically includes: Acquire sixth operation information of a shooting button clicked by a user, and control the camera to shoot to obtain the first image; The first image is acquired and displayed.

7. The teaching method of the train inspection robot according to claim 1, characterized in that: Before step 104, the following steps are also included: The seventh pre-processed operation information selected by the user is obtained, and the first image is pre-processed to obtain a pre-processed image; the pre-processed image is used to assist the user in determining whether the first image meets the first preset condition.

8. A teaching method for a train inspection robot, characterized in that: include: Step 201, the teaching software operator opens the teaching software on the teaching device and establishes a communication connection with the robot; Step 202, the teaching software operator inputs the test items on the teaching software; Step 203, the robot operator moves the robot to a detection position and adjusts the position of the robot and the posture of the camera; Step 204: the robot operator controls the camera to take a picture, or the teaching software operator controls the camera to take a picture, to obtain a first image; Step 205, the teaching software acquires the first image and displays it; Step 206, the teaching software operator controls the teaching software to pre-process the first image to obtain a pre-processed image; Step 207, the teaching software operator determines whether the first image meets the requirements based on the first image and the pre-processed image; Step 208, if the requirements are not met, the robot operator is notified to adjust the robot's posture and the camera's posture, and then steps 204-206 are executed repeatedly until the first image meets the requirements, and the shooting position of the robot taking the first image that meets the requirements is determined as the shooting point of the detection item; Step 209: If the requirements are met, the teaching software operator annotates the first image and generates annotation information; Step 210, the teaching software operator controls the teaching software to obtain the photographing point information corresponding to the first image, and saves the photographing point information, the detection item information and the marking information accordingly, so as to obtain the detection item teaching information of the detection item; Step 211, the operator of the teaching software inputs the next test item, and then loops through steps 202-210; Step 212, looping through step 211 until all detection items have been taught; Step 213, the robot operator controls the robot to move along a preset route between the two photo taking points, selects a number of position points in the preset route as waypoints, and notifies the teaching personnel to obtain and record waypoint information of each waypoint, where the waypoint information includes the position information of the waypoint; Step 214, the software operator controls the teaching software to record the photographing point information and the waypoint information in correspondence; Step 215 , the software operator controls the teaching software to generate a teaching file, wherein the teaching file includes teaching information corresponding to each detection item and waypoint information.

9. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.

Images