Train detection robot teaching method and device and storage medium

Abstract

The application discloses a kind of train detection robot's teaching method, device and storage medium, it is related to train detection technical field.The method comprises the following steps: step 101, obtain the first operation information of user, generate detection item information;Step 102, obtain the first image photographed by the camera on robot, and first image is displayed;Step 103, when first image does not comply with first preset condition, execute step 102 until first image complies with first preset condition;Otherwise, execute step 104;Step 104, obtain the second operation information of user, obtain the photographing point information of robot;Detection item information and photographing point information are saved correspondingly to obtain the detection item teaching information of the detection item;Step 105, when user inputs next detection item, execute step 101-104;Step 106, obtain the third operation information of user, generate teaching file.The present application is used to solve the technical problem that the time required by existing teaching method is long and the process is complicated.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of train detection technology, and particularly relates to a teaching method and device of a train detection robot and a storage medium. BACKGROUND

[0002] In the rail transportation industry, using a robot to replace a person to detect defects of a train is a current intelligent development trend. The robot is provided with a camera or a photographing device, and the robot photographs a detection position, and then an image processing software is used to analyze and process the photographed image to obtain a detection result. The robot can be a drone, an unmanned vehicle, etc. In this process, the robot needs to be taught first. The teaching process is a learning process of the robot, and an operator needs to teach the robot to perform certain actions, and in this process, a control system memorizes the actions in a program form. After the teaching is completed, the program is input to the robot, so that the robot performs the actions according to the memorized program.

[0003] In the prior art, a teaching software and an image viewing software are used in the teaching. The teaching software is used to record position information of the robot when a robot operator controls the robot to be in a pose and photographs a detection position of the train. The image viewing software is used to view the image photographed by the robot. The image processing software is used to process the image to obtain a detection result. The specific process is as follows: 1. The robot operator teaches the robot in a pose, that is, the robot operator moves the robot to a detection position, adjusts a pose of the robot and a photographing angle of a camera, and then photographs an image. The teaching software records position information of the robot, pose information of the robot and the photographing angle of the camera when the image is photographed. Then, the robot operator moves the robot to a next detection position, and continues to photograph until the photographing of all the detection positions is completed. 2. An image algorithm processing personnel packs the photographed image, views the image by using the image viewing software, judges whether the image meets image algorithm processing conditions, and sends a document of a judgment result to a train detection personnel for confirmation. 3. The train detection personnel further confirms whether the image meets requirements by observing whether the image contains detection points, etc. If the image algorithm processing personnel and the train detection personnel think that the image meets all the requirements, the teaching information recorded by the teaching software is available. If some images do not meet the requirements, the robot operator needs to re-photograph and re-record the position information of the robot, the pose information of the robot and the photographing angle of the camera. This process is repeated several times until the teaching is completed.

[0004] Therefore, first, the robot operator needs to be familiar with the train detection procedure to know the best shooting position, shooting angle, etc., and the requirement for the robot operator is high. Second, the demonstration often needs to be repeated several times to complete, and each demonstration needs to complete the demonstration shooting, image algorithm processing personnel confirmation, train detection personnel confirmation three links in turn, and the demonstration needs a long time and a complicated process. SUMMARY

[0005] The purpose of the present application is to provide a train detection robot demonstration method, device and storage medium, which can make the robot operator, image algorithm personnel and train detection personnel can together demonstrate work, there is no repeated confirmation modification process, improve the efficiency and accuracy of the demonstration work.

[0006] In order to achieve the above purpose, the present application adopts the following technical scheme:

[0007] The present application provides a train detection robot demonstration method, comprising:

[0008] Step 101, obtaining the first operation information of the user for inputting the detection item, generating the detection item information;

[0009] Step 102, obtaining the first image shot by the camera on the robot, and displaying the first image; so that the user judges whether the first image meets the first preset condition, the first preset condition includes that the first image contains the detection item point corresponding to the detection item and the first image meets the image algorithm processing condition;

[0010] Step 103, when the first image does not meet the first preset condition, the robot changes the pose and / or the camera changes the attitude to shoot a new first image, executes step 102 until the first image meets the first preset condition; when the first image meets the first preset condition, executes step 104;

[0011] Step 104, obtaining the second operation information of the user for confirming and saving the shooting point, obtaining the shooting point information of the robot, the shooting point information includes the pose information of the robot and the attitude information of the camera when the first image shot by the camera meets the first preset condition; saving the detection item information and the shooting point information correspondingly to obtain the detection item demonstration information of the detection item;

[0012] Step 105, when the user inputs the next detection item, executing steps 101-104;

[0013] Step 106, obtaining the third operation information of the user for inputting to generate the demonstration file, generating the demonstration file, the demonstration file contains the detection item demonstration information corresponding to each detection item.

[0014] Optionally, the step 104 further comprises a step 107 of obtaining fourth operation information of a user for marking on the first image, and generating the marking information.

[0015] The step 104 of saving the detection item information and the photographing point information correspondingly to obtain the detection item teaching information of the detection item specifically comprises:

[0016] The detection item information, the photographing point information and the marking information are saved correspondingly to obtain the detection item teaching information of the detection item.

[0017] Optionally, the step 106 further comprises:

[0018] In the process that the user moves the robot from one photographing point to the next photographing point, the position information of a plurality of position points of the robot is obtained.

[0019] The photographing point information and the position information of the plurality of position points of the robot are saved correspondingly.

[0020] The teaching file generated in the step 106 contains the position information of the plurality of position points.

[0021] Optionally, the step 101 further comprises a step 108 of obtaining a train detection procedure, and generating and displaying a detection item library based on the detection procedure, the detection item library comprising detection items for a user to select.

[0022] The step 101 specifically comprises: obtaining first operation information of a detection item selected by a user from the detection item library, and generating detection item information based on the detection item selected by the user.

[0023] Optionally, the step 102 of obtaining a first image photographed by a camera on a robot and displaying the first image specifically comprises:

[0024] Obtaining fifth operation information of a user for real-time display, obtaining a viewfinder image of the camera in the process that the camera is viewed, and displaying the viewfinder image in real time.

[0025] Obtaining sixth operation information of a user for selecting a photographing button, and controlling the camera to photograph to obtain the first image.

[0026] Obtaining the first image and displaying the first image.

[0027] Optionally, the step 102 of obtaining a first image photographed by a camera on a robot and displaying the first image specifically comprises:

[0028] Obtaining the sixth operation information of the shooting button clicked by the user, and controlling the camera to shoot the first image;

[0029] Obtaining the first image and displaying the first image.

[0030] Optionally, the step 104 further comprises:

[0031] Obtaining the seventh operation information of the pre-processing clicked by the user, and pre-processing the first image to obtain a pre-processed image; the pre-processed image is used to assist the user in judging whether the first image meets the first preset condition.

[0032] The application further provides a teaching method of a train detection robot, comprising:

[0033] In step 201, a teaching software operator opens the teaching software on a teaching device and communicates with the robot;

[0034] In step 202, the teaching software operator inputs a detection item on the teaching software;

[0035] In step 203, a robot operator moves the robot to a detection position and adjusts the position of the robot and the posture of the camera;

[0036] In step 204, the robot operator controls the camera to take a photo, or a teaching software operator controls the camera to take a photo, and obtains a first image;

[0037] In step 205, the teaching software obtains the first image and displays the first image;

[0038] In step 206, the teaching software operator controls the teaching software to pre-process the first image and obtains a pre-processed image;

[0039] In step 207, the teaching software operator judges whether the first image meets the requirement based on the first image and the pre-processed image;

[0040] In step 208, if the requirement is not met, the robot operator is informed to adjust the position and posture of the robot and the posture of the camera, and then the steps 204-206 are circularly executed until the first image meets the requirement, and the shooting position of the robot for the first image meeting the requirement is determined as a shooting point of the detection item;

[0041] In step 209, if the requirement is met, the teaching software operator labels the first image and generates label information;

[0042] In step 210, the teaching software operator controls the teaching software to acquire the photographing point information corresponding to the first image, and saves the photographing point information, the detection item information and the labeling information correspondingly to obtain the detection item teaching information of the detection item;

[0043] In step 211, the teaching software operator inputs the next detection item, and then executes steps 202-210 cyclically.

[0044] In step 212, step 211 is executed cyclically until all the detection items are taught.

[0045] In step 213, the robot operator controls the robot to move along the preset route between the two photographing points, selects a plurality of position points as the waypoints in the preset route, and informs the teaching personnel to acquire and record the waypoint information of each waypoint, wherein the waypoint information comprises the position information of the waypoint.

[0046] In step 214, the software operator controls the teaching software to record the photographing point information and the waypoint information correspondingly.

[0047] In step 215, the software operator controls the teaching software to generate a teaching file, wherein the teaching file comprises the teaching information corresponding to each detection item and the waypoint information.

[0048] The application further provides a computer device comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to realize the steps of any one of the above-described methods.

[0049] The application further provides a computer readable storage medium, wherein a computer program / instruction is stored in the computer readable storage medium, and the computer program / instruction is executed by a processor to realize the steps of any one of the above-described methods.

[0050] In the technical scheme, after the robot shoots the first image, the first image can be sent to the teaching software in time for display, so as to be confirmed by the image processing personnel and the train detection personnel, and when the first image does not meet the requirements, the robot operator can be adjusted in time so as to obtain an image meeting the first requirements and a shooting point meeting the requirements. Thus, the robot operator, the image processing personnel and the train detection personnel can simultaneously perform teaching. When each detection item is taught, the robot operator can be timely informed to make adjustment, so as to obtain an accurate and better shooting point. Unlike the previous method, the robot operator first shoots photos of all detection items, and then records corresponding shooting point information through the teaching software; then the photos are sent to the image algorithm processing personnel, the image algorithm processing personnel opens the image viewing software to view and judge, and then sends to the train detection personnel for judgment. The three steps cannot be simultaneously performed, and the next step can be performed only after the previous step is completed. Thus, when the method is used for teaching, time is saved, efficiency is improved, and accuracy is improved. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 A flow chart of a train detection robot teaching method;

[0052] Figure 2 An interface schematic diagram of a state before the teaching software acquires the first image;

[0053] Figure 3 An interface schematic diagram of a state in which the teaching software displays the first image and makes annotations;

[0054] Figure 4 An interface schematic diagram in which the teaching software displays the preprocessed image after preprocessing the first image;

[0055] Figure 5 A schematic diagram of annotation information represented in the generated teaching file in the.jason format.

[0056] Figure 6 A schematic diagram of annotation information represented in the generated teaching file in the.jason format. DETAILED DESCRIPTION

[0057] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The following description is only one of the exemplary embodiments and does not represent all the embodiments consistent with the present application. Rather, they are merely examples of devices or methods consistent with some aspects of the present application.

[0058] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described in detail below. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work belong to the scope of protection of the present application.

[0059] Hereinafter, the embodiments will be described with reference to the accompanying drawings. In addition, the embodiments shown below do not have any limiting effect on the invention content recited in the claims. In addition, the entire content of the configuration represented by the following embodiments is not limited to what is necessary as a solution to the invention recited in the claims.

[0060] Referring to Figure 1 , the present application provides a teaching method of a train detection robot. From the perspective of program, the execution subject of the flow can be a program loaded on a server or a teaching device; from the perspective of hardware, the execution subject of the flow can be a teaching device, which can be specifically a computer, a control center, etc. As shown in Figure 1 , the method can include the following steps:

[0061] Step 101, obtaining first operation information of a user for inputting a detection item, and generating detection item information.

[0062] The teaching software is opened on the teaching device, and a communication connection is established with the robot to be taught. The user inputs the detection item to be taught on the interface of the teaching software. The input mode of the detection item is not limited, which can be specifically manual typing input, voice input, etc. The detection item library can also be generated in advance on the interface of the teaching software, referring to Figure 2 , one form of presentation of the detection item library can be two columns of option lists on the right side of Figure 2 . The user selects the detection item to be taught in the option list, for example, the detection item is a bolt on the bogie, and the bogie and bolt options are selected. After obtaining the first operation information of the user, the teaching software responds to the first operation information, generates the detection item information corresponding to the detection item, and records and saves it.

[0063] The user here refers to a teaching software operator, which can be specifically an image algorithm processing personnel or a train detection personnel, and of course can also be other personnel with relevant knowledge.

[0064] Step 102, obtaining a first image photographed by a camera on the robot, and displaying the first image; so that the user judges whether the first image meets a first preset condition, the first preset condition including that the first image contains a detection item point corresponding to the detection item and the first image meets an image algorithm processing condition.

[0065] The robot operator moves the robot to a shooting area corresponding to the detection item. For example, if the detection item is a bolt on the bogie on the left side of the carriage 1, the robot is moved to a shooting area where the bolt on the bogie can be shot, and then the position (which can be represented by three-dimensional coordinates) and the attitude (which can be represented by the yaw angle of the robot) of the robot, and the attitude (camera angle) of the camera are further adjusted to shoot a certain part of the train (which can refer to the bogie in this example) corresponding to the detection item by the camera, so as to obtain a first image. The first image is transmitted to the teaching software, and the teaching software displays the first image in the corresponding area of the teaching software interface after obtaining the first image. See Figure 3 The first image is displayed in the image display area. Specifically, the first image can be displayed after the user clicks the photograph point setting display button. When the photograph point setting display button is clicked again, the latest obtained first image is displayed.

[0066] At this time, the user determines whether the first image meets the first preset condition by observing the first image. The first preset condition includes that the first image contains the detection item point corresponding to the detection item and that the first image meets the image algorithm processing condition. Specifically, the image algorithm processing personnel can determine whether the first image can be processed by the image processing software according to experience. The basis for the determination can be whether the first image is too dark or too bright, or whether the shooting angle of the first image is suitable for the image processing software. The train detection personnel can determine whether the first image contains all the detection item points corresponding to the detection item by observing the first image. For example, if the detection item is a bolt on the bogie, the train detection personnel can determine whether the first image contains the bogie and all the bolts on the bogie, and whether the bolts are clearly visible.

[0067] The user can be an image algorithm processing personnel and a train detection personnel, and can also be other personnel with relevant knowledge.

[0068] When shooting, the operator can control the camera to shoot. After shooting the first image, the first image is sent to the teaching software. After the software operator clicks the button for obtaining the first image, the teaching software obtains and displays the first image. The camera can also be controlled by the teaching software to shoot. For example, after the teaching software obtains the operation information of the user for shooting, the teaching software controls the camera to shoot.

[0069] In step 103, when the first image does not meet the first preset condition, the robot changes the position and / or the camera changes the attitude to shoot a new first image, step 102 is executed until the first image meets the first preset condition; when the first image meets the first preset condition, step 104 is executed.

[0070] When the user (herein can refer to the image algorithm processing personnel and the train detection personnel) considers that the first image does not meet the first preset condition, the robot operating personnel will be timely notified to change the pose of the robot and / or the pose of the camera, and then the first image is rephotographed. The rephotographed first image is transmitted to the teaching software, the teaching software displays the newly photographed first image after obtaining the newly photographed first image, and the image algorithm processing personnel and the train detection personnel continue to judge the new first image. In this way, until the first image photographed by the robot under this pose and the pose of the camera meets the first preset condition. This indicates that the first image photographed by the robot under this pose and the pose of the camera meets the first preset condition, and in actual inspection, the robot is controlled to reach the corresponding position and maintain the corresponding pose, and the camera is controlled to maintain the corresponding pose, so that the first image meeting the requirements can be photographed. The position point of the robot for photographing the qualified first image is a qualified photographing point. Through this step, the train detection personnel can immediately check whether the first image contains the required detection item point, and the image algorithm processing personnel can also immediately check whether the photographed first image meets the image algorithm processing condition, so that the detection points of each detection item can be immediately confirmed and adjusted in the entire detection process, instead of sending the photographed photos to the image algorithm processing personnel, the image algorithm processing personnel checks through the image checking software, judges, and then sends to the train detection personnel for judgment, and then several complete processes need to be repeated to complete the teaching. In this way, the work efficiency is improved.

[0071] In step 104, second operation information used by the user to confirm and save the photographing point is obtained, photographing point information of the robot is obtained, the photographing point information includes pose information of the robot and pose information of the camera when the camera photographs the first image meeting the first preset condition; the third operation information is an operation performed by the user when it is determined that the first image meets the first preset condition; the detection item information, the labeling information and the photographing point information are correspondingly saved to obtain the detection item teaching information.

[0072] When the robot photographs the image, the teaching software obtains the photographing point information of the robot, the photographing point information includes the pose information of the robot and the pose information of the camera when the robot photographs the image, the pose information refers to the position information and the attitude information, the position information can be represented by three-dimensional coordinates, and the attitude information can be represented by the yaw angle of the robot; the pose information of the camera can be represented by the angle of the camera. When the image algorithm processing personnel and the train detection personnel confirm that the first image meets the first preset condition, the confirmation and saving of the photographing point option is clicked, this clicking operation can be referred to as a second operation, after the teaching software obtains the second operation information, the teaching software correspondingly saves the photographing point information and the detection item information, so as to obtain the detection item teaching information of the detection item. One kind of detection item teaching information can be represented in the form of an EXCEL table, seeFigure 5 The information of a row in the table can represent a detection item teaching information, and the detection item teaching information can include photographing point information and detection item information. The photographing point information can include X-axis coordinate, Y-axis coordinate, Z-axis coordinate, yaw angle YAW, and camera angle of the robot. The detection item information can include car number, position, large component, small component, and small component task. The detection item teaching information can further include name of the first image, exposure time, and the first image itself. The current detection item teaching is completed. Then, teaching of the next detection item is entered.

[0073] In step 105, when the user inputs the next detection item, steps 101-104 are executed.

[0074] After teaching of one detection item is completed, teaching of the next detection item is entered. The user inputs a new detection item. If the next detection item is to detect another detection position of the train, the robot operator moves the robot to a photographing area where the detection position can be photographed, adjusts the pose of the robot and the pose of the camera, takes a photograph, and then executes steps 101-104 until teaching of the detection item is completed.

[0075] In step 106, third operation information used by the user to input generation of the teaching file is obtained, and the teaching file is generated. The teaching file includes detection item teaching information corresponding to each detection item.

[0076] Then, by analogy, until all detection items are taught. The number of detection items is determined according to actual conditions, and can be one or more. More than two (including two) are referred to as more than one. When all detection items are taught, the user will click the save teaching file button. The user's clicking operation is referred to as a third operation. After the teaching software obtains the third operation information, the teaching file is generated. The teaching file includes detection item teaching information corresponding to each detection item. The specific format of the teaching file is not limited. Specifically, the EXCEL table form can be used to represent, the.json file form can be used to represent, etc. See Figure 5 The teaching file includes detection item teaching information corresponding to a plurality of detection items.

[0077] After teaching is completed, a program based on the teaching file is written and input to the robot. The robot will take a photograph at the photographing point corresponding to each detection item. After the photographing is completed, the photograph and related information are input to the image processing software for processing and analysis, and the detection result is obtained.

[0078] The method can send the first image to the teaching software for display in time after the robot shoots the first image, so that the image processing personnel and the train detection personnel can confirm, and when the first image does not meet the requirements, the robot operator can be adjusted in time to obtain an image meeting the first requirements and a shooting point meeting the requirements. Thus, the robot operator, the image processing personnel and the train detection personnel can simultaneously perform teaching. When each detection item is taught, the robot operator can be timely informed to make adjustment, so that an accurate and better shooting point is obtained. Unlike the previous method, the robot operator first shoots photos of all detection items, records corresponding shooting point information through the teaching software, then sends the photos to the image algorithm processing personnel, the image algorithm processing personnel opens the image viewing software to view and judge, and sends to the train detection personnel for judgment. The three steps cannot be performed simultaneously, and the next step can be performed only after the previous step is completed. Thus, the three steps need to be repeated every time the modification is needed. Thus, when the method is used for teaching, the time consumption is short, the efficiency is high, and the accuracy is high.

[0079] Optionally, the step 104 further includes a step 107 of acquiring fourth operation information of a user for marking on the first image to generate marking information.

[0080] The step 104 of saving the detection item information and the shooting point information in correspondence to obtain the detection item teaching information of the detection item specifically includes:

[0081] The detection item information, the shooting point information and the marking information are saved in correspondence to obtain the detection item teaching information of the detection item.

[0082] If the image algorithm processing personnel and the train detection personnel both think that the first image meets the first preset condition, the user can mark on the first image. This marking operation can be referred to as a fourth operation, and the teaching software generates marking information after acquiring the fourth operation information. The marking manner can be marking a marking box on the first image. Referring to FIG. 6, a marking box 601 is marked on the first image 600. Figure 3 Taking a bolt as an example, a rectangular box can be marked on the first image to frame the bolt on the first image. The marking information can provide a reference for subsequent image processing software when processing the first image, so that the image processing software can more specifically identify and detect, and thus obtain more accurate detection results. The marking information, the detection item information and the shooting point information are saved in correspondence to jointly constitute the detection item teaching information of the detection item. Referring to FIG. 6, the marking information (for example, the marking box) can be saved in the form of a.json file. Figure 6

[0083] Optionally, the step 106 further includes a step 108 of acquiring fifth operation information of a user for marking on the first image to generate marking information.​

[0084] In the process that the user moves the robot from one photographing point to the next photographing point, the waypoint information of the robot is acquired, the waypoint information including position information of a plurality of position points of the robot;

[0085] The waypoint information is correspondingly saved with the photographing point information.

[0086] The teaching file generated in the step 106 contains the waypoint information.

[0087] When the train is detected, some robots can have a preset fixed track. For example, when the robot is a robot car, the fixed track can be set in advance, so that the robot car travels along the fixed track. At this time, the photographing points are determined. Since there is a fixed track between two photographing points, the travel route of the robot car is also determined. The robot car can move from one photographing point to the next photographing point along the preset track according to its own movement strategy. In this case, the waypoint information can not be involved.

[0088] When the robot has no fixed track, for example, when the robot is a drone, the drone can be hindered by the train when flying from one photographing point to the next photographing point according to its own movement strategy, and cannot travel in the best route (shortest distance or shortest time). The flight path needs to be planned. When the drone is taught, the user can hold the drone and move it from one photographing point to the next photographing point along the planned route. In this process, when the robot moves to a position point, the robot operator can inform the teaching software operator to record the position information of the position point. The teaching software operator can click the waypoint button. The operation of clicking the waypoint button is referred to as the eighth operation. After the teaching software obtains the eighth operation information, the waypoint information of the position point can be obtained. The waypoint information includes the position information of the position point. In this way, a plurality of waypoint information can be obtained between two photographing points. The waypoint information is correspondingly saved with the photographing point information. When the teaching file is generated, the waypoint information is also saved in the teaching file. The plurality of waypoints can form the travel route of the drone from one photographing point to the next photographing point. In this way, the best route can be planned in advance for the robot to travel.

[0089] Optionally, the step 101 further includes a step 108 of acquiring a train detection procedure, generating and displaying a detection item library based on the detection procedure, the detection item library including detection items for the user to select.

[0090] The step 101 specifically includes: acquiring first operation information of a detection item selected by the user from the detection item library, generating detection item information based on the detection item selected by the user.

[0091] For different trains, there can be different detection requirements for some reasons, the detection procedures are different, and the required detection items are different. For example, different trains can have different structures, and the required detection parts are different. Therefore, the detection procedure of the train can be imported into the teaching software, and the teaching software generates a detection item library based on the detection procedure. When the user inputs the detection item, the user can select it. The teaching software generates detection item information based on the detection item selected by the user. See Figure 2 The user can select the two columns of detection item library on the right side of the teaching software. See Figure 5 The corresponding generated detection item information can include car number, position (left side or right side of the vehicle), large part, small part, small part task, etc.

[0092] Optionally, the step 102 of acquiring the first image captured by the camera on the robot and displaying the first image specifically includes:

[0093] Acquire the fifth operation information of the user for real-time display, acquire the viewfinder image of the camera in real time during the viewfinder process of the camera, and display the viewfinder image in real time;

[0094] Acquire the sixth operation information of the user selecting the shooting button, and control the camera to capture the first image;

[0095] Acquire the first image and display the first image.

[0096] In the process that the robot manipulator controls the robot to make the camera take a picture, the teaching software can acquire and display the viewfinder image of the camera in real time (that is, the image taken by the camera when the camera is turned on but the shutter is not pressed to form a picture). The teaching software can have a real-time image display button. After the software manipulator clicks the real-time image display button, the operation of clicking the real-time image display button is referred to as the fifth operation information. After the fifth operation information is acquired, the teaching software acquires the viewfinder image in real time and displays the viewfinder image in the image display area of the teaching software interface. In this way, when the pose of the robot changes or the angle of the camera changes, the viewfinder image of the camera can be transmitted to the teaching software in real time for the teaching software manipulator to view. When the picture taken by the camera does not meet the first preset condition, the robot manipulator can be guided to fine-tune according to the viewfinder image. When the teaching software manipulator considers that it is appropriate, the teaching software manipulator can click the shooting button (named as the photographing point in the figure) on the teaching software. The clicking operation is referred to as the sixth operation. After the teaching software acquires the sixth operation information, the teaching software controls the camera to take the first image. At this time, the image display area of the teaching software no longer displays the real-time viewfinder image but displays the first image taken. In this way, the photographing point information can be determined more conveniently. The real-time image display button and the shooting button can be two buttons or one button. When the button is one button, the display state is switched once every time the button is clicked. For example, when the viewfinder image is displayed, the button is clicked to display the first image taken. When the first image is displayed, the button is clicked to display the viewfinder image in real time.

[0097] It should be noted that the angle of the camera can be controlled and adjusted by the robot manipulator or controlled and adjusted by the teaching software.

[0098] Optionally, the step 102 of acquiring the first image taken by the camera on the robot and displaying the first image specifically includes:

[0099] Acquiring the sixth operation information of the shooting button clicked by the user and controlling the camera to take the first image;

[0100] Acquiring the first image and displaying the first image.

[0101] Of course, the teaching software can also not have the function of displaying the viewfinder image in real time. Only after the user clicks the shooting button, the camera is controlled to take the first image, the first image is acquired, and the first image is displayed.

[0102] Optionally, the step 104 further includes:

[0103] The seventh operation information of the user's click is acquired, the first image is preprocessed, and a preprocessed image is output; the preprocessed image is used to assist the user in judging whether the first image meets the first preset condition.

[0104] After the teaching software displays the first image, the software operator can click the preprocessing button, and the click operation is referred to as the seventh operation information; after the first operation information is acquired, the teaching software can preprocess the first image to obtain a preprocessed image and display the preprocessed image. Figure 4 , Figure 4 The preprocessed image is displayed in the image display area in FIG. 6, and the preprocessed image can be used to reflect the result of the preliminary identification and detection of the first image by the image processing software. Before the preprocessing function is added, whether the first image can be processed by the image processing software mainly depends on the experience of the image algorithm personnel. After the image preprocessing function is added, the image algorithm processing personnel can more accurately judge whether the image processing software can process the first image.

[0105] The teaching software interface can further include buttons for adding and deleting waypoints and photographing points, a display area for the number of waypoints, a display area for the number of photographing points, and a display area for the camera angle.

[0106] The application further provides a teaching method of a train detection robot, including the following steps.

[0107] In step 201, a teaching software operator opens the teaching software on a teaching device and forms a communication connection with the robot. Specifically, the communication connection can be formed based on the IP of the robot.

[0108] In step 202, the teaching software operator inputs a detection item on the teaching software.

[0109] In step 203, a robot operator moves the robot to a detection position and adjusts the position of the robot and the posture of the camera.

[0110] In step 204, the robot operator controls the camera to take a photograph, or the teaching software operator controls the camera to take a photograph through the teaching software, and a first image is obtained.

[0111] In step 205, the teaching software acquires the first image and displays the first image.

[0112] In step 206, the teaching software operator controls the teaching software to preprocess the first image and obtain a preprocessed image.

[0113] In step 207, the teaching software operator judges whether the first image meets the requirements based on the first image and the preprocessed image.

[0114] Step 208, if the requirement is not met, the robot operator is informed to adjust the pose of the robot and the attitude of the camera, and then steps 204-206 are executed in a loop until the first image meets the requirement, and the photographing position of the robot for taking the first image meeting the requirement is determined as the photographing point of the detection item;

[0115] Step 209, if the requirement is met, the teaching software operator labels the first image and generates label information;

[0116] Step 210, the teaching software operator controls the teaching software to obtain photographing point information corresponding to the first image, and saves the photographing point information, detection item information and label information correspondingly to obtain detection item teaching information of the detection item;

[0117] Step 211, the teaching software operator inputs the next detection item, and then steps 202-210 are executed in a loop;

[0118] Step 212, step 211 is executed in a loop until all detection items are taught;

[0119] Step 213, the robot operator controls the robot to move between two photographing points according to a preset route, and selects a plurality of position points as waypoints in the preset route, and informs the teaching personnel to obtain and record waypoint information of each waypoint, the waypoint information including position information of the waypoint;

[0120] Step 214, the software operator controls the teaching software to record the photographing point information and the waypoint information correspondingly.

[0121] Step 215, the software operator controls the teaching software to generate a teaching file, the teaching file including teaching information corresponding to each detection item and waypoint information.

[0122] The application further provides a computer device, including a memory, a processor and a computer program stored in the memory, characterized in that the processor executes the computer program to realize the steps of any one of the above-described methods.

[0123] The application further provides a computer readable storage medium, which stores a computer program / instruction, characterized in that the computer program / instruction is executed by a processor to realize the steps of any one of the above-described methods.

Claims

1. A teaching method of a train detection robot characterized by comprising: The method comprises the following steps: Step 101: obtaining first operation information of a user for inputting a detection item, and generating detection item information; Step 102: obtaining a first image captured by a camera on a robot, and displaying the first image; so that the user judges whether the first image meets first preset conditions, the first preset conditions comprising that the first image contains a detection item point corresponding to the detection item and the first image meets image algorithm processing conditions; Step 103: when the first image does not meet the first preset conditions, obtaining a new first image after the robot changes a pose and / or the camera changes a posture, executing step 102 until the first image meets the first preset conditions; and executing step 104 when the first image meets the first preset conditions; Step 104: obtaining second operation information of the user for confirming saving of a photographing point, and obtaining photographing point information of the robot, the photographing point information comprising pose information of the robot and posture information of the camera when the camera captures the first image meeting the first preset conditions; correspondingly saving the detection item information and the photographing point information to obtain detection item teaching information of the detection item; Step 105: when the user inputs a next detection item, executing steps 101-104; Step 106: obtaining third operation information of the user for inputting generation of a teaching file, and generating the teaching file, the teaching file containing detection item teaching information corresponding to each detection item; Before the step 106, the method further comprises the following steps: in a process in which the user moves the robot from one photographing point to a next photographing point, obtaining waypoint information of the robot, the waypoint information comprising position information of a plurality of position points of the robot; correspondingly saving the waypoint information and the photographing point information; the teaching file generated in the step 106 contains the waypoint information.

2. The teaching method of a train detection robot according to claim 1, wherein Before the step 104, the method further comprises a step 107: obtaining fourth operation information of the user for labeling on the first image, and generating labeling information; in the step 104, the corresponding saving of the detection item information and the photographing point information to obtain the detection item teaching information of the detection item specifically comprises: correspondingly saving the detection item information, the photographing point information and the labeling information to obtain the detection item teaching information of the detection item.

3. The teaching method of the train detection robot according to claim 1, wherein Before the step 101, the method further comprises a step 108: obtaining a train detection regulation, and generating and displaying a detection item library based on the detection regulation, the detection item library comprising detection items for selection by the user; the step 101 specifically comprises: obtaining first operation information of a detection item selected by the user from the detection item library, and generating detection item information based on the detection item selected by the user.

4. The teaching method of a train detection robot according to claim 1, wherein in the step 102, the obtaining of the first image captured by the camera on the robot and the displaying of the first image specifically comprises: obtaining fifth operation information of the user for real-time display, obtaining a viewfinder image of the camera in a process in which the camera takes a view, and real-time displaying the viewfinder image; Obtaining sixth operation information of a shooting button selected by the user, and controlling the camera to shoot the first image; Obtaining the first image and displaying the first image.

5. The teaching method of a train detection robot according to claim 1, wherein The step 102 of obtaining the first image shot by the camera on the robot and displaying the first image specifically comprises: Obtaining sixth operation information of a shooting button selected by the user, and controlling the camera to shoot the first image; Obtaining the first image and displaying the first image.

6. The teach method of the train detection robot according to claim 1, wherein The step 104 further comprises: Obtaining seventh operation information of pre-processing selected by the user, and pre-processing the first image to obtain a pre-processed image; the pre-processed image is used to assist the user in judging whether the first image meets the first preset condition.

7. A teaching method of a train detection robot characterized by comprising: Comprise: Step 201, the teaching software operator opens the teaching software on the teaching device and communicates with the robot; Step 202, the teaching software operator inputs the detection item on the teaching software; Step 203, the robot operator moves the robot to the detection position and adjusts the position of the robot and the posture of the camera; Step 204, the robot operator controls the camera to take a picture, or the teaching software operator controls the camera to take a picture, to obtain a first image; Step 205, the teaching software obtains the first image and displays it; Step 206, the teaching software operator controls the teaching software to pre-process the first image to obtain a pre-processed image; Step 207, the teaching software operator judges whether the first image meets the requirements based on the first image and the pre-processed image; Step 208, if the requirements are not met, the robot operator is notified to adjust the pose of the robot and the pose of the camera, and then steps 204-206 are executed cyclically until the first image meets the requirements, and the shooting position of the robot that meets the requirements is determined as the shooting point of the detection item; Step 209, if the requirements are met, the teaching software operator labels the first image and generates label information; Step 210, the teaching software operator controls the teaching software to obtain the shooting point information corresponding to the first image, and saves the shooting point information, the detection item information and the label information correspondingly to obtain the detection item teaching information of the detection item; Step 211, the teaching software operator inputs the next detection item, and then executes steps 202-210 cyclically; Step 212, step 211 is executed cyclically until all detection items are taught; Step 213, the robot operator controls the robot to move along a preset route between two shooting points, and selects a plurality of position points as waypoints, and notifies the teaching personnel to obtain and record the waypoint information of each waypoint, the waypoint information including the position information of the waypoint; Step 214, the software operator controls the teaching software to record the shooting point information and the waypoint information correspondingly; Step 215, the software operator controls the teaching software to generate a teaching file, and the teaching file includes the teaching information corresponding to each detection item and the waypoint information.

8. A computer apparatus comprising a memory, a processor, and a computer program stored on the memory, wherein the computer program, when executed by the processor, causes the processor to perform the method of any one of claims 1 to 7. The processor executes the computer program to implement the steps of the method of any one of claims 1-6.

9. A computer readable storage medium having stored thereon computer programs / instructions, characterized in that, The computer program / instructions, when executed by the processor, implement the steps of the method of any one of claims 1-6.

Images