A robot for train inspection and identification control method

Abstract

The disclosed embodiment provides a robot for train inspection and an identification control method, wherein the robot includes a main body, a control device is arranged inside the main body, a walking device is arranged below the main body, and a detection device is arranged on the main body, wherein the detection device includes at least a visual detection device and a sound detection device, wherein the visual detection device is used to identify and locate the relief valve and / or handle on the train carriage, and the sound detection device is used to determine whether the exhaust action of the train is completed. The disclosed embodiment performs ventilation operations and vehicle bottom inspections on railway open cars, replacing manual ventilation and inspection operations, which can reduce personnel safety risks, reduce manual labor intensity, improve the automation level of railway open car ventilation operations, and further improve the unmanned and automated operation level of the tipping machine system.

Description

Technical Field

[0001] The present disclosure relates to the field of robots, and in particular to a robot for train inspection and an identification and control method. Background Art

[0002] To release the brake function of the carriage to be released in a railway open car, it is generally necessary to perform a venting operation. The venting operation here includes two steps: venting and pulling air. Venting refers to opening the corner gate on one side of the train before the train is released, releasing a part of the compressed air stored in the brake main pipe of each vehicle due to the charging effect of the total air of the locomotive, and then closing it; pulling air refers to exhausting the remaining air in the auxiliary air cylinder of each open car by manually pulling the relief valve.

[0003] The venting operation here usually only needs to be done once near the locomotive and is done manually, which takes a short time, and there is not much demand for machines to replace people; while the pulling operation requires people to walk along the open car train, find the relief valve of each open car and perform the pulling operation, which is time-consuming, repetitive, and labor-intensive. In addition, people also need to visually inspect the space between the bottom of the railway open car and the track to prevent foreign objects from affecting the operation of the train.

[0004] The above-mentioned process of pulling air has the following problems when it is done manually: poor working environment and long time consumption: work along the railway line is characterized by all-weather dynamic operation, and shift work is required in open-air places regardless of seasonal climate conditions; heavy-loaded trains are long, and each train usually has about 100 carriages. It takes a long time to perform manual pulling air on the entire train, with an average round trip of more than 1.5 hours; the working environment of the pulling air operation is located along the railway line, and the operator walks along the train. The track foundation "ground" is uneven, and sometimes it is necessary to cross the railway track diagonally, which has certain safety hazards; during the pulling air operation, manual work needs to be performed on each carriage to pull air and visually inspect the bottom of the car, bend down and pull the handle, and bend down and stand up more than a hundred times for each train, which is labor-intensive and easy to get occupational diseases. In addition, although the current types of railway robots have different functions, they are all fixed or track-based trolley robots, which are not suitable for pulling air over long distances (typical trains are up to 1.5 km long) and under harsh working conditions (such as gravel or sleeper track subgrades, sometimes needing to cross the track). Summary of the invention

[0005] The purpose of the embodiments of the present disclosure is to provide a robot for train inspection and an identification and control method to solve the problems existing in the prior art.

[0006] In order to solve the above technical problems, the embodiments of the present disclosure adopt the following technical solutions:

[0007] A robot for train inspection comprises a main body, a control device is arranged inside the main body, a walking device is arranged below the main body, and a detection device is arranged on the main body, wherein the detection device comprises at least a visual detection device and a sound detection device, wherein the visual detection device is used to identify and locate the relief valve and / or handle on the train carriage, and the sound detection device is used to determine whether the exhaust action of the train is completed.

[0008] In some embodiments, the walking device includes four legs, and the four legs are used to achieve walking and the lifting and lowering of the main body in the vertical direction.

[0009] In some embodiments, a robotic arm is disposed on the main body, and an execution device is disposed on the robotic arm.

[0010] The embodiment of the present disclosure further provides an identification control method for a robot, which is used to identify and locate a relief valve and / or a handle on a train carriage. The robot is any one of the robots described above, wherein the visual detection device uses a depth 3D camera, and the identification control method includes:

[0011] Acquire first image data and second image data of the relief valve and / or the handle respectively through two cameras in the depth 3D camera;

[0012] Performing image feature matching based on the first image data, the second image data, and a template image;

[0013] Image depth fusion is performed based on the image data after image feature matching and the parallax data to obtain the three-dimensional position information of the relief valve and / or handle.

[0014] In some embodiments, the image feature matching includes:

[0015] Abstract the image features of the template image and the camera image into feature descriptors;

[0016] Calculate the degree of match between the feature descriptors of the template image and the camera image.

[0017] In some embodiments, the feature descriptor of the template image is generated by:

[0018] Acquire an image that contains only the relief valve and / or handle.

[0019] Detecting the contours of the relief valve and / or the handle using a contour detection algorithm;

[0020] Scan the contour points from any point of the image and sample the contour points to obtain a point set P with a fixed length = (p1, p2, p3, ..., p n ), where pi =(x old ,y old ) are the two-dimensional point coordinates of the contour point;

[0021] Map the coordinates of each point in the point set to an area of ​​fixed size to generate a new point set as the feature descriptor of the image contour in the template image. The expression is P′=(p′1,p′2,…,p′ n ), where p′ i =(x new ,y new ) is the new point coordinate after mapping, and the mapping formula is as follows:

[0022] x new =x old ×W new / W old

[0023] y new =y old ×H new / H old

[0024] Among them, W old ,H old is the width and height of the original image, W new ,H new is the width and height of the mapped image.

[0025] In some embodiments, the feature descriptor of the camera image is generated by:

[0026] Using a sliding window of a predetermined size to perform sliding sampling on the camera image in a predetermined order;

[0027] For each sub-image after sliding sampling, the contour of the relief valve and / or the handle is detected;

[0028] Sample the contour points to obtain the feature descriptor of each sub-image, denoted as As a feature descriptor of the camera image.

[0029] In some embodiments, the degree of matching is calculated by Euclidean distance, which is implemented by the following formula:

[0030]

[0031] The present disclosure provides an identification control method for a robot, for determining whether an exhaust action is completed, wherein the robot is any one of the above-mentioned robots, and the identification control method comprises:

[0032] After removing the noise signal and the power frequency signal preprocessing, the MFCC Mel-frequency cepstral coefficient technology is used to extract audio features to reduce the dimension of the sound data and retain the effective features of the sound signal;

[0033] Based on the extracted audio features, deep learning technology is used to obtain the exhaust sound features;

[0034] The acquired exhaust sound characteristics are compared with the preset threshold to determine whether the exhaust action is completed.

[0035] In some embodiments, deep learning technology is used to obtain exhaust sound features, including:

[0036] Obtaining the audio features to be detected and the audio features of the standard exhaust sound;

[0037] Training is performed based on the training model by calculating the feature distance between the audio feature to be detected and the audio feature of the standard exhaust sound;

[0038] The exhaust sound characteristics are obtained based on the training results.

[0039] Compared with the prior art, the disclosed embodiments are equipped with high-precision sensors, intelligent controllers and actuators such as binocular depth cameras, laser radars, voiceprint collectors, and intelligent robotic arms, and integrate railway open car ventilation and vehicle bottom inspection robots based on artificial intelligence technology to perform ventilation operations and vehicle bottom inspections on railway open cars, replacing manual ventilation and inspection operations. This can reduce personnel safety risks, reduce manual labor intensity, improve the level of automation in railway open car ventilation operations, and further improve the unmanned and automated operation level of the tipping machine system; further promote the intelligent construction of factory areas and improve the level of factory management. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0041] Figure 1 A schematic diagram of the movement of a robot for train inspection according to an embodiment of the present disclosure;

[0042] Figure 2 Schematic diagram of the structure of a robot for train inspection according to an embodiment of the present disclosure;

[0043] Figure 3 is a schematic diagram of visual detection in a robot for train inspection according to an embodiment of the present disclosure;

[0044] Figure 4 is a schematic diagram of visual detection in a robot for train inspection according to an embodiment of the present disclosure;

[0045] Figure 5 The figure is a schematic diagram of visual detection in a robot used for train inspection according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0046] Various aspects and features of the present disclosure are described herein with reference to the drawings.

[0047] It should be understood that various modifications may be made to the embodiments of the present application. Therefore, the above description should not be considered as limiting, but only as an example of an embodiment. Other modifications within the scope and spirit of the present disclosure will occur to those skilled in the art.

[0048] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the general description of the present disclosure given above and the detailed description of the embodiments given below, serve to explain the principles of the present disclosure.

[0049] These and other characteristics of the present disclosure will become apparent from the following description of a preferred form of embodiment given as a non-limiting example, with reference to the attached drawings.

[0050] It should also be understood that, although the present disclosure has been described with reference to some specific examples, those skilled in the art will be able to realize many other equivalent forms of the present disclosure that have the features described in the claims and are therefore within the scope of protection defined thereby.

[0051] The above and other aspects, features and advantages of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings.

[0052] Specific embodiments of the present disclosure are described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments claimed are merely examples of the present disclosure, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to avoid obscuring the present disclosure with unnecessary or redundant details. Therefore, the specific structural and functional details claimed herein are not intended to be limiting, but merely serve as a basis and representative basis for the claims to teach those skilled in the art to use the present disclosure in a variety of ways with substantially any suitable detailed structure.

[0053] This specification may use the phrases "in one embodiment," "in another embodiment," "in a further embodiment," or "in other embodiments," all of which may refer to one or more of the same or different embodiments according to the present disclosure.

[0054] The first embodiment of the present disclosure provides a robot for train inspection operations, wherein the robot needs to perform ventilation operations and vehicle bottom inspection operations on each carriage in an open train, and can autonomously cruise along the line from a warehouse (such as a car dumper power distribution room) to the entire open train, and automatically return to the departure point, such as Figure 1 As shown, Figure 1 The walking route of the robot is shown, which includes 4 main routes, among which route ① indicates that during the inspection process, the robot first moves from the warehouse (such as the power distribution room of the tipping machine) to the route along the train. In this process, the robot needs to autonomously navigate to the train side; route ② indicates that the robot continues to navigate along the train side to perform wind pulling and undercarriage inspection operations; route ③ indicates that when the robot completes the wind pulling and other operations, it turns around from the rear of the car and still navigates along the train side; route ④ indicates the final route of the robot autonomously navigating from the train side to the warehouse.

[0055] Continue as Figure 1 And refer to Figure 2 As shown, the robot provided by the embodiment of the present disclosure includes a main body 1, wherein the main body 1 can be in the shape of a simulated dog, or other simulated or non-simulated shapes, a control device 10 is arranged in the main body 1, and a walking device 20 is arranged below the main body 1. Considering that the robot needs to walk back and forth along the open car line of the railway for up to about 3 km during operation, and the walking surface is basically a natural path or sleepers, gravel roadbed, and sometimes it is necessary to cross the track diagonally, wheeled or tracked robots are often used in the prior art. Obviously, this type of robot is not suitable in terms of walking stability or power supply endurance. For this reason, the walking device 20 here is arranged to have four legs 21, and the four legs 21 are used to realize, for example, the walking of a dog. The walking device 20 in this way has natural adaptability to the path conditions along the railway line relative to wheeled or tracked robots. In addition, the walking device 20 with the above-mentioned setting can also realize the lifting and lowering of the main body 1 in the vertical direction.

[0056] A mechanical arm 30 is arranged on the main body 1, and an execution device is arranged on the mechanical arm 30. The execution device is used to execute relevant operations when the robot reaches a specified position, and the relevant operations here at least include a pulling operation.

[0057] Furthermore, in order to ensure that the robot can accurately perform relevant operations at a predetermined position, a detection device is provided on the main body 1, wherein the detection device here includes at least a visual detection device and a sound detection device, wherein the visual detection device includes at least a camera device and a laser radar. Considering the uncertainty of the train stop position, length and line, the navigation method based on satellite positioning and mapping SLAM is not suitable in the scenario applicable to this embodiment. Therefore, in this embodiment, the visual detection device is used to realize the recognition of the road and the recognition of the operation position.

[0058] In addition, a power supply device is also provided in the main body 1, and the power supply device can provide power to the walking device 20, the mechanical arm 30, the detection device, etc. The power supply device can be, for example, a rechargeable lithium battery pack, which can support an operating cruising time of up to 2.5 to 3 hours, and support the completion of the pulling and inspection operations of the entire train.

[0059] Among them, when the train passes through the above-mentioned lines ①, ②, ③ and ④ in sequence, the robot uses the detection device to perform road identification and operation position identification. For example, along lines ① and ④, signs can be set up in advance along the way and the robot can be navigated by using image recognition; along lines ② and ③, the robot navigates along the side of the train and performs related operations by combining image recognition and lidar data processing. The operations here can be finding and identifying the carriage relief valve and handle, completing the action of pulling the valve, judging whether the valve exhaust action is completed, and automatically lowering the identification to observe the bottom of the vehicle.

[0060] Specifically, along lines ② and ③, the detection device can use image recognition technology based on binocular depth vision to identify and determine the operating position, where the operating position is determined based on the position of the relief valve and / or handle of the carriage.

[0061] Among them, in the process of identifying the position of the relief valve and / or handle of the carriage, considering that the relief valve and handle of the carriage are usually located on the side bottom surface of the carriage, and are often mixed in various pipelines or steel structures on the side bottom surface of the carriage, their contour shape and color characteristics are not obvious. Whether using laser radar or ordinary image recognition technology, it is difficult to locate the exact position of the relief valve and the handle. Based on this, the embodiment of the present disclosure intends to use the image recognition technology based on binocular depth vision of the visual detection device such as a camera device to realize the detection of the position of the relief valve and / or handle.

[0062] Specifically, pictures taken by ordinary camera devices such as color cameras can see and record all objects within the camera's field of view, but the data recorded does not include the distance of these objects from the camera. Instead, it can only determine which objects are farther away and which are closer through semantic analysis of the image, and it is impossible to obtain exact data. For this reason, the camera device here uses, for example, a depth 3D camera, and the depth 3D camera here includes two cameras. The depth 3D camera can detect the depth of field distance of the shooting space. Specifically, the image data obtained by the depth 3D camera can accurately obtain the distance of each point in the image data from the camera. In this way, combined with the (x, y) coordinates of the point in the 2D image, the three-dimensional spatial coordinates of each point in the image data can be obtained. In this way, the real scene can be restored through the three-dimensional coordinates, and applications such as scene modeling can be realized.

[0063] The core algorithm for identifying and locating the relief valve and / or handle method here mainly includes handle contour recognition and image processing algorithm based on depth of field information.

[0064] The principle of contour recognition and image processing algorithm of relief valve and / or handle based on depth of field information is as follows Figure 3-Figure 5 As shown, for this purpose, the depth estimation method based on binocular vision provided by the second embodiment of the present disclosure includes:

[0065] S11, respectively obtaining first camera image data (e.g., left image data) and second camera image data (e.g., right image data) of the relief valve and / or handle position through two cameras in the depth 3D camera;

[0066] S12, performing image feature matching based on the first camera image data, the second camera image data and a template image;

[0067] S13, performing image depth fusion based on the image data after feature matching and the parallax data, and finally obtaining the three-dimensional coordinate position information of the relief valve and the handle.

[0068] Specifically, the camera images of the relief valve and / or the handle position are obtained respectively through the two cameras in the depth 3D camera, including the first camera image data (such as the left image data) and the second camera image data (such as the right image data), and further,

[0069] Furthermore, image feature matching is performed based on the first camera image data, the second camera image data and the template image. The image feature matching method mentioned here first abstracts the image features in the image data into feature descriptors x, and then calculates the matching degree between the feature descriptors x, where the matching degree is achieved by calculating the Euclidean distance between the feature descriptors of the template image and the camera image.

[0070] Specifically, the feature descriptor generation process of the template image here is as follows:

[0071] S101, pre-collect an image containing only the relief valve and / or the handle.

[0072] S102: Detect the contour of the relief valve and / or the handle using, for example, a Canny contour detection algorithm.

[0073] S103, scanning contour points from any point of the image, such as the upper left corner, and sampling the contour points to obtain a point set P with a fixed length = (p1, p2, p3, ..., p n ), where p i =(x old ,y old ) are the two-dimensional point coordinates of the contour point.

[0074] S104, mapping the coordinates of each point in the point set to an area of ​​a fixed size to generate a new point set, called the descriptor of the image contour, expressed as P′=(p′1,p′2,…,p′ n ), where p′ i =(x new ,y new ) is the new point coordinate after mapping, and the mapping formula is as follows:

[0075] x new =x old ×W new / W old

[0076] y new =y old ×H new / H old

[0077] Among them, W old ,H old is the width and height of the original image, W new ,H new is the width and height of the mapped image.

[0078] Therefore, the feature descriptor of the template image can be collected and generated in advance, and can be updated or not updated as needed during the actual matching process.

[0079] Furthermore, the feature descriptor generation process of the camera image is as follows:

[0080] S201, such as Figure 5 As shown, a sliding window of a predetermined size is used to slide and sample on the camera image in a predetermined order, such as from left to right, from top to bottom;

[0081] S202, for each sub-image after sliding sampling, the same processing as the template image is performed from S102 to S104. Finally, each sub-image will generate a feature descriptor, denoted as

[0082] Finally, the Euclidean distance between descriptors is calculated:

[0083]

[0084] Here, the sliding window position with the smallest d is selected as the final successful matching position.

[0085] After image feature matching, image depth fusion is performed based on the image data after image feature matching and the parallax data to obtain the three-dimensional position information of the relief valve and / or the handle. Figure 4 As shown, the projection center of the two cameras is set to O L O R The distance between the lines (i.e. the baseline) is b, and the imaging point of any pixel point P in the three-dimensional space passing through the first camera on the left is P L , the imaging point of the second camera on the right is P R According to the principle of rectilinear propagation of light, point P is the intersection of the projection center points of the two cameras and the line connecting the imaging points. L and X R are the distances from the imaging point of the first camera and the second camera to the imaging surface, respectively. Then the parallax of point P between the first camera and the second camera is:

[0086] d = x L -x R

[0087] Two imaging points P L and P R The distance between them is:

[0088]

[0089] According to the theory of similar triangles, we can conclude that:

[0090]

[0091] The distance Z from point P to the center plane of the projection (that is, the depth of field information) can be obtained:

[0092]

[0093] When point P moves in the three-dimensional space, the imaging position of point P under the first camera and the second camera will also change, so the parallax will also change accordingly. From the above formula, we can see that the parallax is proportional to the distance from the point in the three-dimensional space to the projection center plane. Therefore, as long as the parallax of a specific point is known, the depth information of the point can be calculated.

[0094] After obtaining the depth information Z of all pixels, where Z is the value of each pixel in the depth map, it can be represented by an n×m depth matrix, and the coordinates of each pixel in the real world coordinate system can be calculated by the following formula.

[0095] X=(u-Cx)*Z / Fx

[0096] Y=(v-Cy)*Z / Fy

[0097] Among them, u and v are the x and y pixel coordinates corresponding to each point in the depth of field matrix Z, and the ranges are [0,n) and [0,m) respectively. Fx and Fy are parameters related to the zoom focal length of the camera. Cx and Cy are parameters of the camera, which are used to indicate the translation distance.

[0098] Furthermore, after manually performing the air-pulling action, especially pulling the handle of the relief valve, you can hear the exhaust sound from loud to soft. In practice, you need to wait until the compressed air is exhausted before releasing the handle. Based on this, after the relief valve and the handle are accurately positioned by the robot, the sound of exhaust can be effectively identified by the sound detection device, that is, after identifying the sound of compressed air being exhausted, the robot arm 30 can further simulate manual action to pull the handle. The sound detection device here can be, for example, a microphone or a voiceprint sensor.

[0099] Specifically, the robot can obtain live audio through a microphone, where the unprocessed sound audio is multi-dimensional and highly complex data.

[0100] To this end, a method for identifying a sound signal by a sound detection device is provided in a third embodiment of the present disclosure, comprising:

[0101] S21, after removing noise signals and power frequency signals from the audio signal for preprocessing, applying MFCC Mel-frequency cepstral coefficient technology to extract audio features, so as to reduce the dimension of the sound data and retain the effective features of the sound signal;

[0102] S22, further, using deep learning technology, automatically learns and obtains exhaust sound features from the collected data, and can also intelligently perceive the sound target category;

[0103] S23, comparing the acquired exhaust sound characteristics with a preset threshold to determine whether the exhaust action is completed.

[0104] Specifically, an LMS adaptive notch filter can be used to remove noise signals and power frequency signals. The LMS adaptive notch filter method is suitable for monochromatic interference noise, such as single-frequency sine wave noise, and its principle is as follows:

[0105] For example, for an LMS adaptive notch filter with two weight coefficients, the input signal is a superposition signal of a pure speech signal and a single-frequency interference signal, which is expressed as follows:

[0106]

[0107] After sampling, it is expressed as:

[0108]

[0109] in, f0 is the initial value, f s is the sampling frequency, and n is the number of sampling times.

[0110] Furthermore, the reference input signal is set to be a superposition of a standard sine wave and a cosine wave, where:

[0111]

[0112] Here x1 and x2 can form a sine wave y(n) with arbitrary amplitude and phase.

[0113] By adjusting the LMS adaptive coefficient, the reference input signal y(n) is made to be the same as the original input single-frequency interference signal in amplitude and phase, so as to remove the single-frequency interference signal and achieve the effect of notching.

[0114] After filtering to remove noise signals and power frequency signals, the MFCC Mel-frequency cepstral coefficient technology is further used to extract audio features to reduce the dimension of sound data and retain the effective features of the sound signal.

[0115] Specifically, the extraction method of the MFCC Mel-frequency cepstral coefficient technology here includes: first, the preprocessed signal x t After the frequency domain transformation of (τ) is performed, the audio signal to be processed is obtained. The audio signal is convolved with a triangular filter bank distributed according to the Mel scale based on the logarithmic power spectrum, and then the output vector of the triangular filter bank is discrete cosine transformed, so that the first n-dimensional vector obtained becomes the Mel frequency cepstrum coefficient MFCC, and the formula is as follows:

[0116]

[0117] Among them, n is the number of Mel frequency cepstral coefficients MFCC, C t(n) is the nth Mel-frequency cepstral coefficient MFCC coefficient of the tth frame; S t (m) is the logarithmic power spectrum of the audio signal; M is the number of triangular filters.

[0118] Furthermore, the first-order difference is performed on the Mel frequency cepstrum MFCC to obtain the differential coefficient, and the differential coefficient is used as the audio feature, which is expressed as follows:

[0119] Where k is a constant.

[0120] Here, deep learning technology is used to obtain the exhaust sound characteristics, including:

[0121] Obtaining the audio features to be detected and the audio features of the standard exhaust sound;

[0122] Training is performed based on the training model by calculating the feature distance between the audio feature to be detected and the audio feature of the standard exhaust sound;

[0123] The exhaust sound characteristics are obtained based on the training results.

[0124] Specifically, after obtaining the audio features of the audio signal, further, the standard exhaust sound characteristics are used as a benchmark to determine the various parameters in the model, and combined with model training and deep learning, the exhaust audio signal analysis and feature extraction are finally achieved.

[0125] Specifically, after extracting audio features through MFCC, the distance matrix between features is further calculated to construct feature samples for neural network training.

[0126] The audio signal feature matrix is ​​obtained by the above differential coefficient calculation formula. t represents the feature vector of the tth frame. For example, the feature matrix x1=(D1,…,D n ), x2=(D1,…,D n ), one of the above audio signal feature matrices can be set as the audio feature to be detected, and the other audio signal feature matrix can be set as the audio feature of the standard exhaust sound, and the feature distance can be further calculated, which can be expressed as follows in the form of a feature distance matrix:

[0127] Feature distance matrix d = |x1-x2|

[0128] The neural network training process here is as follows Figure 5 As shown, the neural network includes a convolution layer, an excitation layer, a pooling layer and a fully connected layer, wherein, in the convolution layer, a two-dimensional convolution is performed with a two-dimensional matrix I and a two-dimensional convolution kernel A, and the mathematical model of the output matrix G is:

[0129]

[0130] Where: G(a,b) is the pixel value of the output image G at point (a,b); A(c,d) is the parameter value of the convolution kernel A at point (c,d). The parameters of the convolution kernel are obtained by training the model through the learning algorithm.

[0131] The main function of the excitation layer is to perform a nonlinear mapping on the output result of the convolution layer. The model formula of the excitation layer is as follows:

[0132]

[0133] The main functions of the pooling layer here are mainly three points: first, to reduce information redundancy; second, to improve the scale invariance and rotation invariance of the model; third, to prevent overfitting. The pooling layer formula is as follows:

[0134] r=0,1,2,3,…,p-1

[0135] s=0,1,2,3,…,q-1

[0136] y ij =max(x i.t+r,j.t+s )

[0137] i≤(mp) / t

[0138] j≤(nq) / t

[0139] The fully connected layer here is located at the end of the neural network. Each node of the fully connected layer is interconnected with all the nodes of the previous layer, so as to integrate the features extracted by the previous layer network and map these features to the sample label space. The fully connected layer here performs weighted summation on the features output by the previous layer and inputs the result into the activation function to finally complete the classification of the target. The weighted summation calculation formula is as follows:

[0140]

[0141] Among them, w i is the weight coefficient in the fully connected layer, x i is the value of the i-th neuron in the previous layer, b i is the bias of the fully connected layer.

[0142] The neural network finally normalizes the result to between (0, 1) through the activation function, and then compares it with the manually set threshold to determine whether the exhaust action is completed. The larger the normalized value is, the closer it is to the positive sample, that is, the greater the probability that the exhaust action has been completed. The judgment formula is as follows:

[0143]

[0144] In addition, after completing the wind-pulling operation, the robot then performs actions such as "squatting and observing". For example, the robot with a simulated dog form can realize the standard squatting function of the robot through the walking device 20. Specifically, when the robot performs the squatting action, the height of the camera device located at the head of the robot is between the ground and the bottom of the carriage. Since the camera device can rotate left and right, and its rotation range can reach 180 degrees, the viewing angle can completely cover the space under the carriage, so that it is suitable for observing the situation in the space under the carriage. In addition, the robot can also use image recognition technology to timely identify foreign objects appearing in the space under the carriage (especially near the track) and give an alarm in time.

[0145] The disclosed embodiments are equipped with high-precision sensors, intelligent controllers and actuators such as binocular depth cameras, laser radars, voiceprint collectors, and intelligent robotic arms, and integrate railway open car ventilation and vehicle bottom inspection robots based on artificial intelligence technology to perform ventilation operations and vehicle bottom inspections on railway open cars, replacing manual ventilation and inspection operations. This can reduce personnel safety risks, alleviate manual labor intensity, improve the level of automation in railway open car ventilation operations, and further improve the unmanned and automated operation level of the tipping machine system; further promote the intelligent construction of factory areas and improve the level of factory area management.

[0146] Another embodiment of the present disclosure relates to an identification control device for a robot, which is used to identify and locate a relief valve and / or a handle on a train carriage. The robot is any one of the above-mentioned robots, wherein the visual detection device uses a deep 3D camera, and the identification control device includes an image data acquisition module, a matching module, and a position information acquisition module coupled to each other, wherein:

[0147] The image data acquisition module is used to respectively acquire first image data and second image data of the relief valve and / or the handle through two cameras in the depth 3D camera;

[0148] The matching module is used to perform image feature matching based on the first image data, the second image data and the template image;

[0149] The position information acquisition module is used to perform image depth fusion based on the image data after image feature matching and the parallax data to obtain the three-dimensional position information of the relief valve and / or the handle.

[0150] Another embodiment of the present disclosure relates to an identification control device for a robot, used to determine whether an exhaust action is completed, the robot is any one of the above-mentioned robots, the identification control device comprises an extraction module, an acquisition module and a judgment module coupled to each other, wherein:

[0151] The extraction module is used to extract audio features by applying MFCC Mel-frequency cepstral coefficient technology after removing noise signals and power frequency signals from the audio signals, so as to reduce the dimension of the sound data and retain the effective features of the sound signal;

[0152] The acquisition module is used to acquire exhaust sound features by using deep learning technology based on the extracted audio features;

[0153] The judgment module is used to compare the acquired exhaust sound characteristics with a preset threshold to determine whether the exhaust action is completed.

[0154] The disclosed embodiments are equipped with high-precision sensors, intelligent controllers and actuators such as binocular depth cameras, laser radars, voiceprint collectors, and intelligent robotic arms, and integrate railway open car ventilation and vehicle bottom inspection robots based on artificial intelligence technology to perform ventilation operations and vehicle bottom inspections on railway open cars, replacing manual ventilation and inspection operations. This can reduce personnel safety risks, alleviate manual labor intensity, improve the level of automation in railway open car ventilation operations, and further improve the unmanned and automated operation level of the tipping machine system; further promote the intelligent construction of factory areas and improve the level of factory area management.

[0155] Another embodiment of the present disclosure provides a storage medium, which is a computer-readable medium and stores a computer program. When the computer program is executed by a processor, the method provided by the first embodiment of the present disclosure is implemented, including the following steps S11 to S13 and S14-S16:

[0156] S11, respectively acquiring first image data and second image data of the relief valve and / or the handle through two cameras in the depth 3D camera;

[0157] S12, performing image feature matching based on the first image data, the second image data, and a template image;

[0158] S13, performing image depth fusion based on the image data after image feature matching and the parallax data to obtain the three-dimensional position information of the relief valve and / or the handle.

[0159] as well as

[0160] S14, after removing noise signals and power frequency signals from the audio signals for preprocessing, applying MFCC Mel-frequency cepstrum coefficient technology to extract audio features, so as to reduce the dimension of the sound data and retain the effective features of the sound signal;

[0161] S15, using deep learning technology to obtain exhaust sound features based on the extracted audio features;

[0162] S16, comparing the acquired exhaust sound characteristics with a preset threshold to determine whether the exhaust action is completed.

[0163] Furthermore, when the computer program is executed by the processor, other methods provided in the first embodiment of the present disclosure are implemented.

[0164] The disclosed embodiments are equipped with high-precision sensors, intelligent controllers and actuators such as binocular depth cameras, laser radars, voiceprint collectors, and intelligent robotic arms, and integrate railway open car ventilation and vehicle bottom inspection robots based on artificial intelligence technology to perform ventilation operations and vehicle bottom inspections on railway open cars, replacing manual ventilation and inspection operations. This can reduce personnel safety risks, alleviate manual labor intensity, improve the level of automation in railway open car ventilation operations, and further improve the unmanned and automated operation level of the tipping machine system; further promote the intelligent construction of factory areas and improve the level of factory area management.

[0165] Another embodiment of the present disclosure provides an electronic device, which includes at least a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the method provided by any embodiment of the present disclosure when executing the computer program in the memory. Exemplarily, the steps of the electronic device computer program are as follows S21 to S23 and S24 to S26:

[0166] S21, respectively acquiring first image data and second image data of the relief valve and / or the handle through two cameras in the depth 3D camera;

[0167] S22, performing image feature matching based on the first image data, the second image data, and a template image;

[0168] S23, performing image depth fusion based on the image data after image feature matching and the parallax data to obtain the three-dimensional position information of the relief valve and / or the handle.

[0169] as well as

[0170] S24, after removing noise signals and power frequency signals from the audio signals for preprocessing, applying MFCC Mel-frequency cepstral coefficient technology to extract audio features, so as to reduce the dimension of the sound data and retain the effective features of the sound signal;

[0171] S25, using deep learning technology to obtain exhaust sound features based on the extracted audio features;

[0172] S26, comparing the acquired exhaust sound characteristics with a preset threshold to determine whether the exhaust action is completed.

[0173] Furthermore, the processor also executes the computer program in the above embodiment

[0174] The disclosed embodiments are equipped with high-precision sensors, intelligent controllers and actuators such as binocular depth cameras, laser radars, voiceprint collectors, and intelligent robotic arms, and integrate railway open car ventilation and vehicle bottom inspection robots based on artificial intelligence technology to perform ventilation operations and vehicle bottom inspections on railway open cars, replacing manual ventilation and inspection operations. This can reduce personnel safety risks, alleviate manual labor intensity, improve the level of automation in railway open car ventilation operations, and further improve the unmanned and automated operation level of the tipping machine system; further promote the intelligent construction of factory areas and improve the level of factory area management.

[0175] The storage medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

[0176] The storage medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains at least two Internet Protocol addresses; sends a node evaluation request including at least two Internet Protocol addresses to a node evaluation device, wherein the node evaluation device selects an Internet Protocol address from the at least two Internet Protocol addresses and returns it; receives the Internet Protocol address returned by the node evaluation device; wherein the obtained Internet Protocol address indicates an edge node in a content distribution network.

[0177] Alternatively, the storage medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device: receives a node evaluation request including at least two Internet Protocol addresses; selects an Internet Protocol address from the at least two Internet Protocol addresses; and returns the selected Internet Protocol address; wherein the received Internet Protocol address indicates an edge node in a content distribution network.

[0178] Computer program code for performing operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the passenger computer, partially on the passenger computer, as a separate software package, partially on the passenger computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the passenger computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0179] It should be noted that the storage medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any storage medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0180] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0181] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.

[0182] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0183] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0184] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0185] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0186] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

[0187] Multiple embodiments of the present disclosure are described in detail above, but the present disclosure is not limited to these specific embodiments. Those skilled in the art can make various variations and modifications to the embodiments based on the concepts of the present disclosure, and these variations and modifications should all fall within the scope of protection required by the present disclosure.

[0188] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0189] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

[0190] Multiple embodiments of the present disclosure are described in detail above, but the present disclosure is not limited to these specific embodiments. Those skilled in the art can make various variations and modifications to the embodiments based on the concepts of the present disclosure, and these variations and modifications should all fall within the scope of protection required by the present disclosure.

Claims

1. A recognition control method for a robot, used for identifying and locating a relief valve and / or a handle on a train carriage, wherein the robot comprises a main body, a control device is arranged inside the main body, and a walking device is arranged below the main body, characterized in that: A detection device is provided on the main body, and the detection device at least includes a visual detection device and a sound detection device, wherein the visual detection device is used to identify and locate the relief valve and / or handle on the train carriage, and the sound detection device is used to determine whether the exhaust action of the train is completed, wherein the visual detection device adopts a deep 3D camera, and is characterized in that the recognition control method includes: Acquire first image data and second image data of the relief valve and / or the handle respectively through two cameras in the depth 3D camera; Performing image feature matching based on the first image data, the second image data, and the template image, wherein the image feature matching includes: abstracting the image features of the template image and the camera image into feature descriptors, and calculating the matching degree between the feature descriptors of the template image and the camera image; Perform image depth fusion based on the image data after image feature matching and the parallax data to obtain the three-dimensional position information of the relief valve and / or the handle; The feature descriptor of the template image is generated in the following manner: Acquire images containing only the relief valve and / or handle; Detecting the contours of the relief valve and / or the handle using a contour detection algorithm; Scan the contour points from any point of the image and sample the contour points to obtain a point set P with a fixed length = (p1, p2, p3, ..., p n ), where p i =(x old ,y old ) are the two-dimensional point coordinates of the contour point; Map the coordinates of each point in the point set to a fixed-size area to generate a new point set as the feature descriptor of the image contour in the template image. The expression is P ′ =(p ′ 1,p ′ 2,…,p ′ n ), where p i ′ =(x new ,y new ) is the new point coordinate after mapping, and the mapping formula is as follows: x new =x old ×W new / W old y new =y old ×H new / H old Among them, W old ,H old is the width and height of the original image, W new ,H new is the width and height of the mapped image.

2. The identification control method according to claim 1, characterized in that: The walking device comprises four legs, through which walking and the lifting and lowering of the main body in the vertical direction are realized.

3. The identification control method according to claim 1, characterized in that: A mechanical arm is arranged on the main body, and an execution device is arranged on the mechanical arm.

4. The identification control method according to claim 1, characterized in that: The feature descriptor of the camera image is generated in the following way: Using a sliding window of a predetermined size to perform sliding sampling on the camera image in a predetermined order; For each sub-image after sliding sampling, the contour of the relief valve and / or the handle is detected; Sample the contour points to obtain the feature descriptor of each sub-image, denoted as As a feature descriptor of the camera image.

5. The identification control method according to claim 4, characterized in that: The degree of matching is calculated by Euclidean distance, which is achieved by the following formula:

6. A recognition control method for a robot, used to determine whether an exhaust action is completed, the robot comprising a main body, a control device arranged inside the main body, and a walking device arranged below the main body, characterized in that: A detection device is provided on the main body, and the detection device at least includes a visual detection device and a sound detection device, wherein the visual detection device is used to identify and locate the relief valve and / or handle on the train carriage, and the sound detection device is used to determine whether the exhaust action of the train is completed, characterized in that the identification control method includes: After removing the noise signal and the power frequency signal preprocessing, the MFCC Mel-frequency cepstral coefficient technology is used to extract audio features to reduce the dimension of the sound data and retain the effective features of the sound signal; Based on the extracted audio features, deep learning technology is used to obtain the exhaust sound features; The acquired exhaust sound characteristics are compared with the preset threshold to determine whether the exhaust action is completed.

7. The identification control method according to claim 6, characterized in that: Deep learning technology is used to obtain exhaust sound features, including: Obtaining the audio features to be detected and the audio features of the standard exhaust sound; Training is performed based on the training model by calculating the feature distance between the audio feature to be detected and the audio feature of the standard exhaust sound; The exhaust sound characteristics are obtained based on the training results.

Images