Train car dumper unloading intelligent robot automatic car clearing system and method

Through computer vision and artificial intelligence technology, the water outlet pressure in the train compartment is adaptively adjusted, which solves the problem of low efficiency and safety hazards of material residue cleaning in the unloading of the train compartment rollover, and achieves an efficient and safe cleaning effect.

CN118458413BActive Publication Date: 2024-11-05SHANDONG HAIDA ROBOT TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410878244.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-02
Publication Date
2024-11-05
Estimated Expiration
2044-07-02

AI Technical Summary

Technical Problem

In the prior art, material residues during unloading of train car overturners lead to low cleaning efficiency, and manual cleaning poses safety hazards.

Method used

Using computer vision technology and artificial intelligence technology, the residual material distribution state image is collected through the visual guidance system, and the image acquisition module, the state feature extraction module, the embedded fusion encoding module and the water pressure recommendation module are used to adaptively adjust the water outlet pressure value to clean the residual material. .

Benefits of technology

It improves cleaning efficiency, reduces manual intervention, ensures safety, and adaptively adjusts water pressure according to different dirt types to meet cleaning needs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118458413B_ABST
    Figure CN118458413B_ABST
Patent Text Reader

Abstract

The present invention discloses an automatic car clearing system and method of a car dumper unloading intelligent robot, which mainly relates to the field of intelligent robots. The system comprises: a visual guidance system, a six-axis robot and an automatic water supply system; the visual guidance system is installed at the end of the six-axis robot; the automatic water supply system comprises a water gun, an insulated water tank, a regulating water pump, a water level controller and a water supply controller, wherein the water supply controller is communicatively connected to the visual guidance system, and can use computer vision technology and artificial intelligence technology to analyze the state of residual materials and the distribution state images of obstacles in the car collected by the visual guidance system, so as to complete the car clearing operation according to the actual residual material distribution state characteristic information in the car.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of intelligent robots, and more specifically, to an automatic clearing system and method for an intelligent robot for unloading train carriage tippers. Background Art

[0002] Railway train coal transportation is the main mode of bulk coal transportation due to its advantages of high transportation efficiency, low cost of long-distance and large-volume transportation, energy saving and environmental protection. The car dumper unloading method is a convenient and fast unloading method after the train transportation arrives. It is mainly used to dump loose materials such as coal and ore loaded in railway general open carriages. However, due to material humidity, bumps during transportation and other reasons, the four corners of the bottom of the carriage and the inner wall of the carriage are very likely to form compaction. That is, during the unloading process, the material will remain on the side wall of the carriage and cannot be unloaded smoothly.

[0003] In order to solve this problem, the current method is manual cleaning. However, due to the large number of carriages, heavy workload, and low cleaning efficiency, the efficiency of unloading and cleaning in this way is relatively low. At the same time, the manual cleaning method also has safety hazards. Therefore, an automatic cleaning system and method of an intelligent robot for unloading train carriage tippers are expected. Summary of the invention

[0004] In order to solve the above technical problems, the present application is proposed. The present application provides a train car dumper unloading intelligent robot automatic car cleaning system and method thereof, which can use computer vision technology and artificial intelligence technology to analyze the state of residual materials on the residual material distribution state image collected by the visual guidance system, so as to adaptively adjust the water outlet pressure value according to the actual residual material distribution state characteristic information in the car.

[0005] According to one aspect of the present application, there is provided a train car dumper unloading intelligent robot automatic car clearing system, which includes: a visual guidance system, a six-axis robot and an automatic water supply system;

[0006] The visual guidance system is installed at the end of the six-axis robot;

[0007] The automatic water supply system includes a water gun, an insulated water tank, a regulating water pump, a water level controller and a water supply controller, wherein the water supply controller is communicatively connected to the visual guidance system and performs adaptive adjustment of the water outlet pressure based on information collected by the visual guidance system.

[0008] In the above-mentioned train car dumper unloading intelligent robot automatic car cleaning system, the water supply controller includes:

[0009] An image acquisition module, used for acquiring an image of the residual material distribution state acquired by a visual guidance system;

[0010] A state feature extraction module, used for extracting material state features from the residual material distribution state image to obtain a plurality of residual material state semantic feature vectors;

[0011] an embedding fusion coding module, used for embedding and fusion of position information of the plurality of residual material state semantic feature vectors to obtain a plurality of position embedded residual material state semantic feature vectors; and

[0012] The water pressure recommendation module is used to determine the recommended water pressure value based on the residual material global state information of the residual material state semantic feature vectors embedded in the multiple positions.

[0013] In the above-mentioned train car dumper unloading intelligent robot automatic car clearing system, the state feature extraction module includes:

[0014] an area of ​​interest extraction unit, configured to extract an area of ​​interest for residual materials from the residual material distribution state image to obtain a plurality of images of the area of ​​interest for residual materials; and

[0015] An image feature extraction unit is used to extract image features from the multiple residual material region of interest images using a deep learning network model to obtain the multiple residual material state semantic feature vectors.

[0016] In the above-mentioned train car dumper unloading intelligent robot automatic car clearing system, the region of interest extraction unit is used to:

[0017] The residual material distribution state image is passed through a residual material region of interest extractor based on an R-CNN network to obtain the multiple residual material region of interest images.

[0018] In the above-mentioned train car dumper unloading intelligent robot automatic car clearing system, the image feature extraction unit is used to:

[0019] The multiple residual material region of interest images are passed through a residual material state feature extractor based on a convolutional neural network model to obtain the multiple residual material state semantic feature vectors.

[0020] In the above-mentioned train car dumper unloading intelligent robot automatic car clearing system, the embedded fusion coding module includes:

[0021] an encoding unit, configured to extract position information of the plurality of residual material region of interest images in the residual material distribution state image, and respectively encode the position information to obtain a plurality of region of interest position information encoding feature vectors; and

[0022] A fusion unit is used to process the multiple residual material state semantic feature vectors and the multiple interest region position information encoding feature vectors using a position information embedding fusion module to obtain the multiple position embedded residual material state semantic feature vectors.

[0023] In the above-mentioned train car dumper unloading intelligent robot automatic car clearing system, the fusion unit is used to:

[0024] Each of the region of interest position information encoding feature vectors is concatenated at the end of the corresponding residual material state semantic feature vector to obtain the plurality of position embedded residual material state semantic feature vectors.

[0025] In the above-mentioned train car dumper unloading intelligent robot automatic car clearing system, the water pressure recommendation module includes:

[0026] An attention integration unit, used for embedding the plurality of positions into the residual material state semantic feature vector through a probability-based attention integration model to obtain a residual material global distribution state feature vector;

[0027] The water pressure recommendation unit is used to pass the residual material global distribution state feature vector through a decoder-based water pressure recommender to obtain a decoded value, and the decoded value is used to represent the recommended water pressure value.

[0028] In the above-mentioned train car dumper unloading intelligent robot automatic car clearing system, the attention integration unit is used to:

[0029] The semantic feature vectors of the residual material state embedded in the multiple positions are processed by the following attention integration formula to obtain the feature vector of the residual material global distribution state; wherein the attention integration formula is:

[0030] ;

[0031] in, For the The attention score coefficient, For the The attention score coefficient, The first position of the residual material state semantic feature vector is embedded in the plurality of positions. The positions are embedded with the semantic feature vector of the residual material state. For the The positions are embedded with the semantic feature vector of the residual material state. For the A weight coefficient matrix, For the A vector of weight coefficients, For the A bias vector, represents the hyperbolic tangent function processing, represents the exponential function processing with e as the base, For the The attention weight coefficient, For the The attention weight coefficient, is the length of the semantic feature vectors of the residual material states embedded in the multiple positions, represents a cascade function, is the global distribution state characteristic vector of the residual material.

[0032] According to another aspect of the present application, a method for automatically clearing a train car by an intelligent robot for unloading a train car dumper is provided, comprising:

[0033] Acquire the residual material distribution state image collected by the vision guidance system;

[0034] Performing material state feature extraction on the residual material distribution state image to obtain a plurality of residual material state semantic feature vectors;

[0035] Performing position information embedding fusion on the multiple residual material state semantic feature vectors to obtain multiple position embedded residual material state semantic feature vectors; and

[0036] The recommended water pressure value is determined based on the residual material global state information of the residual material state semantic feature vectors embedded in the multiple positions.

[0037] Compared with the prior art, the present application provides a train car dumper unloading intelligent robot automatic car cleaning system and method thereof, the system includes: a visual guidance system, a six-axis robot and an automatic water supply system; the visual guidance system is installed at the end of the six-axis robot; the automatic water supply system includes a water gun, an insulated water tank, a regulating water pump, a water level controller and a water supply controller, wherein the water supply controller is communicatively connected to the visual guidance system, and performs adaptive adjustment of the water outlet pressure based on the information collected by the visual guidance system. In this way, the water outlet pressure value can be adaptively adjusted according to the actual residual material distribution state characteristic information in the car, so as to meet different cleaning needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. The following drawings are not intentionally scaled to the actual size, and the focus is on illustrating the main purpose of the present application.

[0039] Figure 1It is a block diagram of the water supply controller in the automatic car clearing system of the train car tipping machine unloading intelligent robot according to the embodiment of the present application.

[0040] Figure 2 It is a block diagram of the state feature extraction module in the automatic car clearing system of the train car tipping machine unloading intelligent robot according to the embodiment of the present application.

[0041] Figure 3 It is a block diagram of the embedded fusion coding module in the automatic car clearing system of the train car tipping machine unloading intelligent robot according to the embodiment of the present application.

[0042] Figure 4 It is a block diagram of the water pressure recommendation module in the automatic car clearing system of the train car tipper unloading intelligent robot according to the embodiment of the present application.

[0043] Figure 5 This is a flow chart of the automatic car clearing method of the intelligent robot for unloading of train car tipping machine according to the embodiment of the present application.

[0044] Figure 6 This is an application scenario diagram of the automatic car clearing system of the intelligent robot for unloading train car tipping machine according to the embodiment of the present application. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work also fall within the scope of protection of the present application.

[0046] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0047] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0048] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0049] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0050] Due to the large number of carriages, heavy workload, and low cleaning efficiency, this method of manual cleaning has the problem of relatively low unloading and cleaning efficiency. At the same time, the manual cleaning method also has safety hazards. The present application provides an automatic car cleaning system for a train car tipper unloading intelligent robot, which specifically includes: a visual guidance system, a six-axis robot, and an automatic water supply system; wherein the visual guidance system is installed at the end of the six-axis robot; the automatic water supply system includes a water gun, an insulated water tank, a regulating water pump, a water level controller, and a water supply controller. More specifically, the water supply controller is communicatively connected to the visual guidance system, and performs adaptive adjustment of the water outlet pressure based on the information collected by the visual guidance system.

[0051] It should be understood that it is very important to adjust and control the water outlet pressure during the car cleaning process. This is because the materials transported in the train carriages are diverse, which may lead to different types and degrees of dirt in the carriages. There are different cleaning requirements for these different dirts, and different cleaning requirements require different water outlet pressures to cooperate with the cleaning of dirt. For example, removing stubborn dirt may require a higher water pressure to ensure the cleaning effect. However, too high water pressure may damage the surface of the train carriage. In addition, too high water pressure may also mean that a lot of water resources will be wasted. At present, intelligent car cleaning systems usually use a fixed water outlet pressure for car cleaning, which may lead to problems such as the inability to effectively remove dirt, damage to the surface of the carriage, and waste of water resources.

[0052] In response to the above technical problems, the technical concept of the present application is: to use computer vision technology and artificial intelligence technology to analyze the status of residual materials on the residual material distribution state image collected by the visual guidance system, so as to adaptively adjust the water outlet pressure value according to the actual residual material distribution state characteristic information in the car, so as to meet different cleaning needs.

[0053] Based on this, Figure 1 FIG. 1 is a block diagram of the water supply controller in the automatic car cleaning system of the train car dumper unloading intelligent robot according to an embodiment of the present application. Figure 1 As shown, according to the automatic car clearing system of the intelligent robot for unloading of train car tippers according to the embodiment of the present application, the water supply controller 100 includes: an image acquisition module 110, used to obtain the residual material distribution state image acquired by the visual guidance system; a state feature extraction module 120, used to extract the material state features of the residual material distribution state image to obtain a plurality of residual material state semantic feature vectors; an embedded fusion coding module 130, used to embed and fuse the position information of the plurality of residual material state semantic feature vectors to obtain a plurality of position embedded residual material state semantic feature vectors; and a water pressure recommendation module 140, used to determine the recommended water pressure value based on the global state information of the residual material of the plurality of position embedded residual material state semantic feature vectors.

[0054] Specifically, in the technical solution of the present application, the specific encoding process of the water supply controller includes: first, obtaining the residual material distribution state image collected by the visual guidance system. Wherein, the visual guidance system includes a visual sensor, wherein the visual sensor is a sensor device that can perceive and capture visual information, such as a camera, which is usually used to obtain image data of the surrounding environment. In this way, by collecting the residual material distribution state image through the visual sensor, the distribution of residual materials in the car can be provided, providing accurate data support for subsequent cleaning operations. The image acquisition module can also be used to obtain the obstacle distribution state image in the car collected by the visual guidance system. The automatic car cleaning system can determine the working path and range of the robot based on the obstacle distribution state image in the car.

[0055] Considering that when analyzing the distribution of residual materials in the car and adjusting and controlling the water outlet pressure value based on the real-time distribution status, more attention should be paid to the residual material area in the residual material distribution state image, while reducing excessive attention to the surrounding useless background information. Therefore, in the technical solution of the present application, the residual material distribution state image is passed through a residual material region of interest extractor based on the R-CNN network to obtain multiple residual material region of interest images. Among them, R-CNN (Region-based Convolutional Neural Network) is a classic target detection algorithm, which locates and selects the area in the image that may contain the target object by selecting and sliding the candidate area in the image.

[0056] Next, the multiple residual material region of interest images are passed through a residual material state feature extractor based on a convolutional neural network model to obtain multiple residual material state semantic feature vectors. Among them, the convolutional neural network (CNN) is a deep learning model specifically used to process data with a grid-like topological structure, such as images and videos. Through components such as convolutional layers, pooling layers, and activation layers, CNN can effectively learn features in images and achieve good performance in tasks such as image classification and target detection. In the technical solution of the present application, the residual material state feature extractor is constructed by a convolutional neural network model to automatically learn abstract features in each of the residual material region of interest images through multi-layer convolution and pooling operations, including edge, texture, shape and other information.

[0057] Accordingly, if Figure 2 As shown, the state feature extraction module 120 includes: an interest region extraction unit 121, used to extract the residual material interest region of the residual material distribution state image to obtain a plurality of residual material interest region images; and an image feature extraction unit 122, used to perform image feature extraction on the plurality of residual material interest region images using a deep learning network model to obtain the plurality of residual material state semantic feature vectors.

[0058] In one example, the region of interest extraction unit 121 is used to: pass the residual material distribution state image through a residual material region of interest extractor based on an R-CNN network to obtain the multiple residual material region of interest images.

[0059] In one example, the image feature extraction unit 122 is used to: pass the multiple residual material region of interest images through a residual material state feature extractor based on a convolutional neural network model to obtain the multiple residual material state semantic feature vectors.

[0060] At the same time, the position information of the multiple residual material region of interest images in the residual material distribution state image is extracted, and the position information is respectively encoded to obtain multiple region of interest position information encoding feature vectors. Among them, extracting the position information of each local residual material region of interest image can locate the residual material in the residual material distribution state image, so as to understand the spatial layout relationship between each local residual material region of interest image, so as to more comprehensively describe and characterize the distribution status of the residual material. In the technical solution of the present application, encoding the position information to obtain each region of interest position information encoding feature vector can make the position information and the image semantic representation of each local residual material region of interest image have the same representation method and form, which is convenient for subsequent feature fusion and analysis.

[0061] Subsequently, the multiple residual material state semantic feature vectors and the multiple interest region position information encoding feature vectors are processed using a position information embedding fusion module to obtain multiple position embedded residual material state semantic feature vectors. In this way, through further embedding of position information, each of the position embedded residual material state semantic feature vectors can combine the semantic features of the residual material state with the position information, thereby enhancing the spatial perception of the residual material distribution status, so that the model can fully understand the state information and spatial distribution structure of the residual material.

[0062] Accordingly, if Figure 3 As shown, the embedded fusion coding module 130 includes: a coding unit 131, used to extract the position information of the multiple residual material region of interest images in the residual material distribution state image, and encode the position information respectively to obtain multiple region of interest position information coding feature vectors; and a fusion unit 132, used to use the position information embedding fusion module to process the multiple residual material state semantic feature vectors and the multiple region of interest position information coding feature vectors to obtain the multiple position embedded residual material state semantic feature vectors.

[0063] In one example, the fusion unit 132 is used to: concatenate each of the region of interest position information encoding feature vectors at the end of the corresponding residual material state semantic feature vector to obtain the plurality of position embedded residual material state semantic feature vectors.

[0064] It is worth mentioning that the position information embedding fusion module is a computer vision module used to embed the position information of the region of interest in the image into the semantic features of the image. The position information embedding fusion module can improve the model's understanding of the position and shape of objects in the image by combining position information with semantic features. This is particularly useful for the following tasks: The position information embedding fusion module usually contains two components: an encoding unit: extracts the position information of the region of interest in the image and encodes it into a feature vector; a fusion unit: fuses the position information feature vector with the semantic feature vector of the image to create a new position embedded semantic feature vector. The fusion unit can adopt different methods, such as: splicing the position information feature vector at the end of the semantic feature vector; using an attention mechanism to weight the position information feature vector into the semantic feature vector. By embedding the position information into the semantic features, the model can better understand the shape and position of objects in the image, thereby improving the accuracy of target detection and segmentation tasks.

[0065] Further, the semantic feature vectors of residual material state embedded in the multiple positions are passed through a probability-based attention integration model to obtain a feature vector of the global distribution state of residual materials. That is, the full spatiotemporal autocorrelation in the semantic feature vectors of residual material state embedded in the multiple positions is obtained through the attention integration model, capturing the long-term dependency and spatial dynamic distribution characteristics in the data. Among them, the essence of the attention integration model is a weighted probability distribution mechanism, that is, a greater weight is assigned to important content and a reduced weight is given to other content. Such a mechanism is more focused on finding useful information in the input data that is significantly related to the current data, and discovering the autocorrelation between the semantic feature vectors of residual material state embedded in each position. Here, the attention integration model can learn the dependency and importance between the semantic feature vectors of residual material state embedded in each position in the semantic feature vectors of residual material state embedded in the multiple positions and the overall residual material state information, and selectively emphasize the more representative regional information based on this dependency and importance, thereby forming an expression and characterization of the global distribution state of residual materials. Then, the feature vector of the global distribution state of residual materials is passed through a water pressure recommender based on a decoder to obtain a decoded value, and the decoded value is used to represent the recommended water pressure value.

[0066] Accordingly, if Figure 4 As shown, the water pressure recommendation module 140 includes: an attention integration unit 141, used to embed the multiple positions into the residual material state semantic feature vector through a probability-based attention integration model to obtain a residual material global distribution state feature vector; and a water pressure recommendation unit 142, used to pass the residual material global distribution state feature vector through a decoder-based water pressure recommender to obtain a decoded value, and the decoded value is used to represent the recommended water pressure value.

[0067] In one example, the attention integration unit 141 is used to process the semantic feature vectors of the residual material state embedded in the multiple positions using the following attention integration formula to obtain the residual material global distribution state feature vector; wherein the attention integration formula is:

[0068] ;

[0069] in, For the The attention score coefficient, For the The attention score coefficient, The first position of the residual material state semantic feature vector is embedded in the plurality of positions. The positions are embedded with the semantic feature vector of the residual material state. For the The positions are embedded with the semantic feature vector of the residual material state. For the A weight coefficient matrix, For the A vector of weight coefficients, For the A bias vector, represents the hyperbolic tangent function processing, represents the exponential function processing with e as the base, For the The attention weight coefficient, For the The attention weight coefficient, is the length of the semantic feature vectors of the residual material states embedded in the multiple positions, represents a cascade function, is the global distribution state characteristic vector of the residual material.

[0070] In the above technical solution, each position-embedded residual material state semantic feature vector in the multiple position-embedded residual material state semantic feature vectors represents a fusion feature representation of the position coding information of the residual material interest region image in the residual material distribution state image and the image semantic features of the residual material distribution state image. Therefore, after the multiple position-embedded residual material state semantic feature vectors are passed through the probability-based attention integration model, the residual material global distribution state feature vector obtained will be weightedly fused based on the image feature distribution containing the position coding information of each residual material interest region image, and will also cause the residual material global distribution state feature vector to deviate from the local-global correlation features in the local image space domain of the multiple position-embedded residual material state semantic feature vectors, which makes the global feature distribution of the residual material global distribution state feature vector have the problem of insufficient feature distribution aggregation, thereby affecting its decoding convergence efficiency through the decoder-based water pressure recommender, that is, affecting the efficiency of decoding training and the accuracy of decoding results.

[0071] In a preferred embodiment, the global distribution state feature vector of the residual material is passed through a decoder-based water pressure recommender to obtain a decoding value, which specifically includes: dot-multiplying the global distribution state feature vector of the residual material with the length of the global distribution state feature vector of the residual material, and then subtracting the dot-multiply ... The square root of the absolute value of each eigenvalue of is taken to obtain the full semantic modulation vector of the global distribution state of residual materials and the semi-semantic modulation vector of the global distribution state of residual materials; the logarithm with base 2 of each eigenvalue of the full semantic modulation vector of the global distribution state of residual materials and the semi-semantic modulation vector of the global distribution state of residual materials are calculated respectively to obtain the interactive full semantic information vector of the global distribution state of residual materials and the interactive semi-semantic information vector of the global distribution state of residual materials; the weighted sum of the full semantic information vector of the global distribution state of residual materials and the semi-semantic information vector of the global distribution state of residual materials is calculated with the weighting coefficient as a hyperparameter to obtain the optimized feature vector of the global distribution state of residual materials; the optimized feature vector of the global distribution state of residual materials is passed through the decoder-based water pressure recommender to obtain the decoding value.

[0072] Based on this, the present application takes the global distribution state feature vector of the residual material as a feature set, wherein the change semantics represented by the eigenvalue of the global distribution state feature vector of the residual material is used as a unit. In order to dynamically aggregate the semantic set composed of different change semantics of the global distribution state feature vector of the residual material without ignoring the individual semantic change information, the individual features of the global distribution state feature vector of the residual material and the collective expression of the vector scale of the global distribution state feature vector of the residual material are used as full-width and half-width feature representations, and the low-rank norms of different dimensions of the overall semantics of the feature set of the global distribution state feature vector of the residual material are used as phases through negative correlation calculation and amplitude-phase information representation modulation is performed to dynamically adjust the semantic information change relationship of the global distribution state feature vector of the residual material, thereby improving the overall semantic information expression aggregation of the feature set of the global distribution state feature vector of the residual material, and improving the decoding convergence efficiency of the global distribution state feature vector of the residual material when it is decoded by a decoder-based water pressure recommender.

[0073] In one embodiment of the present application, the visual guidance system can also automatically identify the size of the car, automatically allocate the robot's working area, and control the water or gas outlet time and the robot's running speed according to the residual amount of residual material to achieve energy saving effects, and can automatically identify obstacles in the car and avoid them reasonably.

[0074] In summary, the automatic car cleaning system of the intelligent robot for unloading the train car tipper based on the embodiment of the present application is explained, which can adaptively adjust the water outlet pressure value according to the actual residual material distribution state characteristic information in the car to meet different cleaning needs.

[0075] As described above, according to the embodiment of the present application, the automatic car clearing system of the intelligent robot for unloading the train carriage tipping machine based on the embodiment of the present application can be implemented in various terminal devices, such as a server having an automatic car clearing algorithm of the intelligent robot for unloading the train carriage tipping machine based on the embodiment of the present application. In one example, the automatic car clearing system of the intelligent robot for unloading the train carriage tipping machine based on the embodiment of the present application can be integrated into the terminal device as a software module and / or a hardware module. For example, the automatic car clearing system of the intelligent robot for unloading the train carriage tipping machine based on the embodiment of the present application can be a software module in the operating system of the terminal device, or it can be an application developed for the terminal device; of course, the automatic car clearing system of the intelligent robot for unloading the train carriage tipping machine based on the embodiment of the present application can also be one of the many hardware modules of the terminal device.

[0076] Alternatively, in another example, the train car tipper unloading intelligent robot automatic car clearing system based on the embodiment of the present application and the terminal device may also be separate devices, and the train car tipper unloading intelligent robot automatic car clearing system may be connected to the terminal device via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0077] Figure 5 This is a flow chart of the automatic car clearing method of the intelligent robot for unloading of train car dumpers according to an embodiment of the present application. Figure 5 As shown, according to the embodiment of the present application, the automatic car clearing method of the train car tipper unloading intelligent robot includes: S110, acquiring the residual material and the distribution state image of the obstacles in the car collected by the visual guidance system; S120, performing material state feature extraction on the residual material distribution state image to obtain a plurality of residual material state semantic feature vectors; S130, performing position information embedding and fusion on the plurality of residual material state semantic feature vectors to obtain a plurality of position embedded residual material state semantic feature vectors; and, S140, determining the recommended water pressure value, the working path and range of the robot based on the residual material global state information of the plurality of position embedded residual material state semantic feature vectors.

[0078] Here, those skilled in the art can understand that the specific operations of each step in the above-mentioned train car tipper unloading intelligent robot automatic car cleaning method have been referenced above. Figures 1 to 4 The description of the train car tipper unloading intelligent robot automatic car clearing system has been introduced in detail, and therefore, its repeated description will be omitted.

[0079] Figure 6 This is an application scenario diagram of the automatic car clearing system of the train car tipping machine unloading intelligent robot according to the embodiment of the present application. Figure 6 As shown, in this application scenario, first, a residual material distribution state image (for example, Figure 6 D), then, the residual material distribution state image is input to a server (for example, Figure 6 In S) as shown in , the server can use the automatic car clearing algorithm of the train car tipper unloading intelligent robot to process the residual material distribution state image to obtain a decoded value for representing the recommended water pressure value.

[0080] Furthermore, in another example of the present application, an AI-controlled automatic cleaning system for residual coal during unloading of train carriages is disclosed, which includes a six-axis robot, an automatic water supply system, and an AI control system. The six-axis robot is arranged on the front crossbeam of the car dumper as the main power of the cleaning system. A water gun of the water supply system and an AI visual scanner are installed at the end of the robot. The water gun is connected to the water supply system, and the single gun and single control are coordinated with the robot. The AI ​​visual scanner automatically scans the residual coal in the carriage and the obstacles in the carriage through the robot's movements. Among them, a water gun of the water supply system and an AI visual scanner are installed at the end of the robot.

[0081] Furthermore, the six-axis robot is used as an actuator in the cleaning process and is arranged on the front beam of the tipper, and a single system is equipped with three robots; the automatic water supply system includes an insulated water tank, a speed-regulating water pump, a water level controller, a water gun, an automatic water supply control system, etc.; the water gun is a single-gun single-control type, and is controlled in coordination with the robot.

[0082] Furthermore, the AI ​​control system serves as the core of the entire system, including an AI visual scanner and software algorithms. The AI ​​visual scanner uses a 3D visual guidance system, combined with independently developed structured light technology and hybrid light field control technology, to quickly scan the train compartments, and obtain a three-dimensional point cloud map with complete colors of the remaining materials and obstacles in the compartments. Through the visual point cloud parsing algorithm and the use of self-learning methods, it can accurately identify materials and obstacles, and automatically plan the robot's cleaning route, thereby truly realizing the automatic cleaning of residual coal.

[0083] Through this automatic cleaning system for residual coal after train carriage overturning and unloading based on AI technology, it can automatically identify obstacles in the carriage and guide the robot to learn and avoid them reasonably, automatically identify the specifications and models of the train carriage, reasonably allocate the cleaning operation range of each robot, automatically scan the location and amount of residual coal in the carriage, and reasonably plan the cleaning path, truly replacing manual coal unloading and carriage cleaning after train carriage overturning and unloading, thereby reducing the labor intensity of operators and increasing the speed of coal unloading.

[0084] According to another aspect of the present application, a non-volatile computer-readable storage medium is provided, on which computer-readable instructions are stored. When the instructions are executed by a computer, the above-mentioned method can be executed.

[0085] The program part of the technology can be considered as a "product" or "manufactured product" in the form of executable code and / or related data, which is participated in or realized by computer-readable media. Tangible and permanent storage media can include any memory or storage used by any computer, processor, or similar device or related module. For example, various semiconductor memories, tape drives, disk drives or any similar devices that can provide storage functions for software.

[0086] The present application uses specific words to describe the embodiments of the present application. For example, "first / second embodiment", "one embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of the present application. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or multiple times in different positions in this specification does not necessarily refer to the same embodiment. In addition, some features, structures or characteristics in one or more embodiments of the present application can be appropriately combined.

[0087] In addition, it will be appreciated by those skilled in the art that various aspects of the present application may be illustrated and described by a number of patentable categories or situations, including any new and useful process, machine, product or combination of substances, or any new and useful improvements thereto. Accordingly, various aspects of the present application may be performed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software may all be referred to as "data blocks", "modules", "engines", "units", "components" or "systems". In addition, various aspects of the present application may be represented as a computer product located in one or more computer-readable media, which includes computer-readable program code.

[0088] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art to which this application belongs. It should also be understood that terms such as those defined in common dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an idealized or extremely formal sense, unless explicitly defined as such herein.

[0089] The above is an explanation of the present application and should not be considered as a limitation thereof. Although several exemplary embodiments of the present application are described, it will be readily understood by those skilled in the art that many modifications may be made to the exemplary embodiments without departing from the novel teachings and advantages of the present application. Therefore, all of these modifications are intended to be included within the scope of the present application as defined by the claims. It should be understood that the above is an explanation of the present application and should not be considered to be limited to the specific embodiments disclosed, and modifications to the disclosed embodiments and other embodiments are intended to be included within the scope of the appended claims. The present application is defined by the claims and their equivalents.

Claims

1. An automatic car clearing system for a train car dumper unloading intelligent robot, It is characterized in that include: Vision guidance system, six-axis robot and automatic water supply system; The visual guidance system is installed at the end of the six-axis robot; The automatic water supply system includes a water gun, an insulated water tank, a regulating water pump, a water level controller and a water supply controller, wherein the water supply controller is communicatively connected to the visual guidance system and performs adaptive adjustment of the water outlet pressure based on the information collected by the visual guidance system; The water supply controller comprises: An image acquisition module, used for acquiring an image of the residual material distribution state acquired by a visual guidance system; A state feature extraction module, used for extracting material state features from the residual material distribution state image to obtain a plurality of residual material state semantic feature vectors; an embedding fusion coding module, used for embedding and fusion of position information of the plurality of residual material state semantic feature vectors to obtain a plurality of position embedded residual material state semantic feature vectors; and The water pressure recommendation module is used to determine the recommended water pressure value based on the residual material global state information of the residual material state semantic feature vectors embedded in the multiple positions.

2. The train car dumper unloading intelligent robot automatic car clearing system according to claim 1, It is characterized in that The state feature extraction module comprises: an area of ​​interest extraction unit, configured to extract an area of ​​interest for residual materials from the residual material distribution state image to obtain a plurality of images of the area of ​​interest for residual materials; and An image feature extraction unit is used to extract image features from the multiple residual material region of interest images using a deep learning network model to obtain the multiple residual material state semantic feature vectors.

3. The train car dumper unloading intelligent robot automatic car clearing system according to claim 2, It is characterized in that The region of interest extraction unit is used to: The residual material distribution state image is passed through a residual material region of interest extractor based on an R-CNN network to obtain the multiple residual material region of interest images.

4. The train car dumper unloading intelligent robot automatic car clearing system according to claim 3, It is characterized in that The image feature extraction unit is used to: The multiple residual material region of interest images are passed through a residual material state feature extractor based on a convolutional neural network model to obtain the multiple residual material state semantic feature vectors.

5. The train car dumper unloading intelligent robot automatic car clearing system according to claim 4, It is characterized in that The embedded fusion coding module comprises: an encoding unit, configured to extract position information of the plurality of residual material region of interest images in the residual material distribution state image, and respectively encode the position information to obtain a plurality of region of interest position information encoding feature vectors; and A fusion unit is used to process the multiple residual material state semantic feature vectors and the multiple interest region position information encoding feature vectors using a position information embedding fusion module to obtain the multiple position embedded residual material state semantic feature vectors.

6. The train car dumper unloading intelligent robot automatic car clearing system according to claim 5, It is characterized in that The fusion unit is used for: Each of the region of interest position information encoding feature vectors is concatenated at the end of the corresponding residual material state semantic feature vector to obtain the plurality of position embedded residual material state semantic feature vectors.

7. The train car dumper unloading intelligent robot automatic car clearing system according to claim 6, It is characterized in that The water pressure recommendation module comprises: An attention integration unit, used for embedding the plurality of positions into the residual material state semantic feature vector through a probability-based attention integration model to obtain a residual material global distribution state feature vector; The water pressure recommendation unit is used to pass the residual material global distribution state feature vector through a decoder-based water pressure recommender to obtain a decoded value, and the decoded value is used to represent the recommended water pressure value.

8. The train car dumper unloading intelligent robot automatic car clearing system according to claim 7, It is characterized in that The attention integration unit is used to: The semantic feature vectors of the residual material state embedded in the multiple positions are processed by the following attention integration formula to obtain the feature vector of the global distribution state of the residual material; wherein the attention integration formula is: H=Concat[a 1 h 1 ;h 2 ;···;a i h i ;···;a t h t ] Among them, e i is the i-th attention scoring coefficient, e k is the kth attention scoring coefficient, h i is the residual material state semantic feature vector embedded in the i-th position among the plurality of residual material state semantic feature vectors embedded in the positions, h t is the semantic feature vector of the residual material state embedded in the t-th position, W i is the i-th weight coefficient matrix, is the i-th weight coefficient vector, B i is the i-th bias vector, tanh(·) represents the hyperbolic tangent function processing, exp(·) represents the exponential function processing with e as the base, a i is the i-th attention weight coefficient, a t is the tth attention weight coefficient, t is the length of the semantic feature vectors of the residual material state embedded in the multiple positions, Concat[·;·] represents the cascade function, and H is the global distribution state feature vector of the residual material.

9. A method for automatically clearing a train carriage tipper unloading intelligent robot for use in the automatic clearing system for a train carriage tipper unloading intelligent robot as claimed in any one of claims 1 to 8, It is characterized in that include: Acquire the residual material distribution state image collected by the vision guidance system; Performing material state feature extraction on the residual material distribution state image to obtain a plurality of residual material state semantic feature vectors; Performing position information embedding and fusion on the multiple residual material state semantic feature vectors to obtain multiple position embedded residual material state semantic feature vectors; as well as The recommended water pressure value is determined based on the residual material global state information of the residual material state semantic feature vectors embedded in the multiple positions.