System and method for wireless remote control of excavator

Abstract

The invention discloses a system and a method for wireless remote control of an excavator. The method comprises the following steps: acquiring perceived image information of the excavator; preprocessing the excavator sensing image information to obtain target image processing information; and analyzing and processing the target image processing information to obtain target analysis result information.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of excavators, and in particular to a system and method for wireless remote control of an excavator. BACKGROUND

[0002] With the rapid development of modern industry, excavators, as important engineering machinery, play a crucial role in the fields of construction, mining, road construction, etc. Traditional excavator operation relies on manual control by the driver in the cab, which has many limitations. First, the driver is in a noisy and strongly vibrating environment for a long time, which affects physical and mental health. Second, manual operation is less efficient and has a higher risk in complex or dangerous environments. Finally, with the development of automation and intelligent technology, there is an increasing demand for remotely controlled and intelligent excavators. Currently, there are some solutions for remotely controlling excavators on the market, but these solutions mostly use wired connections, which have problems such as complex wiring, poor flexibility, and difficult maintenance. In addition, existing wireless remote control systems often lack stability and real-time performance, making it difficult to meet the high requirements for operation accuracy and response speed in complex conditions. At the same time, existing systems also have deficiencies in the collaborative work between the software platform, controller, and excavator, and cannot achieve efficient and intelligent remote control. Therefore, it is necessary to provide a system and method for wireless remote control of an excavator to achieve efficient, stable, and intelligent operation of the excavator and improve the flexibility and accuracy of wireless control of the excavator to meet the demand for remote control of the excavator. SUMMARY

[0003] The technical problem to be solved by the present application is to provide a system and method for wireless remote control of an excavator to facilitate efficient, stable, and intelligent operation of the excavator, improve the flexibility and accuracy of wireless control of the excavator, and meet the demand for remote control of the excavator.

[0004] To solve the above technical problems, the first aspect of the embodiment of the present application discloses a method for wireless remote control of an excavator, which comprises: obtaining excavator perception image information; preprocessing the excavator perception image information to obtain target image processing information; analyzing and processing the target image processing information to obtain target analysis result information.

[0005] The second aspect of the embodiment of the present application discloses a system for wireless remote control of an excavator, which comprises: an acquisition module for acquiring excavator perception image information; a first processing module for preprocessing the excavator perception image information to obtain target image processing information; A second processing module is configured to analyze and process the target image processing information to obtain target analysis result information.

[0006] A third aspect of the present application discloses another system for wireless remote control of excavators, which comprises: a memory storing executable program codes; a processor coupled with the memory; The processor invokes the executable program codes stored in the memory to execute part or all of the steps of the method for wireless remote control of excavators disclosed in the first aspect of the present application.

[0007] A fourth aspect of the present application discloses a computer readable storage medium storing computer instructions, which, when invoked, are configured to execute part or all of the steps of the method for wireless remote control of excavators disclosed in the first aspect of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0009] Figure 1 is a scene schematic diagram of the system for wireless remote control of excavators provided by the embodiments of the present application; Figure 2 is a flow schematic diagram of the method for wireless remote control of excavators disclosed by the embodiments of the present application; Figure 3 is a structural schematic diagram of the system for wireless remote control of excavators disclosed by the embodiments of the present application; Figure 4 is a structural schematic diagram of another system for wireless remote control of excavators disclosed by the embodiments of the present application; Figure 5 is a structural schematic diagram of a target image recognition model disclosed by the embodiments of the present application; Figure 6 is a structural schematic diagram of a first feature extraction module disclosed by the embodiments of the present application. DETAILED DESCRIPTION

[0010] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by a person of ordinary skill in the art without creative effort belong to the protection scope of the present application.

[0011] The terms "first", "second", and the like in the description and claims of the present application and the above drawings are used to distinguish different objects, rather than to describe a particular order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device including a series of steps or units is not limited to the listed steps or units, but can optionally include other steps or units not listed, or can optionally include other steps or units inherent to the process, method, product, or device.

[0012] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily all refer to the same embodiment, nor is it necessarily mutually exclusive of other embodiments. It is explicitly and implicitly understood that the embodiments described herein can be combined with other embodiments.

[0013] In this application, the word "exemplary" is used to mean "serving as an example, instance, or illustration." Any implementation described as "exemplary" is not necessarily to be construed as preferred or advantageous over other implementations. The following description is presented to enable any person skilled in the art to make and use the application. In the following description, for purposes of explanation, specific details are set forth. It is apparent to those skilled in the art that the present application can be practiced without using these specific details. In other instances, well-known structures and processes are not described in detail in order to avoid obscuring the description of the application. Thus, the present application is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features presented herein.

[0014] It should be noted that the method of the present application is executed in a computer device, and the processing objects of each computer device exist in the form of data or information, such as time, which is essentially time information. It can be understood that if the size, quantity, position, etc. are mentioned in subsequent embodiments, they all exist in the form of corresponding data for processing by the computer device, and specific details are not described here.

[0015] It should be noted that the artificial intelligence related technologies involved in the present application are briefly described. Artificial intelligence (AI) is the use of digital computers or digital computer controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision making.

[0016] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software technologies. Artificial intelligence basic technologies generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0017] Computer vision (CV) is a science that studies how to make machines "see". Further, it refers to using cameras and computers to replace human eyes to identify and measure targets, and further to do image processing, so that the computer processing becomes more suitable for human eye observation or image transmission to instrument detection. As a scientific discipline, computer vision researches related theories and technologies, trying to establish artificial intelligence systems that can obtain information from images or multidimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, etc. It also includes common face recognition, fingerprint recognition and other biometric identification technologies.

[0018] Single modal information is only one type of data, such as text, image, audio, video, electromagnetic signal, etc. Multi-modal information is data information including at least two single modal information. Further, multi-modal information is suitable for complex tasks that require the integration of multiple information sources, such as sentiment analysis, robot interaction, autonomous driving, etc. By integrating information of multiple modalities, higher performance and accuracy can usually be achieved in tasks.

[0019] A large model refers to an artificial neural network model with a very large number of parameters. In the field of artificial intelligence, a large model generally refers to a model with hundreds of millions to trillions of parameters. The model usually needs to be trained on a large-scale dataset and requires a large amount of computing resources for optimization and adjustment. Large models are usually used to solve complex natural language processing, computer vision, and speech recognition tasks. Generative AI is an AI that can create new content and ideas, including conversations, stories, images, videos, and music. In the embodiments of the present application, the large model can be a large language model such as ChatGPT, BERT, XLNet, Zhibu model, Claude, Moonshot AI model, ChatGLM model, Tongwen Qiyi model, MiniMax model, Xinghuo model, Llama model, 360GPT model, Qwen model, Baichuan model, Yunque model, vivoLM model, and Wenxin Yiyang, etc. The embodiments of the present application are not limited.

[0020] The embodiments of the present application provide a method, system, computer device and computer readable storage medium for wireless remote control of excavators, which are described in detail below.

[0021] Please refer to Figure 1 , Figure 1 The scene schematic diagram of the system for wireless remote control of excavators provided by the embodiments of the present application can include a computer device 100, and the computer device 100 is integrated with a system for wireless remote control of excavators, such as Figure 1 the computer device in the

[0022] The computer device 100 in the embodiments of the present application is mainly used to obtain excavator perception image information. The excavator perception image information is preprocessed to obtain target image processing information. The target image processing information is analyzed and processed to obtain target analysis result information.

[0023] It can realize efficient, stable and intelligent control of the excavator, improve the flexibility and accuracy of wireless control of the excavator, and meet the demand for remote control of the excavator.

[0024] In the embodiments of the present application, the computer device 100 can be a standalone server, or a server network or server cluster composed of servers. For example, the computer device 100 described in the embodiments of the present application includes but is not limited to a computer, a network host, a single network server, a plurality of network server sets, or a cloud server composed of a plurality of servers. The cloud server is composed of a large number of computers or network servers based on cloud computing.

[0025] It can be understood that the computer device 100 used in the embodiments of the present application can be a device that includes receiving and transmitting hardware, that is, a device with receiving and transmitting hardware capable of performing bidirectional communication on a bidirectional communication link. Such a device can include a cellular or other communication device with a single-line display or a multi-line display or a cellular or other communication device without a multi-line display. The computer device 100 can be a desktop terminal or a mobile terminal in particular, and the computer device 100 can also be one of a mobile phone, a tablet computer, a notebook computer, etc.

[0026] Those skilled in the art can understand that, Figure 1 The application environment shown in the above Figure 1 The application environment shown in the above Figure 1 Only one computer device is shown in the above It can be understood that the system for wireless remote control of the excavator can also include one or more other services, which are not limited specifically herein.

[0027] In addition, as shown in the above Figure 1 The system for wireless remote control of the excavator can also include a memory 200 for storing data such as image data, position information, etc.

[0028] It should be noted that Figure 1 The scene diagram of the system for wireless remote control of the excavator shown in the above is only an example, and the system for wireless remote control of the excavator and the scene described in the embodiments of the present application are for more clearly illustrating the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, as the system for wireless remote control of the excavator evolves and new service scenarios appear, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0029] The present application discloses a system and method for wireless remote control of an excavator, which is beneficial to realize efficient, stable and intelligent control of the excavator, improve the flexibility and accuracy of wireless control of the excavator, and meet the demand for remote control of the excavator. The following will be described in detail.

[0030] Embodiment one Please refer to Figure 2 , Figure 2 is a flowchart of a method for wireless remote control of an excavator disclosed by the embodiments of the present application. In the above Figure 2 The method for wireless remote control of the excavator described above is applied in a management system, such as a local server or a cloud server for management, etc., which is not limited by the embodiments of the present application. As shown in the above Figure 2As shown, the method for wireless remote control of the excavator can include the following operations: 101, obtain excavator sensing image information.

[0031] 102, pre-process the excavator sensing image information to obtain target image processing information.

[0032] 103, analyze and process the target image processing information to obtain target analysis result information.

[0033] It should be noted that the above-mentioned excavator sensing image information is photographed by a binocular shooting device arranged on the excavator working device, representing the image of the target object to be worked and the working environment under the current working environment, and the embodiments of the present application are not limited.

[0034] It should be noted that the above-mentioned target analysis result information represents the distance from the excavator working device to the target object to be worked, so as to perform wireless remote control operation, and the embodiments of the present application are not limited.

[0035] It should be noted that the method of the present application is used in the intelligent management system of the excavator, which can be constructed in the form of software platform + controller + excavator, and the embodiments of the present application are not limited. Further, the power source of the above-mentioned excavator can be realized by pump control, and the embodiments of the present application are not limited. Further, the above-mentioned software platform can be a control system based on a local server or a cloud software system, and the embodiments of the present application are not limited. Further, the above-mentioned controller (which can be designed based on a RuiCore chip) is mounted on the excavator end to control the excavator working device, and the embodiments of the present application are not limited.

[0036] It can be seen that the method for wireless remote control of the excavator described in the embodiments of the present application is beneficial to realize efficient, stable and intelligent control of the excavator, improve the flexibility and accuracy of wireless control of the excavator, and meet the demand for remote control of the excavator.

[0037] In an optional embodiment, the above-mentioned pre-processing of the excavator sensing image information to obtain target image processing information includes: filtering the excavator sensing image information to obtain first image processing information; performing image size adjustment processing on the first image processing information to obtain second image processing information; performing normalization processing on the second image processing information to obtain target image processing information.

[0038] It should be noted that the above due to the relatively harsh working environment of the excavator, suspended particles, so that the image taken by the excavator is prone to be unclear, and then it is not easy to quickly and accurately identify and distance calculation of the working object, so the image is pre-processed to improve the clarity of the image, so as to subsequent more rapid and efficient working object identification and distance calculation, improve the efficiency and accuracy of the wireless remote control of the excavator, the embodiment of the application does not limit.

[0039] It should be noted that the above image size adjustment processing of the first image processing information is to adjust the image size to 256x256, so as to process the target image recognition model, and the embodiment of the application does not limit.

[0040] It should be noted that the above normalization processing of the second image processing information is to scale the pixel value of the image to the range of [0, 1], and the embodiment of the application does not limit.

[0041] It can be seen that the method for wireless remote control of the excavator described in the embodiment of the application is beneficial to realize efficient, stable and intelligent control of the excavator, improve the flexibility and accuracy of the wireless control of the excavator, and meet the demand of remote control of the excavator.

[0042] In another optional embodiment, the excavator sensing image information is filtered to obtain first image processing information, comprising: The excavator sensing image information is subjected to median filtering to obtain filtered image processing information; The filtered image processing information is subjected to bilateral filtering to obtain the first image processing information.

[0043] It should be noted that the above median filtering of the excavator sensing image information can be performed by using an adaptive median filter, so that the dark image pixels in some regions of the image are more comparable, and the embodiment of the application does not limit.

[0044] It should be noted that the above bilateral filtering of the filtered image processing information is to process the remaining Gaussian noise, avoid edge over-smoothing through gray similarity constraint, overcome the fine distortion problem caused by median filtering, so that the image is more real, and the embodiment of the application does not limit.

[0045] It should be noted that the combination of the above median filtering and bilateral filtering is the result of balancing the noise hierarchical processing and edge protection requirements: the former removes impulse noise, and the latter suppresses Gaussian noise and preserves edges, and the embodiment of the application does not limit.

[0046] It can be seen that the method for wireless remote control of the excavator described in the embodiment of the present application is beneficial to realize efficient, stable and intelligent control of the excavator, improve the flexibility and accuracy of wireless control of the excavator, and meet the demand for remote control of the excavator.

[0047] In yet another optional embodiment, the target image processing information is analyzed and processed to obtain target analysis result information, including: The target image processing information is transmitted from the CPU end of the excavator to the constant memory of the GPU end of the software platform; The target image recognition model is used to recognize and process the target image processing information at the GPU end of the software platform to obtain target recognition result information; Based on the target recognition result information, the first analysis result information is determined; The first analysis result information is transmitted from the GPU end of the software platform to the CPU end of the excavator to obtain the target analysis result information.

[0048] It should be noted that the excavator sensing image information is first preprocessed at the excavator CPU end, mainly considering that the preprocessing process is a logical processing process, which can be efficiently processed using CPU resources, and the target image processing information obtained after preprocessing is more focused than the excavator sensing image information, and the transmission resources required for transmitting the target image processing information to the software platform in the back end are less, and the efficiency is higher. Further, since the excavator working environment image needs to be recognized and the distance estimated, the GPU can be used to analyze and process more quickly and accurately. Further, after obtaining the first analysis result information representing the distance between the excavator working device and the excavator working object, the CPU end of the excavator is transmitted back, considering that the subsequent path planning is essentially a logical control process, which can be more efficiently processed using CPU resources, and the subsequent working device control is based on the controller of the excavator. Therefore, transmitting the recognized data result back to the excavator end will be more beneficial to the wireless control of the excavator, and the embodiment of the present application is not limited.

[0049] It should be noted that the target image processing information is transmitted from the CPU end of the excavator to the constant memory of the GPU end of the software platform, considering that the GPU constant memory memory accelerates the reading of static parameters through on-chip cache mechanism, significantly reduces the data migration frequency between heterogeneous systems, and effectively suppresses the processing delay. The embodiment of the present application is not limited.

[0050] It should be noted that the above-mentioned first analysis result information is determined based on the target recognition result information after the target object is identified by the target image recognition model, and then stereo matching and disparity calculation are performed to obtain a disparity map (the disparity value is the difference between the same feature points of the matched left and right target images on the x-coordinate axis), and the actual distance between the target object and the excavator is calculated based on the disparity value in the disparity map and the camera parameters (such as focal length, baseline length) (which can be achieved based on image depth estimation), so as to obtain the first analysis result information characterizing the distance from the excavator operating device to the excavator operating object, so that the excavator can be operated by remote wireless remote control. The embodiment of the present invention does not limit this.

[0051] It can be seen that implementing the method for wireless remote control of an excavator described in the embodiment of the present invention is conducive to achieving efficient, stable and intelligent control of the excavator, improving the flexibility and accuracy of wireless control of the excavator, and meeting the needs of remote control of the excavator.

[0052] In another optional embodiment, Figure 5 As shown, the target image recognition model includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a first fusion module, a second fusion module, a third fusion module, a fourth fusion module, a fifth fusion module, a first convolution module, a second convolution module, a third convolution module, a first sampling module, a first normalization module, a first connection module, and a first activation module; wherein, The input end of the first feature extraction module, the input end of the second feature extraction module, and the input end of the third feature extraction module are all configured to receive the model input of the target image recognition model; the output end of the first feature extraction module, the output end of the second feature extraction module, and the output end of the third feature extraction module are all connected to the input end of the first fusion module; the output end of the first fusion module is connected to the input end of the second fusion module; the input end of the second fusion module is also configured to receive the model input of the target image processing model; the output end of the second fusion module is respectively connected to the input end of the first convolution module and the input end of the third fusion module; the first convolution module, the first sampling module, the third fusion module, the second convolution module, the fourth fusion module, and the fifth fusion module are connected in sequence; the output end of the third fusion module is also respectively connected to the input end of the third convolution module and the input end of the fifth fusion module; the output end of the third convolution module is connected to the input end of the fifth fusion module; the output end of the fifth fusion module is connected to the input end of the first normalization module; the first normalization module, the first connection module, and the first activation module are connected in sequence; the output end of the first activation module is configured to output the model output of the target image recognition model.

[0053] It should be noted that the model architectures of the first feature extraction module, the second feature extraction module, and the third feature extraction module are consistent, and the embodiment of the present invention does not limit this.

[0054] It should be noted that the first fusion module, the second fusion module, the third fusion module, the fourth fusion module and the fifth fusion module are constructed based on the feature fusion operation of Concat BiFPN, and the embodiments of the present application are not limited. Further, the fusion module constructed based on the feature fusion operation of Concat BiFPN better fuses multi-scale features of different levels through bidirectional transmission, thereby greatly reducing the loss of information. By establishing a bidirectional path between different levels, the information is more effectively transmitted and fused between different levels, which helps to improve the representation ability of the network for multi-scale features, and the embodiments of the present application are not limited.

[0055] It should be noted that the convolution kernel of the first convolution module, the second convolution module and the third convolution module can be one of 1x1, 3x1, 3x3, 5x1, 5x3, 5x5, 7x5 and 7x7, and the step is 1, and the embodiments of the present application are not limited.

[0056] It should be noted that the first sampling module is constructed based on the up-sampling operation to restore a low-resolution image to a high-resolution image, and the embodiments of the present application are not limited.

[0057] It should be noted that the first normalization module is constructed based on the max-pooling layer, and the embodiments of the present application are not limited.

[0058] It should be noted that the first connection module is constructed based on the full connection layer, and the embodiments of the present application are not limited.

[0059] It should be noted that the first activation module is constructed based on the RELU activation function, and the embodiments of the present application are not limited.

[0060] It should be noted that the target image recognition model performs multi-dimensional feature extraction processing on the image through the plurality of feature extraction modules, then performs deep fusion of the features through the bidirectional transmission feature fusion module, and then further extracts and fuses the features through the plurality of convolution modules, which can improve the deep extraction ability of the image features, and finally maps the extracted features through the full connection layer and the activation function to obtain a more reasonable prediction recognition result, thereby improving the target object recognition accuracy of the excavator photographed image, and the embodiments of the present application are not limited.

[0061] It can be seen that the method for wireless remote control of the excavator described in the embodiments of the present application is beneficial to realize efficient, stable and intelligent control of the excavator, improve the flexibility and accuracy of wireless control of the excavator, and meet the demand for remote control of the excavator.

[0062] In an optional embodiment, as Figure 6As shown, the first feature extraction module comprises a first convolution unit, a second convolution unit, a third convolution unit, a first fusion unit, a second fusion unit, a third fusion unit, a first pooling unit, a second pooling unit, a first normalization unit, and a first activation unit; wherein, The input end of the first convolution unit and the input end of the second convolution unit are both configured to receive the input end of the first feature extraction module; the output end of the first convolution unit and the output end of the second convolution unit are both connected to the input end of the first fusion unit; the output end of the first fusion unit is connected to the input end of the first pooling unit and the input end of the second pooling unit respectively; the output end of the first pooling unit and the output end of the second pooling unit are both connected to the input end of the second fusion unit; the second fusion unit, the third fusion unit, the third convolution unit, the first normalization unit, and the first activation unit are sequentially connected in order; the input end of the third fusion unit is also configured to be the input end of the first feature extraction module; the output end of the first activation module is configured to be the output end of the first feature extraction module.

[0063] It should be noted that the convolution kernel of the first convolution unit, the second convolution unit, and the third convolution unit can be one of 1x1, 3x1, 3x3, 5x1, 5x3, 5x5, 7x5, and 7x7, and the step is 1, which is not limited by the embodiment of the present application.

[0064] It should be noted that the first fusion unit, the second fusion unit, and the third fusion unit are constructed based on a splicing operation, which is not limited by the embodiment of the present application.

[0065] It should be noted that the first pooling unit and the second pooling unit are constructed based on a maximum pooling layer, which is not limited by the embodiment of the present application.

[0066] It should be noted that the first normalization unit is constructed based on a batch normalization layer, which is not limited by the embodiment of the present application.

[0067] It should be noted that the first activation unit is constructed based on a RELU activation function, which is not limited by the embodiment of the present application.

[0068] It should be noted that the first feature extraction module first extracts and fuses features through the convolution unit, then reduces the spatial size (i.e. width and height) of the data through the pooling unit to reduce the dimension of the feature map, reduce the number of parameters and the amount of calculation of the subsequent layer, and at the same time, helps to extract the invariance feature in the image, and finally, through a standard feature extraction process of convolution, normalization, and activation, further extracts the feature to form a rich feature representation, which is not limited by the embodiment of the present application.

[0069] It can be seen that the method for excavator wireless remote control described in the embodiment of the application facilitates efficient, stable and intelligent control of the excavator, improves the flexibility and accuracy of wireless control of the excavator, and meets the demand for remote control of the excavator.

[0070] In another optional embodiment, the target image recognition model is obtained by training based on the following manner: Determine N initial image samples as target image samples from an initial image sample set; Train the initial image recognition model using the target image samples to obtain training parameter information and a training image recognition model; Calculate and process the training parameter information using a loss function to obtain loss function value information; The loss function is: ; In the formula, represent the loss function value in the loss function value information; represent the parameter values corresponding to the training parameter information; and represent the first training coefficient and the second training coefficient, respectively; and represent the mean and variance of the parameter values corresponding to the training parameter information, respectively; Determine whether the loss function value in the loss function value information meets a training termination condition to obtain a training determination result; When the training determination result is no, the training image recognition model is determined as a new initial image recognition model; When the training determination result is yes, the training image recognition model is determined as the target image recognition model.

[0071] It should be noted that the target image recognition model can be implemented based on PYTHON 3.7.10 and above versions, trained on an NVIDIA GeForce RTX 4090 graphics card, the initial image samples can be obtained by labeling pictures taken by a user using a high-definition camera, the iteration round is not less than 100 times, the optimizer is Adam, and the learning rate is not greater than 1x10 -4 In order to better train the model, the learning rate is automatically adjusted with the increase of the training round, that is, the learning rate is reduced by 10 times every 20 rounds, the optimizer can accelerate the convergence of the network, and the accuracy, precision and recall rate can be used to evaluate the trained model, which is not limited in the embodiment of the application.

[0072] It should be noted that the above-mentioned Sun loss function can significantly improve the robustness of the model to dynamic data distribution and noise interference through the four-fold design of parameter standardization, learnable error direction control, Softplus smoothing and adaptive attenuation. It can eliminate the negative impact of data distribution offset, dynamically adjust the error penalty strategy through learnable coefficients, and suppress outlier interference to ensure training convergence. Specific parameter standardization ( ) will train the parameters Transformed into a distribution with zero mean and unit variance; learnable error direction control ( ) can adjust the error direction and weight; Softplus smoothing ( ) can smoothly cut off negative inputs to ensure differentiability; adaptive attenuation ( ) can suppress extreme errors and prevent gradient explosion. Compared to traditional loss functions such as MSE and MAE, it introduces normalization and attenuation factors to combat outliers, supports asymmetric penalties, eliminates the need for manual parameter tuning, and can dynamically adjust error sensitivity. This is not limited in this embodiment of the present invention.

[0073] It should be noted that the above N is a positive integer not less than 10, and is not limited in the embodiment of the present invention.

[0074] It should be noted that the model architectures of the above-mentioned target image recognition model, initial image recognition model and training image recognition model are consistent. The difference is that the parameter weights may be different, which is not limited in the embodiment of the present invention.

[0075] It should be noted that the above training termination conditions are that the number of training times reaches the iterative round, or the loss function value converges, which is not limited in the embodiment of the present invention.

[0076] It can be seen that implementing the method for wireless remote control of an excavator described in the embodiment of the present invention is conducive to achieving efficient, stable and intelligent control of the excavator, improving the flexibility and accuracy of wireless control of the excavator, and meeting the needs of remote control of the excavator.

[0077] Example 2 See also Figure 3 , Figure 3 This is a schematic diagram of the structure of a system for wireless remote control of an excavator disclosed in an embodiment of the present invention. Figure 3 The described system can be applied to a management system, such as a local server or a cloud server for management, etc., and the embodiment of the present invention does not limit this. Figure 3 As shown, the system may include: An acquisition module 201 is used to acquire image information perceived by the excavator; The first processing module 202 is used to pre-process the excavator perception image information to obtain target image processing information; The second processing module 203 is configured to analyze and process the target image processing information to obtain target analysis result information.

[0078] It can be seen that, by implementing the system for wireless remote control of the excavator, Figure 3 The system for wireless remote control of the excavator described in the embodiment has the advantages of facilitating efficient, stable and intelligent control of the excavator, improving the flexibility and accuracy of wireless control of the excavator, and meeting the demand for remote control of the excavator.

[0079] In another optional embodiment, as shown in Figure 3 The target image processing information is obtained by preprocessing the excavator sensing image information, and includes: The first image processing information is obtained by filtering the excavator sensing image information; The second image processing information is obtained by adjusting the image size of the first image processing information; The target image processing information is obtained by normalizing the second image processing information.

[0080] It can be seen that, by implementing the system for wireless remote control of the excavator, Figure 3 The system for wireless remote control of the excavator described in the embodiment has the advantages of facilitating efficient, stable and intelligent control of the excavator, improving the flexibility and accuracy of wireless control of the excavator, and meeting the demand for remote control of the excavator.

[0081] In yet another optional embodiment, as shown in Figure 3 The first image processing information is obtained by filtering the excavator sensing image information, and includes: The filter image processing information is obtained by median filtering the excavator sensing image information; The first image processing information is obtained by bilateral filtering the filter image processing information.

[0082] It can be seen that, by implementing the system for wireless remote control of the excavator, Figure 3 The system for wireless remote control of the excavator described in the embodiment has the advantages of facilitating efficient, stable and intelligent control of the excavator, improving the flexibility and accuracy of wireless control of the excavator, and meeting the demand for remote control of the excavator.

[0083] In yet another optional embodiment, as shown in Figure 3 The target analysis result information is obtained by analyzing and processing the target image processing information, and includes: The target image processing information is transmitted from the CPU end of the excavator to the constant storage of the GPU end of the software platform; The target recognition result information is obtained by using the target image recognition model to recognize the target image processing information at the GPU end of the software platform; The first analysis result information is determined based on the target recognition result information; The first analysis result information is transmitted from a GPU end of the software platform to a CPU end of the excavator to obtain target analysis result information.

[0084] It can be seen that, in the embodiments Figure 3 The system for wireless remote control of the excavator facilitates efficient, stable and intelligent control of the excavator, improves flexibility and accuracy of wireless control of the excavator, and meets the demand for remote control of the excavator.

[0085] In yet another optional embodiment, as Figure 3 shown, the target image recognition model includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a first fusion module, a second fusion module, a third fusion module, a fourth fusion module, a fifth fusion module, a first convolution module, a second convolution module, a third convolution module, a first sampling module, a first normalization module, a first connection module and a first activation module; wherein, The input end of the first feature extraction module, the input end of the second feature extraction module and the input end of the third feature extraction module are configured to receive model input of the target image recognition model; the output end of the first feature extraction module, the output end of the second feature extraction module and the output end of the third feature extraction module are connected to the input end of the first fusion module; the output end of the first fusion module is connected to the input end of the second fusion module; the input end of the second fusion module is also configured to receive model input of the target image processing model; the output end of the second fusion module is connected to the input end of the first convolution module and the input end of the third fusion module respectively; the first convolution module, the first sampling module, the third fusion module, the second convolution module, the fourth fusion module and the fifth fusion module are sequentially connected in order; the output end of the third fusion module is also connected to the input end of the third convolution module and the input end of the fifth fusion module respectively; the output end of the third convolution module is connected to the input end of the fifth fusion module; the output end of the fifth fusion module is connected to the input end of the first normalization module; the first normalization module, the first connection module and the first activation module are sequentially connected in order; the output end of the first activation module is configured to output model output of the target image recognition model.

[0086] It can be seen that, in the embodiments Figure 3 The system for wireless remote control of the excavator facilitates efficient, stable and intelligent control of the excavator, improves flexibility and accuracy of wireless control of the excavator, and meets the demand for remote control of the excavator.

[0087] In yet another optional embodiment, as Figure 3 shown, the first feature extraction module includes a first convolution unit, a second convolution unit, a third convolution unit, a first fusion unit, a second fusion unit, a third fusion unit, a first pooling unit, a second pooling unit, a first normalization unit and a first activation unit; wherein, The input end of the first convolution unit and the input end of the second convolution unit are both configured to receive the input end of the first feature extraction module; the output end of the first convolution unit and the output end of the second convolution unit are both connected to the input end of the first fusion unit; the output end of the first fusion unit is connected to the input end of the first pooling unit and the input end of the second pooling unit respectively; the output end of the first pooling unit and the output end of the second pooling unit are both connected to the input end of the second fusion unit; the second fusion unit, the third fusion unit, the third convolution unit, the first normalization unit and the first activation unit are sequentially connected in order; the input end of the third fusion unit is also configured to be the input end of the first feature extraction module; the output end of the first activation module is configured to be the output end of the first feature extraction module.

[0088] It can be seen that, in the embodiments Figure 3 The system for excavator wireless remote control described herein facilitates efficient, stable and intelligent control of the excavator, improves the flexibility and accuracy of wireless control of the excavator, and meets the demand for remote control of the excavator.

[0089] In yet another optional embodiment, as Figure 3 The target image recognition model is trained based on the following manner: Determine N initial image samples from the initial image sample set as target image samples; Train the initial image recognition model using the target image samples to obtain training parameter information and a training image recognition model; Calculate and process the training parameter information using a loss function to obtain loss function value information; The loss function is: ; In the formula, represents the loss function value in the loss function value information; represents the parameter value corresponding to the training parameter information; and represent the first training coefficient and the second training coefficient, respectively; and represent the mean and variance of the parameter value corresponding to the training parameter information, respectively; Determine whether the loss function value in the loss function value information meets a training termination condition to obtain a training determination result; When the training determination result is no, the training image recognition model is determined as a new initial image recognition model; When the training determination result is yes, the training image recognition model is determined as the target image recognition model.

[0090] It can be seen that, in the embodiments Figure 3The system for wireless remote control of excavators described is beneficial to realize efficient, stable and intelligent control of excavators, improve the flexibility and accuracy of wireless control of excavators, and meet the needs of remote control of excavators.

[0091] Embodiment three Please refer to Figure 4 , Figure 4 is another structural schematic diagram of a system for wireless remote control of excavators disclosed by the embodiments of the present application. Among them, Figure 4 The system described can be applied to a management system, such as a local server or a cloud server for management, etc., which is not limited by the embodiments of the present application. As shown in Figure 4 The system can include: a memory 301 storing executable program codes; a processor 302 coupled with the memory 301; The processor 302 invokes the executable program codes stored in the memory 301, for executing the steps in the method for wireless remote control of excavators described in embodiment one.

[0092] Embodiment four The embodiments of the present application disclose a computer readable storage medium storing a computer program for electronic data exchange, wherein the computer program causes a computer to execute the steps in the method for wireless remote control of excavators described in embodiment one.

[0093] Embodiment five The embodiments of the present application disclose a computer program product including a non-transitory computer readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute the steps in the method for wireless remote control of excavators described in embodiment one.

[0094] The system embodiments described above are only schematic, wherein the modules illustrated as separate components can or can not be physically separated, and the components illustrated as modules can or can not be physical modules, i.e., can be located in one place, or can be distributed on multiple network modules. Part or all of the modules can be selected to achieve the purpose of the present embodiment scheme according to actual needs. Those skilled in the art can understand and implement it without creative labor.

[0095] Those skilled in the art can clearly understand the technical solutions of the various embodiments through the above specific description of the embodiments, and the various embodiments can be realized by means of software and necessary universal hardware platforms, and of course, can also be realized by hardware. Based on such understanding, the above technical solutions, essentially or in terms of contribution to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer readable storage medium, including a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM), or other optical disk storage, magnetic disk storage, magnetic tape storage, or any other medium that can be used to carry or store data in a computer readable manner.

[0096] Finally, it should be noted that: the system and method for excavator wireless remote control disclosed by the embodiments of the present application are only the preferred embodiments of the present application, and are used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for wireless remote control of an excavator, characterized in that: The method comprises: Obtaining excavator perception image information; Preprocessing the excavator-perceived image information to obtain target image processing information; The target image processing information is analyzed and processed to obtain target analysis result information.

2. The method for wireless remote control of an excavator according to claim 1, characterized in that: The preprocessing of the excavator-perceived image information to obtain target image processing information includes: Performing filtering processing on the excavator-perceived image information to obtain first image processing information; performing image resizing processing on the first image processing information to obtain second image processing information; Normalization is performed on the second image processing information to obtain target image processing information.

3. The method for wireless remote control of an excavator according to claim 2, characterized in that: The filtering process of the excavator-perceived image information to obtain first image processing information includes: Performing median filtering on the excavator perception image information to obtain filtered image processing information; Perform bilateral filtering on the filtered image processing information to obtain first image processing information.

4. The method for wireless remote control of an excavator according to claim 1, characterized in that: The analyzing and processing the target image processing information to obtain target analysis result information includes: The target image processing information is transmitted from the CPU of the excavator to the constant memory of the GPU of the software platform; Performing recognition processing on the target image processing information using a target image recognition model on the GPU side of the software platform to obtain target recognition result information; Determining first analysis result information based on the target recognition result information; The first analysis result information is transmitted from the GPU side of the software platform to the CPU side of the excavator to obtain target analysis result information.

5. The method for wireless remote control of an excavator according to claim 4, characterized in that: The target image recognition model includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a first fusion module, a second fusion module, a third fusion module, a fourth fusion module, a fifth fusion module, a first convolution module, a second convolution module, a third convolution module, a first sampling module, a first normalization module, a first connection module, and a first activation module; wherein, The input end of the first feature extraction module, the input end of the second feature extraction module, and the input end of the third feature extraction module are all configured to receive the model input of the target image recognition model; the output end of the first feature extraction module, the output end of the second feature extraction module, and the output end of the third feature extraction module are all connected to the input end of the first fusion module; the output end of the first fusion module is connected to the input end of the second fusion module; the input end of the second fusion module is also configured to receive the model input of the target image processing model; the output end of the second fusion module is respectively connected to the input end of the first convolution module and the output end of the third fusion module input end; the first convolution module, the first sampling module, the third fusion module, the second convolution module, the fourth fusion module and the fifth fusion module are connected in sequence; the output end of the third fusion module is also connected to the input end of the third convolution module and the input end of the fifth fusion module respectively; the output end of the third convolution module is connected to the input end of the fifth fusion module; the output end of the fifth fusion module is connected to the input end of the first normalization module; the first normalization module, the first connection module and the first activation module are connected in sequence; the output end of the first activation module is configured to output the model output of the target image recognition model.

6. The method for wireless remote control of an excavator according to claim 5, characterized in that: The first feature extraction module includes a first convolution unit, a second convolution unit, a third convolution unit, a first fusion unit, a second fusion unit, a third fusion unit, a first pooling unit, a second pooling unit, a first normalization unit, and a first activation unit; wherein, The input end of the first convolution unit and the input end of the second convolution unit are both configured to receive the input end of the first feature extraction module; the output end of the first convolution unit and the output end of the second convolution unit are both connected to the input end of the first fusion unit; the output end of the first fusion unit is respectively connected to the input end of the first pooling unit and the input end of the second pooling unit; the output end of the first pooling unit and the output end of the second pooling unit are both connected to the input end of the second fusion unit; the second fusion unit, the third fusion unit, the third convolution unit, the first normalization unit and the first activation unit are connected in sequence; the input end of the third fusion unit is also configured as the input end of the first feature extraction module; the output end of the first activation module is configured as the output end of the first feature extraction module.

7. The method for wireless remote control of an excavator according to claim 4, characterized in that: The target image recognition model is trained based on the following method: Determining N of the initial image samples from the initial image sample set as target image samples; Using the target image sample to train the initial image recognition model to obtain training parameter information and a training image recognition model; Calculating the training parameter information using a loss function to obtain loss function value information; Wherein, the loss function is: ； In the formula, represents the loss function value in the loss function value information; represents the parameter value corresponding to the training parameter information; and respectively represent the first training coefficient and the second training coefficient; and respectively represent the mean and variance of the parameter value corresponding to the training parameter information; Determine whether the loss function value in the loss function value information meets the training termination condition, and obtain a training judgment result; When the training judgment result is no, determining the training image recognition model as a new initial image recognition model; When the training judgment result is yes, the training image recognition model is determined to be the target image recognition model.

8. A system for wireless remote control of an excavator, characterized in that: The system comprises: An acquisition module is used to obtain the excavator's perception image information; A first processing module is used to pre-process the excavator perception image information to obtain target image processing information; The second processing module is used to analyze and process the target image processing information to obtain target analysis result information.

9. A system for wireless remote control of an excavator, characterized in that: The system comprises: a memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the method for wireless remote control of an excavator according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and when the computer instructions are called, they are used to execute the method for wireless remote control of an excavator according to any one of claims 1 to 7.

Images