A control method and device of a mechanical arm, an electronic device, and a storage medium

Abstract

This application discloses a control method, device, electronic device, and storage medium for a robotic arm, relating to the field of data processing technology. The method includes: acquiring image data of an object to be transported placed on a vehicle via a PLC; determining control parameters for the robotic arm based on the image data; and sending the control parameters to the PLC, so that the PLC controls the robotic arm according to the control parameters to grasp the object to be transported and transport it to a designated location.

Description

Technical Field

[0001] This application relates to, but is not limited to, the field of control technology for robotic arms of vehicles, and particularly to a control method, device, electronic device, and storage medium for a robotic arm. Background Technology

[0002] Currently, vehicles are controlled using simple methods, such as mechanical buttons, to operate the vehicle's robotic arm.

[0003] However, for vehicles in certain scenarios, this approach results in low control efficiency for the robotic arm, hindering its effectiveness. Summary of the Invention

[0004] One embodiment of this application provides a control method, device, electronic device, and storage medium for a robotic arm. In this method, the electronic device performs image processing on the acquired image data of the object to be transported to determine the control parameters of the robotic arm when transporting the object. These parameters are then sent to a PLC so that the PLC controls the robotic arm according to the control parameters to complete the transport of the object.

[0005] One embodiment of this application provides a control method, device, electronic device, and storage medium for a robotic arm. The image data described above can be used to determine the relative position of the object to be transported and the end effector in three-dimensional space, the shape of the object to be transported, and the size of the object to be transported. This allows the control parameters of the robotic arm to be determined, enabling the PLC to control the robotic arm to transport the object to be transported according to the control parameters, thereby improving the control efficiency of the robotic arm.

[0006] One embodiment of this application provides a control method, device, electronic device, and storage medium for a robotic arm. In this method, point cloud data of the object to be transported is first determined using image processing technology. Then, the relative position of the object to be transported and the end effector in three-dimensional space, the shape of the object to be transported, and the size of the object to be transported are determined, providing a data basis for determining the control parameters of the robotic arm.

[0007] One embodiment of this application provides a control method, device, electronic device, and storage medium for a robotic arm. In this method, a preset AI model is used to determine the relative position of the object to be transported and the end effector in three-dimensional space, as well as the shape and size of the object to be transported. This makes the determined relative position of the object to be transported and the end effector in three-dimensional space, the shape and size of the object to be transported more accurate, and provides a strong data foundation for determining accurate control parameters.

[0008] One embodiment of this application provides a control method, device, electronic device, and storage medium for a robotic arm. The relative positions of the arm can determine the rotation parameters, extension parameters, and movement parameters of the arm. The placement parameters can be used to determine the gripping parameters of the end effector, thereby determining control parameters suitable for the current specific handling conditions to ensure the smooth completion of the handling.

[0009] One embodiment of this application provides a control method, device, electronic device, and storage medium for a robotic arm. The gripping angle and gripping force of the end effector are determined by the placement parameters and weight of the object to be transported, thereby enabling the determination of accurate control parameters for the robotic arm to ensure the smooth completion of the transport.

[0010] One embodiment of this application provides a control method, device, electronic device, and storage medium for a robotic arm. The weight of the object to be transported can be determined through the image data, which helps to determine the gripping force of the end effector, ensuring that the end effector can accurately grip the object to be transported, so that the transport can be completed smoothly.

[0011] One embodiment of this application provides a control method, device, electronic device, and storage medium for a robotic arm. The weight of the object to be moved is determined through interaction between the electronic device and the user, enabling timely knowledge of the weight of the object to be moved when it cannot be known through image data, thus providing a data basis for determining the precise gripping force.

[0012] One embodiment of this application provides a control method, device, electronic device, and storage medium for a robotic arm, wherein the arm's movement speed is determined by current weather information, making the arm's movement speed closely related to the current weather information, which helps to determine a suitable movement speed for the robotic arm to ensure the smooth completion of the transport.

[0013] One embodiment of this application provides a control method, device, electronic device, and storage medium for a robotic arm. By adjusting the control parameters of the robotic arm using sensor data, the control parameters of the robotic arm can be made more accurate, ensuring the smooth completion of the transport process.

[0014] To achieve at least one of the above objectives, in a first aspect, one embodiment of this application provides a control method for a robotic arm applied to an electronic device, the robotic arm being mounted on a vehicle, comprising:

[0015] Image data of the objects to be moved placed on the vehicle are acquired by the PLC;

[0016] Based on the image data, the control parameters of the robotic arm are determined;

[0017] The control parameters are sent to the PLC so that the PLC controls the robotic arm according to the control parameters to grab the object to be transported and transport it to the designated location.

[0018] Secondly, one embodiment of this application provides a control device for a robotic arm, the device being disposed in an electronic device, the robotic arm being disposed on a vehicle, comprising:

[0019] The acquisition module is used to acquire image data of the objects to be transported placed on the vehicle via the PLC;

[0020] The determining module is used to determine the control parameters of the robotic arm based on the image data;

[0021] The control module is used to send the control parameters to the PLC, so that the PLC controls the robotic arm according to the control parameters to grasp the object to be transported and transport it to the designated position.

[0022] Thirdly, embodiments of this application provide an electronic device, including a memory and a processor. The memory stores a computer program that can run on the processor, and the processor executes the program to implement some or all of the steps in the above-described method.

[0023] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements some or all of the steps in the above-described method.

[0024] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this application. Attached Figure Description

[0025] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0026] Figure 1 A flowchart illustrating an optional robotic arm control method provided in an embodiment of this application;

[0027] Figure 2 A flowchart illustrating an example of an optional robotic arm control method provided in this application embodiment;

[0028] Figure 3 A flowchart illustrating an example of an optional robotic arm control method provided in this application embodiment;

[0029] Figure 4 A schematic diagram of the structure of an optional control device for a robotic arm provided in an embodiment of this application;

[0030] Figure 5 This is a schematic diagram of an optional electronic device provided in an embodiment of this application. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application are further described in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0032] In the following description, references to "some embodiments" refer to a subset of all possible embodiments. It is understood that "some embodiments" may be the same or different subsets of all possible embodiments and may be combined with each other without conflict. The terms "first / second / third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0033] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terminology used herein is for descriptive purposes only and is not intended to limit the scope of this application.

[0034] Currently, robotic arms on vehicles are controlled using mechanical buttons. However, this method requires a human to operate the mechanical buttons to control the robotic arm, resulting in low control efficiency.

[0035] To improve the control efficiency of robotic arms on vehicles, this application provides a control method for robotic arms. This method is applied to electronic devices, with the robotic arm mounted on a vehicle. The robotic arm is controlled by a programmable logic controller (PLC) on the vehicle. Here, the electronic device controls the robotic arm through the PLC.

[0036] Figure 1 A flowchart illustrating an optional robotic arm control method provided in an embodiment of this application is shown below. Figure 1 As shown, the control method for this robotic arm may include:

[0037] S101: Acquire image data of the objects to be transported placed on the vehicle via PLC;

[0038] The robotic arm control method provided in this application embodiment can be applied to the use of robotic arms on vehicles to transport or sort goods. For example, the vehicle can be a logistics vehicle used for transporting goods in autonomous driving, or a rescue vehicle used for emergency material transport in natural disasters or emergencies. Here, this application embodiment does not specifically limit it.

[0039] When the vehicle's robotic arm needs to move an object to a designated location, it needs to acquire image data of the object placed on the vehicle via a PLC.

[0040] In this embodiment, after the vehicle is parked, the user operates the electronic device's interface, causing the electronic device to receive a moving instruction and trigger the PLC to acquire image data of the object to be moved placed on the vehicle. Alternatively, after the vehicle is parked, the user operates the electronic device's interface, causing the electronic device to receive the designated location of the object to be moved and trigger the PLC to acquire image data of the object to be moved placed on the vehicle. Of course, other methods can also be used to trigger the acquisition of image data of the object to be moved placed on the vehicle via the PLC; this embodiment does not specifically limit the methods used in this application.

[0041] As can be seen, in the process of the electronic device acquiring image data of the object to be transported placed on the vehicle through the PLC, after receiving a trigger command, the trigger command is sent to the PLC, the PLC controls the camera on the vehicle to take pictures of the object to be transported, thereby acquiring image data of the object to be transported, and sending the image data to the PLC. The electronic device then acquires the image data of the object to be transported placed on the vehicle through the PLC. Here, the camera can be located on the upper side of the carriage to take pictures of the object to be transported in the carriage.

[0042] The object to be transported can be of a regular shape or an irregular shape. This application embodiment does not specifically limit this.

[0043] S102: Determine the control parameters of the robotic arm based on the image data;

[0044] Image data of the object to be transported can be obtained through S101. In S102, the electronic device determines the control parameters of the robotic arm based on the image data. Here, the position parameters and placement parameters of the object to be transported can be determined based on the image data, thereby determining the control parameters of the robotic arm when transporting the object.

[0045] The aforementioned control parameters may include: rotation parameters, extension parameters, and movement parameters of the robotic arm; and may also include gripping parameters of the end effector in the robotic arm, where rotation parameters may be rotation angles, extension parameters may be extension lengths, movement parameters may be movement speeds and movement trajectories, and gripping parameters may be gripping angles and gripping forces.

[0046] It should be noted that in the above process of determining the control parameters of the robotic arm based on image data, image processing is required to obtain the position and placement parameters of the object to be transported, and then the control parameters of the robotic arm can be determined when transporting the object.

[0047] Here, image processing can be done using traditional image processing techniques or by using a model-based approach; this application does not specifically limit the specific methods used.

[0048] S103: Send control parameters to the PLC so that the PLC controls the robotic arm according to the control parameters to grab the object to be transported and transport it to the designated location.

[0049] The control parameters of the robotic arm can be determined through S102. In S103, the electronic device sends the determined control parameters of the robotic arm to the PLC, so that the PLC controls the robotic arm to grasp the object to be transported and transport it to the designated position according to the control parameters.

[0050] It should be noted that the aforementioned electronic devices can be Android devices. The Android system provides an intuitive user interface, allowing users to trigger the control methods of the robotic arm via touchscreen or voice. The Android system can also communicate with the PLC via network to achieve remote control and monitoring functions.

[0051] The PLC, as the core controller of the robotic arm, is responsible for receiving instructions from the Android system and converting them into actions that the robotic arm can perform. The PLC is connected to the sensors and actuators of the robotic arm through input / output (I / O) modules to achieve precise control of the robotic arm. A specific control program runs inside the PLC, which controls the movement of the robotic arm based on input signals (such as sensor data) and output instructions (such as the movements of the robotic arm).

[0052] In this way, by processing the image data of the object to be transported by electronic devices, the control parameters of the robotic arm when transporting the object are determined, and the parameters are sent to the PLC so that the PLC can control the robotic arm according to the control parameters to complete the transport of the object.

[0053] In order to determine the control parameters of the robotic arm, in one alternative embodiment, S102 may include:

[0054] Based on the image data, determine the relative position of the object to be transported and the end effector of the robotic arm in three-dimensional space, as well as the placement parameters of the object to be transported;

[0055] The control parameters of the robotic arm are determined based on the relative positions of the object to be transported and the end effector of the robotic arm in three-dimensional space, as well as the placement parameters of the object to be transported.

[0056] Understandably, after acquiring the image data of the object to be transported, the relative position of the object and the end effector of the robotic arm in three-dimensional space can be determined based on the image data, and the placement parameters of the object can also be obtained. These placement parameters include the shape and size of the object. In other words, the relative position of the object and the end effector in three-dimensional space, the shape of the object, and the size of the object are determined first.

[0057] Knowing the relative positions of the object to be transported and the end effector in three-dimensional space, as well as the shape and size of the object to be transported, we can calculate how to control the robotic arm to move the end effector to the position of the object to be transported. We can also calculate how the end effector should grasp the object to be transported to successfully move it to the designated position. In this way, we can determine the control parameters of the robotic arm.

[0058] In this way, the relative position of the object to be transported and the end effector in three-dimensional space, the shape of the object to be transported, and the size of the object to be transported can be determined through the above image data. Then, the control parameters of the robotic arm can be determined, so that the PLC can control the robotic arm to transport the object to be transported according to the control parameters, thereby improving the control efficiency of the robotic arm.

[0059] To determine the relative position of the object to be transported and the end effector in three-dimensional space, and the placement parameters of the object to be transported, in one optional embodiment, determining the relative position of the object to be transported and the end effector of the robotic arm in three-dimensional space, and the placement parameters of the object to be transported, based on image data, may include:

[0060] The depth image in the image data is converted to obtain the point cloud data of the object to be moved;

[0061] Based on the point cloud data of the object to be transported, the relative position of the object to be transported and the end effector of the robotic arm in three-dimensional space, as well as the placement parameters of the object to be transported, are determined.

[0062] Understandably, the image data mentioned above includes a depth image, which reflects the distance between the object to be transported and the camera. Here, by converting the depth image in the image data, point cloud data of the object to be transported can be obtained. The point cloud data reflects the outline of the object to be transported in space. Therefore, based on the point cloud data of the object to be transported, the relative position of the object to be transported and the end effector in three-dimensional space, the shape of the object to be transported, and the size of the object to be transported can be determined.

[0063] In this way, by using image processing technology to first determine the point cloud data of the object to be transported, the relative position of the object to be transported and the end effector in three-dimensional space, the shape of the object to be transported and the size of the object to be transported are determined, providing a data basis for determining the control parameters of the robotic arm.

[0064] In addition, to determine the relative position of the object to be transported and the end effector in three-dimensional space, and the placement parameters of the object to be transported, in an optional embodiment, determining the relative position of the object to be transported and the end effector of the robotic arm in three-dimensional space, and the placement parameters of the object to be transported, based on image data, may include:

[0065] The image data is input into a preset artificial intelligence (AI) model to obtain the relative position of the object to be transported and the end effector of the robotic arm in three-dimensional space, as well as the placement parameters of the object to be transported.

[0066] Understandably, after acquiring the image data, it is input into a preset AI model, and the output is the relative position of the object to be transported and the end effector in three-dimensional space, the shape of the object to be transported, and the size of the object to be transported.

[0067] The aforementioned preset AI model is pre-built and trained. Specifically, data collection and processing are carried out first: the construction of the AI ​​model begins with data collection. The quality, quantity and diversity of the data are crucial to the performance of the model. Data processing includes steps such as data cleaning, formatting, and feature selection to ensure that the data is suitable for model training.

[0068] Then, model selection and design are carried out: based on the nature of the problem and the characteristics of the data, an appropriate model architecture is selected. For example, for image classification tasks, a Convolutional Neural Network (CNN) might be chosen; for natural language processing tasks, a Recurrent Neural Network (RNN) or Transformer model architecture might be chosen.

[0069] Next, model training is performed: the processed data is used to train the model. The training process includes forward propagation, loss calculation, backpropagation, and parameter updates. Through multiple iterations, the model gradually learns the inherent patterns and characteristics of the data.

[0070] Finally, model evaluation and optimization are performed: After training, the model needs to be evaluated to understand its performance. Evaluation metrics may include accuracy, recall, F1 score, etc. Based on the evaluation results, the model can be optimized, such as adjusting hyperparameters and adding regularization terms, thus obtaining the AI ​​model.

[0071] In this way, the relative position of the object to be transported and the end effector in three-dimensional space, the shape of the object to be transported, and the size of the object to be transported are determined by the preset AI model. This makes the determined relative position of the object to be transported and the end effector in three-dimensional space, the shape of the object to be transported, and the size of the object to be transported more accurate, providing a strong data foundation for determining accurate control parameters.

[0072] To determine the control parameters of the robotic arm, in one optional embodiment, the control parameters of the robotic arm are determined based on the relative positions of the object to be transported and the end effector of the robotic arm in three-dimensional space and the placement parameters of the object to be transported. This determination may include:

[0073] Based on the relative positions of the object to be transported and the end effector of the robotic arm in three-dimensional space, the rotation parameters, extension parameters, and movement parameters of the robotic arm are determined.

[0074] Based on the placement parameters of the object to be moved, determine the gripping parameters of the end effector in the robotic arm.

[0075] Understandably, based on the relative positions of the object to be transported and the end effector in three-dimensional space, it is possible to calculate how the end effector needs to be controlled when it reaches the position to be transported. Here, the end effector can be rotated, extended, or moved. Based on the above relative positions, the rotation parameters, extension parameters, and movement parameters of the end effector in the robot arm can be determined.

[0076] In addition, based on the shape and size of the object to be transported, it is possible to calculate how the end effector needs to be controlled when it grasps the object. Here, the grasping angle and grasping force of the end effector can be determined to ensure successful grasping of the object to be transported.

[0077] In this way, the rotation parameters, extension parameters, and movement parameters of the arm can be determined by the relative position. The gripping parameters of the end effector can be determined by the above placement parameters, thereby determining the control parameters suitable for the current specific handling conditions to ensure the smooth completion of the handling.

[0078] Furthermore, in order to determine the grasping parameters of the end effector in the robotic arm, in one optional embodiment, determining the grasping parameters of the end effector in the robotic arm based on the placement parameters of the object to be transported may include:

[0079] Based on the placement parameters of the object to be transported and the obtained weight of the object, the gripping angle and gripping force of the end effector in the robotic arm are determined.

[0080] Understandably, the above-mentioned gripping parameters may include gripping angle and gripping force. The gripping angle is related to the placement parameters of the object to be transported, while the gripping force is usually related to the weight of the object to be transported. Therefore, the gripping angle and gripping force of the end effector of the robotic arm can be determined based on the placement parameters and weight of the object to be transported.

[0081] Among them, the gripping surface of the object to be moved can be determined based on the placement parameters of the object, so as to determine the gripping angle. The required gripping degree of the object to be moved can be determined based on its weight, so as to determine the gripping force of the end effector.

[0082] In this way, by determining the placement parameters and weight of the object to be transported, the gripping angle and gripping force of the end effector can be determined, thus providing accurate control parameters for the robotic arm to ensure the smooth completion of the transport.

[0083] To determine the weight of the object to be moved, in one alternative embodiment, the method may further include:

[0084] Based on the image data, determine the type of object to be moved;

[0085] Determine the volume of the object to be moved based on its placement parameters;

[0086] The weight of the object to be moved is determined based on the density and volume of the object.

[0087] Understandably, the type of object to be moved can be determined based on the above image data. For example, the type of object to be moved can be known through image recognition. Taking mineral water as an example, the type of object to be moved can be known to be mineral water through image recognition.

[0088] The volume of the object to be moved can be determined based on its placement parameters. Here, after knowing the shape and size of the object, the volume can be calculated. Of course, if the image data contains the volume of the object, the volume can also be obtained through image recognition. After knowing the type of the object, the density corresponding to that type can be determined. Multiplying the density by the volume of the object gives the mass of the object.

[0089] Of course, if the image data contains the weight of the object to be moved, the weight of the object to be moved can be obtained through image recognition.

[0090] Thus, the weight of the object to be transported can be determined using the aforementioned image data, which helps to determine the gripping force of the end effector, ensuring that the end effector can accurately grip the object to be transported, so that the transport can be completed smoothly.

[0091] In addition to determining the weight of the object to be transported using the methods described above, in an optional embodiment, the method may further include:

[0092] In response to the input of the weight option for the object to be moved, the weight of the object to be moved is determined.

[0093] Understandably, the electronic device's interface has an option to enter the weight of the object to be moved. Users can enter the weight of the object by operating on the interface, so that the electronic device knows the weight of the object to be moved.

[0094] In this way, the weight of the object to be moved can be determined through interaction between electronic devices and users, so that the weight of the object to be moved can be known in a timely manner when it cannot be known through image data, thus providing a data basis for determining the precise gripping force.

[0095] In addition, in order to adopt different movement speeds in different outdoor environments, in an optional embodiment, the above method may further include:

[0096] Based on the obtained current weather information, determine the movement speed among the movement parameters of the robotic arm.

[0097] Understandably, the above control parameters may include the movement speed of the arm. Here, the movement speed in the arm movement parameters is determined based on the current weather information obtained. The current weather information includes one or more of the following: wind force level, rainfall, snowfall, and visibility level.

[0098] In other words, the movement speed of the arm can be determined based on wind force, rainfall, snowfall, and / or visibility level. For example, higher wind force corresponds to lower movement speed, less rainfall corresponds to lower movement speed, less snowfall corresponds to lower movement speed, and lower visibility corresponds to lower movement speed. When determining the arm's movement speed based on at least two of the current weather information, a weight value can be set for each item, and the movement speed can be determined by weighted summation. Alternatively, a correspondence between at least two items and the movement speed can be established to determine the movement speed of the robotic arm's movement parameters. Here, this application embodiment does not specifically limit this approach.

[0099] In this way, by determining the movement speed of the robotic arm based on the current weather information, the movement speed of the robotic arm is closely related to the current weather information, which helps to determine the appropriate movement speed for the robotic arm to ensure the smooth completion of the handling.

[0100] To determine more precise control parameters, in one alternative embodiment, the above method may further include:

[0101] The PLC receives data from sensors on the robotic arm.

[0102] Based on the sensor data, the control parameters of the robotic arm are adjusted to obtain the control parameters of the robotic arm again.

[0103] Understandably, in the control method of the robotic arm, sensor data from the robotic arm can also be received by the PLC. The aforementioned sensor data can be sensor data when the end effector grasps the object to be transported, such as the humidity and temperature of the grasping surface of the object to be transported, the humidity and temperature of the current environment, the actual moving speed of the object to be transported, the weight of the object to be transported, etc.

[0104] Here, the control parameters of the robotic arm, which have already been determined, can be adjusted using this sensor data, thereby obtaining the control parameters of the robotic arm again for the purpose of handling the next object to be moved.

[0105] In this way, by adjusting the control parameters of the robotic arm through sensor data, the control parameters of the robotic arm can be made more accurate, ensuring the smooth completion of the handling.

[0106] The following examples illustrate the control method of the robotic arm described in one or more of the above embodiments.

[0107] In related technologies, applications of the Android system include: Case studies of applying the Android system to various smart devices, including smartphones and tablets. These devices provide rich user interfaces and interaction methods through the Android system, offering insights for the development of intelligent operating systems for in-vehicle robotic arms. Regarding PLC control technology: PLCs, as core control devices in industrial automation, have been widely used in various robotic arm control systems. PLCs are stable, reliable, and flexible in programming, capable of receiving control commands from a host computer and controlling the precise movement of the robotic arm. Regarding wireless communication and data transmission: Wireless communication modules such as Bluetooth and Wireless Fidelity (Wi-Fi) have been widely used for data transmission between various smart devices. Regarding intelligent algorithms and decision-making: In the field of industrial automation, intelligent algorithms such as machine learning and deep learning have begun to be applied to optimize control strategies and improve system performance.

[0108] However, while the intelligent operating system for vehicle-mounted robotic arms based on Android and PLC control combines the advantages of mobile operating systems and industrial automation control technologies, it may still have some technical problems or shortcomings. The following is an analysis of these technical problems or shortcomings, and the most significant technical issue that this example may aim to address:

[0109] 1. System Compatibility and Integration: Incompatibility issues may exist between the communication protocols and data formats between the Android system and the PLC, leading to low data transmission efficiency or inaccurate transmission of control commands. Low system integration, with inconsistent interfaces and communication protocols between modules, increases the complexity of system development and maintenance.

[0110] 2. Real-time performance and stability: Vehicle-mounted robotic arms require rapid response and precise control, but the wireless communication modules in related technologies may suffer from latency or packet loss, affecting the system's real-time performance and stability. The PLC control unit may also experience performance bottlenecks when processing complex control algorithms, leading to a decrease in control accuracy and response speed.

[0111] 3. Application and Optimization of Intelligent Algorithms: Intelligent algorithms in related technologies may not be fully adapted to the working environment and task requirements of onboard robotic arms, resulting in suboptimal control strategies. Parameter adjustment and optimization of intelligent algorithms may require a large amount of experimental data and experience accumulation, increasing the difficulty of system development and debugging.

[0112] 4. Convenience of Human-Computer Interaction: The user interface may not be user-friendly, and touch or voice control methods may suffer from misoperation or inaccurate recognition. The lack of an intuitive visual interface and real-time feedback mechanism makes it difficult for users to understand the current status and control effect of the robotic arm.

[0113] In view of the problems or shortcomings existing in the above-mentioned related technologies, the most important technical problem to be solved in this example may include:

[0114] 1. Improve system compatibility and integration: Design a unified communication protocol and data format to ensure efficient and accurate data transmission between the Android system and the PLC. Optimize system integration by unifying the interfaces and communication protocols between various modules, reducing the complexity of system development and maintenance.

[0115] 2. Enhance real-time performance and stability: Optimize the performance of the wireless communication module, reduce latency and packet loss, and improve the real-time performance and stability of the system. Enhance the processing power of the PLC control terminal to ensure that complex control algorithms can be executed in real time and accurately.

[0116] 3. Optimize the application of intelligent algorithms: Design more optimized control strategies based on the working environment and task requirements of the vehicle-mounted robotic arm. Provide tools for parameter adjustment and optimization of intelligent algorithms to reduce the difficulty of system development and debugging.

[0117] 4. Improve the convenience of human-computer interaction: Design a user-friendly interface to enhance the accuracy and convenience of touch or voice control. Provide an intuitive visual interface and real-time feedback mechanism so that users can easily understand the current status and control effect of the robotic arm.

[0118] Figure 2 A schematic diagram illustrating an example of an optional robotic arm control system provided in this application embodiment, as shown below. Figure 2 As shown, the electronic device 21 of the control system 200 of the robotic arm is equipped with an application (APP) 211, an application service 212, a network communication module 213, a Bluetooth communication module 214, and a cross-platform communication module 215 for controlling the robotic arm. The APP 211 is equipped with a user interface (UI) control interface and a voice control unit. When the APP 211 is opened, the UI control interface and the voice control unit start working. The control system 200 of the robotic arm may also include a PLC control unit 22, an I / O module 23, and a mechanical execution unit 24.

[0119] Service212 sends execution instructions to PLC control unit 22 through network communication module 213, Bluetooth communication module 214 and cross-platform communication module 215. PLC control unit 22 receives and responds to the instructions, and sends control instructions to mechanical execution unit 24 through I / O module 23 to execute mechanical terminal.

[0120] Based on the above Figure 2The structure includes a signal conversion module in electronic device 21. Figure 3 A flowchart illustrating an example of an optional robotic arm control method provided in this application embodiment, as shown below. Figure 3 As shown, the method may include:

[0121] S301: The user clicks on the UI interface;

[0122] S302: The APP receives the operation of the up button and sends the operation information to the Service;

[0123] S303: The Service receives the operation information and returns a message to the App;

[0124] S304: The Service generates a lift command and sends it to the signal conversion module;

[0125] S305: The signal conversion module receives the lift command and sends a Service return message;

[0126] S306: The signal conversion module performs ladder logic conversion on the lift command to obtain the host computer signal;

[0127] S307: The signal conversion module sends signals from the host computer to the PLC;

[0128] S308: The PLC receives a signal from the host computer and sends a return message to the APP.

[0129] S309: PLC control I / O module and sensor for robotic arm;

[0130] S310: The PLC sends a pre-production command to the external machinery;

[0131] S311: External machine executes status callback to Service.

[0132] In this example, focusing on the scenario of cargo handling and sorting in autonomous vehicles, automated cargo handling and sorting are required in autonomous logistics vehicles. These vehicles need to travel between different stations and, as needed, move goods from the vehicle to designated locations or load goods onto the vehicle from outside. This demands that the robotic arm possess high flexibility and accuracy to ensure that goods are not damaged during handling and to efficiently complete sorting tasks.

[0133] This example utilizes an intuitive user interface provided by the Android system, allowing operators to easily control the movement trajectory and force of the robotic arm. Simultaneously, PLC control technology ensures that the robotic arm can execute actions quickly and accurately upon receiving commands. Furthermore, by integrating intelligent algorithms, the system can automatically adjust its control strategy based on the shape, size, and weight of the goods to optimize handling and sorting efficiency.

[0134] In this example, focusing on the scenario of emergency supplies transport in rescue and engineering vehicles, during natural disasters or emergencies, rescue vehicles need to quickly move emergency supplies (such as food, water, and medical equipment) from the vehicle to the disaster area. This requires robotic arms to complete tasks quickly and accurately in complex environments to save valuable time and reduce the burden on rescue personnel.

[0135] This example demonstrates how the Android system provides remote control functionality, allowing rescue personnel to control the robotic arm from a safe location. Simultaneously, PLC control technology ensures the robotic arm responds quickly to commands and accurately moves supplies to designated locations. Furthermore, intelligent algorithms can automatically adjust control strategies based on the environment to handle potential obstacles or uneven terrain.

[0136] In modern agriculture, agricultural vehicles are required to automate crop harvesting and planting. This necessitates that robotic arms can identify different types of crops and automatically adjust harvesting or planting strategies based on their growth conditions. Furthermore, given the typically complex and variable agricultural environment, robotic arms also need to possess high flexibility and adaptability.

[0137] Using this example, by integrating image recognition and machine learning algorithms, the system can automatically identify the type and growth status of crops, and adjust the robotic arm's trajectory and control strategy accordingly. Simultaneously, PLC control technology ensures that the robotic arm can accurately and stably perform harvesting or planting actions upon receiving instructions. Furthermore, the Android system provides real-time monitoring and feedback, allowing operators to understand the robotic arm's working status and crop growth at any time.

[0138] In the field of industrial robotics, precision assembly is a task that demands extremely high precision in the control of robotic arms. For example, in the manufacturing process of electronic products, tiny components need to be precisely assembled. This requires robotic arms to have extremely high positioning accuracy and stability to ensure assembly quality.

[0139] This example demonstrates how optimizing the PLC control algorithm and integrating intelligent algorithms enables the system to achieve precise control of the robotic arm, meeting the requirements of precision assembly tasks. Simultaneously, the Android system provides a user-friendly interface and real-time monitoring capabilities, allowing operators to easily control the robotic arm's movement and monitor the assembly process.

[0140] In this example, the implementation principle of the vehicle-mounted robotic arm system is mainly based on the Android system, PLC control, and the application of intelligent algorithms. The Android system serves as the user interface and remote control platform, the PLC is responsible for the precise control of the robotic arm, and the intelligent algorithms are used to optimize the control strategy and improve the system's flexibility.

[0141] For the Android system: The Android system provides an intuitive user interface, allowing users to control the movement of the robotic arm via touchscreen or voice commands. The Android system can also communicate with a PLC via a network to achieve remote control and monitoring functions.

[0142] For PLC control: The PLC, as the core controller of the robotic arm, is responsible for receiving instructions from the Android system and converting them into actions that the robotic arm can perform. The PLC connects to the robotic arm's sensors and actuators through input / output (I / O) modules to achieve precise control of the robotic arm. A specific control program runs internally within the PLC, which controls the movement of the robotic arm based on input signals (such as sensor data) and output instructions (such as the robotic arm's movements).

[0143] Regarding intelligent algorithms: Intelligent algorithms are used to optimize control strategies and improve the flexibility and adaptability of robotic arms. For example, through image recognition and machine learning algorithms, the system can automatically identify objects in the work environment and automatically adjust the robotic arm's motion trajectory and control strategy based on the object's shape, size, and position. Intelligent algorithms can also automatically adjust the robotic arm's speed and force according to the requirements of the task to improve work efficiency and quality.

[0144] The data flow in this example includes three stages: data generation, processing, and output.

[0145] Step A: Data Generation

[0146] The data is primarily generated by the robotic arm's sensors and actuators. Sensors detect objects, their positions, speeds, and other information in the working environment, converting this information into electrical signals that are sent to the PLC. Actuators are responsible for translating the PLC's instructions into the robotic arm's actual movements, such as rotation, extension, and gripping.

[0147] Step B: Data Processing

[0148] After receiving the sensor data, the PLC performs preliminary processing and conversion, transforming the electrical signal into a digital signal, and then performs logical operations and judgments according to the requirements of the control program. The PLC then sends the processed data to the Android system for real-time monitoring and remote control by the user.

[0149] At the same time, intelligent algorithms will also process and analyze sensor data to optimize control strategies and improve system flexibility.

[0150] Step C: Data Output:

[0151] The final output data mainly includes the robotic arm's motion commands and status information. The motion commands are generated by the PLC based on the control program and sensor data, and sent to the robotic arm's actuators via the output module. The status information includes the robotic arm's current position, speed, force, and other information, which is fed back to the Android system in real time for user monitoring and adjustments.

[0152] The Android system mentioned above is an open-source operating system based on the Linux kernel, primarily used for mobile devices (such as smartphones and tablets), but it can also be used for embedded devices such as in-vehicle robotic arm systems.

[0153] The PLC mentioned above is a programmable logic controller, a type of controller used in industrial automation that can receive input signals and output control signals according to preset logic rules.

[0154] The aforementioned I / O module is an input / output module used to connect the PLC and external devices such as sensors and actuators of the robotic arm to realize signal transmission and control.

[0155] The aforementioned intelligent algorithms refer to a series of algorithms that utilize computer technology and mathematical methods to optimize and make decisions on specific problems. In vehicle-mounted robotic arm systems, intelligent algorithms are mainly used to optimize control strategies and improve system flexibility.

[0156] As can be seen, the above example integrates the Android system and PLC control into a novel intelligent operating system for vehicle-mounted robotic arms. This system combines the user-interactive advantages of Android with the precise execution capabilities of PLC control. Through this integration, users can intuitively and conveniently control the robotic arm via the Android system, while the PLC ensures the precise execution of commands, achieving efficient and flexible control of the vehicle-mounted robotic arm. This solves the problems of complex operation and poor user experience in traditional robotic arm control systems, improving the system's usability and operational efficiency.

[0157] Automated control and optimization based on intelligent algorithms introduces advanced intelligent algorithms, such as machine learning and deep learning, to optimize the control strategy and execution efficiency of robotic arms. These algorithms can adjust the movement trajectory, speed, and force of the robotic arm in real time according to the working environment and task requirements, achieving automated control and optimization. This significantly improves the flexibility and adaptability of the robotic arm, enabling it to efficiently complete tasks in complex and ever-changing environments.

[0158] A real-time data monitoring and feedback system was designed to monitor the robotic arm's motion status and operating environment in real time, and then feed the data back to the Android system. Users can view this data in real time through the Android system and make adjustments and optimizations as needed. This enhances the system's transparency and controllability, allowing users to understand the robotic arm's working status in a timely manner and ensuring safe and efficient operation.

[0159] The highly integrated modular design integrates key components such as the Android system, PLC controller, sensors, and actuators into a compact and efficient system. This design simplifies the installation and maintenance process, reduces system complexity and cost, improves system reliability and maintainability, and lowers user operation and maintenance costs and time.

[0160] Wireless communication and remote control capabilities are supported, allowing users to remotely access the Android system via mobile devices or computers to control and monitor the robotic arm. This functionality enables users to flexibly control the robotic arm from different locations and at different times, improving the system's flexibility and convenience. It also broadens the system's application scenarios and scope, allowing the vehicle-mounted robotic arm system to function in a wider range of environments.

[0161] This example can also use the following alternative:

[0162] Alternative Solution 1: Robotic Arm System Based on Traditional Industrial Controllers

[0163] Key points: It uses a traditional industrial controller (such as a PLC) as the core controller. It is equipped with basic sensors and actuators to achieve basic motion control of the robotic arm. It lacks advanced intelligent algorithms and autonomous learning capabilities. This results in relatively low cost and ease of implementation. The technology is mature and highly reliable. However, it lacks intelligence and adaptability, making it difficult to cope with complex and changing operating environments. It requires complex manual programming and debugging.

[0164] For example, an automobile manufacturing plant uses a robotic arm system based on traditional industrial controllers for its automated assembly line. This system achieves precise motion control of the robotic arm through a PLC controller, but lacks intelligent algorithm support, requiring detailed manual programming and debugging.

[0165] Alternative Option 2: A robotic arm based on a vision servoing system

[0166] Key points: A visual servo system is used as the primary control method for the robotic arm. Information about the working environment is acquired in real time through visual sensors such as cameras. The visual servo system adjusts the robotic arm's trajectory in real time based on this visual information. This provides high intelligence and adaptability, enabling it to cope with complex and changing working environments. It also improves the accuracy and flexibility of the robotic arm. However, the cost is high, and the development and implementation of the visual servo system are complex. It is also sensitive to environmental factors such as lighting and occlusion, which may affect the system's stability and accuracy.

[0167] For example, a robotics manufacturer has developed a robotic arm based on a vision servo system for the automated assembly of precision parts. This system uses cameras to acquire the position and orientation information of the parts in real time, and adjusts the robotic arm's trajectory accordingly, achieving high-precision and flexible assembly operations.

[0168] Alternative Option 3: Machine Learning-Based Robotic Arm Control System

[0169] Key points: Optimizing the control strategy of the robotic arm using machine learning algorithms. Through extensive training data, the robotic arm can autonomously learn and optimize its control strategy. It possesses high intelligence and adaptability, enabling it to handle complex tasks. This strong self-learning and optimization capability continuously improves operational efficiency and accuracy. It also exhibits high intelligence and adaptability, allowing it to cope with complex working environments. However, this requires a large amount of training data and computing resources, resulting in high R&D costs. The implementation and debugging of machine learning algorithms are complex and require specialized technical personnel.

[0170] For example, a research team developed a machine learning-based robotic arm control system for automated handling and sorting operations. Through extensive training data, the system enables the robotic arm to autonomously learn and optimize handling and sorting strategies, improving operational efficiency and accuracy. Furthermore, the system possesses high intelligence and adaptability, capable of handling and sorting objects of varying sizes and shapes.

[0171] Alternative Option 4: Robotic Arm System Based on Multi-Sensor Fusion

[0172] Key points: Information fusion is achieved using multiple sensors (such as force sensors, position sensors, and vision sensors). By fusing information from multiple sensors, precise monitoring and control of the robotic arm's motion state can be realized, improving the reliability and stability of the robotic arm. This enhances the robotic arm's reliability and stability, enabling it to cope with complex and changing working environments. Information fusion from multiple sensors makes the system more accurate and reliable. However, it is costly, requiring the configuration of multiple sensors and corresponding information processing modules. The information fusion and coordination between sensors requires complex algorithms and technical support.

[0173] For example, an automation equipment manufacturer has developed a robotic arm system based on multi-sensor fusion for automated welding operations. This system achieves precise monitoring and control of the welding process by fusing information from force sensors, position sensors, and vision sensors. Simultaneously, the system can adjust welding parameters and the robotic arm's movement trajectory in real time based on the force and position information generated during welding, improving welding quality and efficiency.

[0174] In other words, through the examples above, by integrating advanced PLC control systems and intelligent algorithms, robotic arms can autonomously and precisely complete various production tasks, such as assembly, welding, and painting, significantly improving the automation level of the production line. This automated operation method reduces human intervention, increases production efficiency, and lowers the human error rate, thus significantly improving production efficiency.

[0175] The robotic arm employs advanced sensors and optimized algorithms to achieve high-precision positioning and operation, ensuring that every task achieves the expected results. This high-precision positioning capability enables the robotic arm to operate stably in confined or complex environments, improving production efficiency and product quality. High-precision positioning has been achieved.

[0176] Robotic arms can replace human labor in performing heavy, dangerous, or repetitive tasks, thus saving human resources. Companies can then allocate these saved resources to other, more valuable tasks, improving overall operational efficiency and ultimately conserving human resources.

[0177] Robotic arms can precisely control the amount of materials used during operations, reducing material waste. For example, in painting operations, the robotic arm can automatically adjust the spraying path based on the 3D model of the car body, ensuring even coverage of every corner and reducing over-coating and under-coating. This reduces material waste and lowers production costs.

[0178] The robotic arm maintains consistency and stability during operation, ensuring that the quality of each task meets standards. This consistency and stability give the robotic arm a significant advantage in producing high-quality products.

[0179] Robotic arms can autonomously complete production tasks, reducing the possibility of human error. For example, in assembly operations, robotic arms can precisely assemble parts, avoiding quality problems caused by human error. Reducing human error improves product quality.

[0180] The robotic arm is equipped with multiple safety protection mechanisms, such as force control technology, vision systems, and safety sensors, which can monitor the working environment in real time and take corresponding safety measures. These safety protection mechanisms ensure the safety and reliability of the robotic arm during operation.

[0181] The robotic arm system features fault warning and diagnostic capabilities, enabling timely detection and handling of potential malfunctions. This functionality allows companies to take proactive measures to avoid production interruptions, improving the stability and reliability of production lines and enhancing safety and reliability.

[0182] The robotic arm system employs a modular design, allowing for easy disassembly and replacement of various components. This modular design reduces system maintenance and time costs, and improves system maintainability.

[0183] The software of the robotic arm system can be easily upgraded and updated to adapt to new production needs and technological changes. This software upgrade capability enables the robotic arm system to maintain long-term competitiveness and viability. It is also easy to maintain and upgrade.

[0184] Based on the foregoing embodiments, this application provides a control device for a robotic arm. The device includes various units and modules included in each unit, which can be implemented by a processor in a computer device; of course, it can also be implemented by specific logic circuits. In the implementation process, the processor can be a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), etc.

[0185] This application provides a control device for a robotic arm, which is installed in an electronic device, and the robotic arm is installed on a vehicle. Figure 4 This is a schematic diagram of the structure of a control device for a robotic arm provided in an embodiment of this application, as shown below. Figure 4 As shown, the control device 400 of the robotic arm includes: an acquisition module 41, a determination module 42, and a control module 43, wherein:

[0186] The acquisition module 41 is used to acquire image data of the objects to be transported placed on the vehicle via the PLC;

[0187] The determination module 42 is used to determine the control parameters of the robotic arm based on the image data;

[0188] The control module 43 is used to send control parameters to the PLC so that the PLC can control the robotic arm according to the control parameters to grab the object to be transported and transport it to the designated position.

[0189] In an optional embodiment, the determining module 42 is specifically used to: determine the relative position of the object to be transported and the end effector of the robotic arm in three-dimensional space and the placement parameters of the object to be transported based on the image data; determine the control parameters of the robotic arm based on the relative position of the object to be transported and the end effector of the robotic arm in three-dimensional space and the placement parameters of the object to be transported; wherein, the placement parameters of the object to be transported include: the shape of the object to be transported and the size of the object to be transported.

[0190] In one optional embodiment, the determining module 42 determines the relative position of the object to be transported and the end effector of the robotic arm in three-dimensional space and the placement parameters of the object to be transported based on the image data, including: converting the depth image in the image data to obtain the point cloud data of the object to be transported; and determining the relative position of the object to be transported and the end effector of the robotic arm in three-dimensional space and the placement parameters of the object to be transported based on the point cloud data of the object to be transported.

[0191] In an optional embodiment, the determining module 42 determines the relative position of the object to be transported and the end effector of the robotic arm in three-dimensional space and the placement parameters of the object to be transported based on the image data, including: inputting the image data into a preset AI model to obtain the relative position of the object to be transported and the end effector of the robotic arm in three-dimensional space and the placement parameters of the object to be transported.

[0192] In one optional embodiment, the determining module 42 determines the control parameters of the robotic arm based on the relative position of the object to be transported and the end effector of the robotic arm in three-dimensional space and the placement parameters of the object to be transported. These parameters include: determining the rotation parameters, extension parameters, and movement parameters of the arm in the robotic arm based on the relative position of the object to be transported and the end effector of the robotic arm in three-dimensional space; and determining the grasping parameters of the end effector of the robotic arm based on the placement parameters of the object to be transported.

[0193] In one optional embodiment, the determining module 42 determines the grasping parameters of the end effector in the robotic arm based on the placement parameters of the object to be transported, including: determining the grasping angle and grasping force of the end effector in the robotic arm based on the placement parameters of the object to be transported and the obtained weight of the object to be transported.

[0194] In an optional embodiment, the device is further configured to: determine the type of the object to be transported based on image data; determine the volume of the object to be transported based on the placement parameters of the object to be transported; and determine the weight of the object to be transported based on the density corresponding to the type of the object to be transported and the volume of the object to be transported.

[0195] In an alternative embodiment, the device is further configured to: determine the weight of the object to be moved in response to an input operation of a fill-in option for the weight of the object to be moved.

[0196] In an optional embodiment, the device is further configured to: determine the movement speed of the arm in the robotic arm based on the acquired current weather information; wherein the current weather information includes one or more of the following: wind force level, rainfall, snowfall, and visibility level.

[0197] In an optional embodiment, the device is further configured to: receive data from sensors on the robotic arm via a PLC; and adjust the control parameters of the robotic arm based on the sensor data to obtain the control parameters of the robotic arm again.

[0198] The descriptions of the apparatus embodiments above are similar to those of the method embodiments above, and have similar beneficial effects. In some embodiments, the functions or modules included in the apparatus provided in this application can be used to perform the methods described in the method embodiments above. For technical details not disclosed in the apparatus embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0199] It should be noted that, in the embodiments of this application, if the above-described voice control method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the related technology, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware, software, or firmware, or any combination of hardware, software, and firmware.

[0200] This application provides an electronic device, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements some or all of the steps in the above-described method.

[0201] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements some or all of the steps in the above-described method. The computer-readable storage medium can be transient or non-transient.

[0202] This application provides a computer program including computer-readable code, wherein when the computer-readable code is executed in a computer device, a processor in the computer device performs some or all of the steps in the above-described method.

[0203] This application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, it implements some or all of the steps in the above-described method. This computer program product can be implemented specifically through hardware, software, or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium; in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc.

[0204] It should be noted that the descriptions of the various embodiments above tend to emphasize the differences between them, while their similarities or commonalities can be referred to interchangeably. The descriptions of the above embodiments of the device, storage medium, computer program, and computer program product are similar to the descriptions of the above method embodiments and have similar beneficial effects. For technical details not disclosed in the embodiments of the device, storage medium, computer program, and computer program product of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0205] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 5 As shown, the hardware entity of the electronic device 500 includes a processor 51 and a memory 52, wherein the memory 52 stores a computer program that can run on the processor 51, and the processor 51 executes the program to implement the steps in the method of any of the above embodiments.

[0206] The memory 52 stores computer programs that can run on the processor. The memory 52 is configured to store instructions and applications that can be executed by the processor 51. It can also cache data to be processed or already processed (e.g., image data, audio data, voice communication data and video communication data) of the processor 51 and various modules in the electronic device 500. It can be implemented by flash memory or random access memory (RAM).

[0207] The processor 51 executes the program to implement the steps of the robotic arm control method described above. The processor 51 typically controls the overall operation of the electronic device 500.

[0208] This application provides a computer storage medium storing one or more programs that can be executed by one or more processors to implement the steps of the robotic arm control method as described in any of the above embodiments.

[0209] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0210] The aforementioned processor can be at least one of the following: Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), Controller, Microcontroller, and Microprocessor. It is understood that other electronic devices can also implement the functions of the aforementioned processor, and this application does not specifically limit the specific implementation.

[0211] The aforementioned computer storage media / memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM), etc.; or it can be various terminals that include one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0212] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above steps / processes do not imply a sequential order of execution; the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above embodiments of this application are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0213] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0214] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0215] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0216] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0217] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0218] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, or the part that contributes to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0219] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A control method for a robotic arm, characterized in that, The robotic arm, used in electronic devices and mounted on a vehicle, includes: Image data of the objects to be moved placed on the vehicle are acquired by the PLC; Based on the image data, the control parameters of the robotic arm are determined; The control parameters are sent to the PLC so that the PLC controls the robotic arm according to the control parameters to grab the object to be transported and transport it to the designated location.

2. The method according to claim 1, characterized in that, Determining the control parameters of the robotic arm based on the image data includes: Based on the image data, determine the relative position of the object to be transported and the end effector of the robotic arm in three-dimensional space, and the placement parameters of the object to be transported. Based on the relative position of the object to be transported and the end effector of the robotic arm in three-dimensional space and the placement parameters of the object to be transported, the control parameters of the robotic arm are determined. The placement parameters of the object to be moved include: the shape of the object to be moved and the size of the object to be moved.

3. The method according to claim 2, characterized in that, The step of determining the relative position of the object to be transported and the end effector of the robotic arm in three-dimensional space, and the placement parameters of the object to be transported, based on the image data, includes: The depth image in the image data is converted to obtain the point cloud data of the object to be moved. Based on the point cloud data of the object to be transported, the relative position of the object to be transported and the end effector of the robotic arm in three-dimensional space and the placement parameters of the object to be transported are determined.

4. The method according to claim 2, characterized in that, The step of determining the relative position of the object to be transported and the end effector of the robotic arm in three-dimensional space, and the placement parameters of the object to be transported, based on the image, includes: The image data is input into a preset AI model to obtain the relative position of the object to be transported and the end effector of the robotic arm in three-dimensional space, as well as the placement parameters of the object to be transported.

5. The method according to claim 2, characterized in that, The step of determining the control parameters of the robotic arm based on the relative position of the object to be transported and the end effector of the robotic arm in three-dimensional space and the placement parameters of the object to be transported includes: Based on the relative positions of the object to be transported and the end effector of the robotic arm in three-dimensional space, the rotation parameters, extension parameters, and movement parameters of the robotic arm are determined. Based on the placement parameters of the object to be transported, the grasping parameters of the end effector in the robotic arm are determined.

6. The method according to claim 2, characterized in that, The step of determining the grasping parameters of the end effector in the robotic arm based on the placement parameters of the object to be transported includes: Based on the placement parameters of the object to be transported and the obtained weight of the object to be transported, the gripping angle and gripping force of the end effector in the robotic arm are determined.

7. The method according to claim 6, characterized in that, The method further includes: Based on the image data, determine the type of the object to be moved; The volume of the object to be moved is determined based on the placement parameters of the object to be moved; The weight of the object to be transported is determined based on the density corresponding to the type of the object and the volume of the object.

8. The method according to claim 6, characterized in that, The method further includes: In response to the input of the option to fill in the weight of the object to be moved, the weight of the object to be moved is determined.

9. The method according to claim 5, characterized in that, The method further includes: Based on the obtained current weather information, determine the movement speed in the movement parameters of the robotic arm; The current weather information includes one or more of the following: wind force level, rainfall, snowfall, and visibility level.

10. The method according to any one of claims 2 to 9, characterized in that, The method further includes: The PLC receives data from sensors on the robotic arm. Based on the data from the sensors, the control parameters of the robotic arm are adjusted to obtain the control parameters of the robotic arm again.

11. A control device for a robotic arm, characterized in that, The device is installed in an electronic device, and the robotic arm is installed on a vehicle, including: The acquisition module is used to acquire image data of the objects to be transported placed on the vehicle via the PLC; The determining module is used to determine the control parameters of the robotic arm based on the image data; The control module is used to send the control parameters to the PLC, so that the PLC controls the robotic arm according to the control parameters to grasp the object to be transported and transport it to the designated position.

12. An electronic device, characterized in that, The method includes a memory and a processor, the memory storing a computer program executable on the processor, characterized in that the processor, when executing the program, implements the steps of the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, It stores a computer program, characterized in that, when executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 10.

Images