Binding operation robot control method, device, robot and storage medium

Abstract

The present invention discloses a binding operation robot control method, device, robot and storage medium, including: in the process of the robot performing binding operation, collecting operation images and determining the robot position through the camera on the robot; collecting binding data when binding belts through the binding data acquisition module; inputting the operation image and the binding data into a pre-trained binding operation detection model to obtain the binding operation result; when the binding operation result is abnormal, determining the heavy operation position according to the robot position and the operation image, controlling the robot to move to the heavy operation position, and performing the binding operation at the heavy operation position. The robot of this embodiment can automatically and timely identify binding abnormalities and heavy operation positions during the binding operation, and automatically re-execute the binding operation, which improves the completion and efficiency of the binding operation, improves the firmness of the insulating sheath on the overhead line, and ensures the safety of the overhead line.

Description

Technical Field

[0001] The present invention relates to the technical field of electric power installation, and in particular to a control method and device of a lashing operation robot, a robot and a storage medium. Background Art

[0002] Currently, after installing the insulating sheath on the overhead line, a binding tape needs to be applied to the insulating sheath to fix the insulating sheath on the overhead line, thereby preventing the insulating sheath from falling off.

[0003] At present, robots are mainly used to tie the insulating sheath with tape. However, the existing robots are unable to detect the completion of the tying operation in a timely manner, and are prone to overlooking abnormalities in the tying operation, resulting in the tying not being tied in place or even missing, causing the insulating sheath to fall off and the overhead line to be exposed, posing a threat to public safety. Summary of the invention

[0004] The present invention provides a binding operation robot control method, device, robot and storage medium to solve the problem that the existing robot cannot detect the binding status of the cable tie, resulting in the cable tie not being tied in place or even missing.

[0005] In a first aspect, the present invention provides a method for controlling a tying operation robot, which is applied to controlling a tying belt of the robot, wherein the robot is provided with a camera and a tying data acquisition module, comprising:

[0006] When the robot performs the tying operation, the camera on the robot is used to collect an operation image and determine the position of the robot, wherein the operation image is an image collected of the cable tie tied by the robot;

[0007] The tying data collection module is used to collect tying data when tying the tying belt;

[0008] Inputting the operation image and the lashing data into a pre-trained lashing operation detection model to obtain a lashing operation result;

[0009] When the binding operation result is abnormal, determining a heavy operation position according to the robot position and the operation image, the heavy operation position being the position of the abnormal binding operation;

[0010] The robot is controlled to move to the heavy operation position and perform tying operation at the heavy operation position.

[0011] In a second aspect, the present invention provides a tying operation robot control device, which is used to control the robot tying belt, wherein the robot is provided with a camera and a tying data acquisition module, including:

[0012] An image and position acquisition module, used to collect an operation image through a camera on the robot and determine the position of the robot during the robot's binding operation, wherein the operation image is an image collected of the insulating sheath installed by the robot;

[0013] A lashing operation detection module, used for inputting the operation image into a pre-trained lashing operation detection model to obtain a lashing operation result;

[0014] A heavy operation position determination module, used for determining a heavy operation position according to the robot position and the operation image when the binding operation result is abnormal, wherein the heavy operation position is the position of the abnormal insulating sheath;

[0015] The heavy operation module is used to control the robot to move to the heavy operation position and perform the binding operation at the heavy operation position.

[0016] In a third aspect, the present invention provides a tying operation robot, the tying operation robot comprising:

[0017] A camera, a binding mechanism, at least one processor and a memory, wherein the camera, the binding mechanism and the memory are all connected to the at least one processor, wherein:

[0018] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the binding operation robot control method described in any one of the first aspects of the present invention.

[0019] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement any of the tying operation robot control methods described in the first aspect of the present invention when executed.

[0020] The robot of the embodiment of the present invention is provided with a camera and a binding data acquisition module. When the robot performs binding operations, the camera on the robot is used to collect operation images and determine the position of the robot. The binding data acquisition module is used to collect binding data when binding the belt. The operation images and binding data are input into a pre-trained binding operation detection model to obtain the binding operation result. When the binding operation result is abnormal, the heavy operation position is determined according to the robot position and the operation image, the robot is controlled to move to the heavy operation position, and the binding operation is performed at the heavy operation position. This embodiment can predict the binding operation result through the operation image and the binding data, determine the heavy operation position when the binding operation result is abnormal, and control the robot to perform the binding operation at the heavy operation position. The robot can automatically and timely identify the binding abnormality and the heavy operation position during the binding operation, and automatically re-execute the binding operation, thereby improving the completion and efficiency of the binding operation, improving the firmness of the insulating sheath on the overhead line, and ensuring the safety of the overhead line.

[0021] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 This is a flow chart of a control method for a lashing operation robot provided in Embodiment 1 of the present invention;

[0024] Figure 2A is a flow chart of a control method of a lashing operation robot provided in Embodiment 2 of the present invention;

[0025] Figure 2B It is a schematic diagram of an insulating sheath and a cable tie;

[0026] Figure 3 It is a structural schematic diagram of a control device for a lashing operation robot provided in Embodiment 3 of the present invention;

[0027] Figure 4 It is a schematic diagram of the structure of the robot provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0029] Embodiment 1

[0030] Figure 1 This is a flow chart of a method for controlling a tying operation robot provided in the first embodiment of the present invention. This embodiment can be applied to controlling a robot walking on an overhead line to perform tying. This method can be performed by a tying operation robot control device. The tying operation robot control device can be implemented in the form of hardware and / or software. The tying operation robot control device can be configured in the robot. Figure 1 As shown, the lashing operation robot control method includes:

[0031] S101. When the robot is performing a tying operation, a camera on the robot is used to collect an operation image and determine the position of the robot. The operation image is an image collected of the cable ties tied by the robot.

[0032] The robot of this embodiment is used for binding straps. Optionally, it can be a robot for binding straps on insulating sheaths. The robot is provided with a camera, a positioning module and a binding mechanism. The camera is used to capture images of the straps tied by the robot, and the positioning module is used to obtain the position of the robot.

[0033] During the operation of the robot, the image of the tied belt can be collected by the camera to obtain the operation image, and the position of the robot can be obtained by the positioning module.

[0034] S102, collecting the tying data of the tying belt through the tying data collection module.

[0035] The robot of this embodiment is also provided with a tying data acquisition module. The tying mechanism is used for tying straps. The tying data acquisition module is used to collect data such as the tension when tying straps, the length of the pulled straps, etc. When the robot performs tying operations, the tension when tying straps and the length of the pulled straps can be collected through the tying data acquisition module.

[0036] S103: Input the operation image and the lashing data into a pre-trained lashing operation detection model to obtain a lashing operation result.

[0037] The tying operation detection model can be a pre-trained model that can identify whether the tying is in place from the operation image. After inputting the operation image and tying data, the tying operation detection model can output the probability of the tying being in place in the operation image. The tying operation detection model can be various neural networks, which can be trained in supervised, unsupervised and other ways.

[0038] S104. When the result of the lashing operation is abnormal, determine a heavy operation position according to the robot position and the operation image, and the heavy operation position is the position of the abnormal lashing operation.

[0039] The result of the tying operation includes the probability of the tying being in place. If the probability is less than the preset probability threshold, the result of the tying operation is an abnormal tying operation. The image position of the abnormal tying operation in the operation image can be converted into the position in the camera coordinate system, and then converted into the position in the world coordinate system to obtain the position of the abnormal tying operation.

[0040] S105, controlling the robot to move to a heavy operation position, and performing binding operations at the heavy operation position.

[0041] Specifically, with the heavy work position as the target position, control is performed to generate a travel instruction with the target position so that the robot moves to the heavy work position and re-performs the tying work at the heavy work position.

[0042] The robot of the embodiment of the present invention is provided with a camera and a binding data acquisition module. When the robot performs binding operations, the camera on the robot is used to collect operation images and determine the position of the robot. The binding data acquisition module is used to collect binding data when binding the belt. The operation images and binding data are input into a pre-trained binding operation detection model to obtain the binding operation result. When the binding operation result is abnormal, the heavy operation position is determined according to the robot position and the operation image, the robot is controlled to move to the heavy operation position, and the binding operation is performed at the heavy operation position. This embodiment can predict the binding operation result through the operation image and the binding data, determine the heavy operation position when the binding operation result is abnormal, and control the robot to perform the binding operation at the heavy operation position. The robot can automatically and timely identify the binding abnormality and the heavy operation position during the binding operation, and automatically re-execute the binding operation, thereby improving the completion and efficiency of the binding operation, improving the firmness of the insulating sheath on the overhead line, and ensuring the safety of the overhead line.

[0043] Embodiment 2

[0044] Figure 2A This is a flow chart of a control method for a lashing operation robot provided in the second embodiment of the present invention. The embodiment of the present invention is optimized on the basis of the above-mentioned first embodiment. Figure 2A As shown, the lashing operation robot control method includes:

[0045] S201. When the robot is performing a tying operation, a camera on the robot is used to collect an operation image and determine the position of the robot. The operation image is an image collected of the cable ties tied by the robot.

[0046] The robot of this embodiment is provided with a camera and a positioning module. During the operation of the robot, the camera can capture images of tied cable ties to obtain operation images, and the positioning module can also be used to obtain the robot position. The operation images and robot positions are associated one-to-one, so that each operation image and robot position correspond one-to-one, so that the position of abnormally tied cable ties can be determined later through the operation images and the corresponding robot positions.

[0047] S202, collecting the tension of the binding belt and the length of the binding belt pulled at the end of the binding belt through the data collection module to serve as binding data.

[0048] like Figure 2B As shown, the cable tie b is tied to the insulating sheath a to fix the insulating sheath on the line, and the cable tie b ties the insulating sheath a to the line by pulling. The data acquisition module may include a dynamometer and a displacement sensor, wherein the dynamometer is used to collect the pulling force F of the cable tie during the tying process, and the displacement sensor is used to measure the length L of the pulled cable tie.

[0049] S203: Input the operation image and the lashing data into a pre-trained lashing operation detection model to obtain a lashing operation result.

[0050] In this embodiment, when training the binding operation detection model, training images and binding data for training can be obtained first, the training images include images marked with binding probability, and the binding data include the tension during binding and the length of the pulled tie. N training images and corresponding N frames of binding data are randomly extracted and input into the binding operation detection model to obtain the binding operation result of each training image, the binding operation result includes the predicted binding probability, and the predicted tension and length. The loss rate is calculated according to the predicted and marked binding probability, tension and length, and it is determined whether the loss rate is less than a preset loss rate threshold. If so, stop training the binding operation detection model to obtain a trained binding operation detection model. If not, perform gradient descent on the parameters of the binding operation detection model according to the loss rate, and return to the step of randomly extracting N training images and corresponding N frames of binding data and inputting them into the binding operation detection model.

[0051] Among them, the loss rate can be calculated according to the following loss function:

[0052]

[0053] Among them, Good iis the predicted binding probability in the i-th image, L_Good i is the probability of the marked binding in place in the i-th image, Pull i is the predicted pulling force of the lashing operation in the i-th image, L_Pull i is the tension of the lashing operation marked in the i-th image, Len i is the predicted length of the i-th image binding operation, L_Len i is the length of the annotation of the lashing operation in the i-th image.

[0054] Through the above loss function, the binding operation model can learn to predict the probability of binding in place from the tension during binding, the length of pulling the cable tie and the image features, and through the tension and the length of the cable tie, the model can learn the physical and mechanical process during the binding process. For example, if the tension is too small or the length is too small during the binding process, it can be said that the cable tie is not tied in place. In addition, the tension when the cable tie is tied to the insulating sheath and tied to the wire for the last tightening is different, and the pulling length is also different. Therefore, the image features, tension and length are combined to train the model, so that the model can predict whether the cable tie is tied in place more accurately.

[0055] S204: When the result of the lashing operation is abnormal, determine a heavy operation position according to the robot position and the operation image, and the heavy operation position is the position of the abnormal lashing operation.

[0056] The result of the binding operation includes the probability of binding in place and the image position of the cable tie in the image. If the probability is less than the preset probability threshold, the result of the binding operation is an abnormal binding operation. The image position of the binding in the operation image can be converted into the position in the camera coordinate system, and then converted into the position in the world coordinate system to obtain the position of the abnormal binding operation. The specific coordinate system conversion can refer to the existing technology and will not be described in detail here.

[0057] S205, controlling the robot to move to the heavy operation position, and performing the tying operation at the heavy operation position.

[0058] Specifically, with the heavy work position as the target position, control is performed to generate a travel instruction with the target position so that the robot moves to the heavy work position and re-performs the tying work at the heavy work position.

[0059] S206. Count the number of times the robot performs tying operations at the heavy operation position.

[0060] After the robot re-ties the cable ties at the heavy operation position, it can collect operation images of the re-tied cable ties again and input the operation images into the model to obtain the tying operation detection results. If there is still an abnormality, the cable ties are re-tied. The number of times the robot ties the cable ties at the same heavy operation position can be counted.

[0061] S207: When the number of times is greater than a preset number threshold, stop performing the binding operation at the heavy operation position.

[0062] That is, when the number of times the belt is tied at the same heavy operation position is greater than the preset threshold, it may be that the robot has an abnormality, or the insulating sheath at the heavy operation position is installed abnormally. For example, the insulating sheath may not be installed at the heavy operation position, or the insulating sheath may not completely wrap the wire, making it impossible to tie the belt. In this case, stop performing the tying operation at the heavy operation position.

[0063] S208, generating prompt information, and sending the prompt information and the operation image to the system background.

[0064] When the number of times the strap is tied at the same heavy operation position exceeds the preset threshold, after the tying operation at the heavy operation position is stopped, a prompt message can be generated and the prompt message and the operation image can be sent to the system background. After receiving the prompt message, the monitoring personnel in the system background can view the operation image to check whether multiple re-tying operations are caused by abnormal installation of the insulating sheath. If so, reinstall the insulating sheath and then tie the strap. If not, the robot's tying mechanism may be faulty and the tying mechanism needs to be repaired before performing the tying operation.

[0065] In another embodiment, when the result of the binding operation is abnormal, the total number of abnormal binding operation abnormalities and the total number of bindings of the binding operation can be counted, and the ratio of the total number of abnormalities to the total number of bindings can be calculated. When the ratio is greater than or equal to the ratio threshold, the binding operation is stopped, and binding abnormality information is generated and sent to the system background. By calculating the ratio of the total number of abnormalities to the total number of bindings, if the ratio is too large, it means that the proportion of binding abnormalities is high, which may be due to an abnormality in the binding mechanism or the insulating sheath on a longer line is not installed in place. In this case, the binding operation is stopped, and binding abnormality information including the total number of abnormalities and the total number of bindings is generated and sent to the system background, so that the background monitoring personnel can determine the cause of the abnormality through the prompted information and eliminate the cause of the abnormality in time.

[0066] In another embodiment, the binding operation result also includes the type of abnormality, which may include missing binding and not binding in place. Then, the number of missing bindings and the total number of bindings of the missing binding belt can be counted, and the ratio of the number of missing bindings to the total number of bindings can be calculated. When the ratio is greater than or equal to the ratio threshold, the binding operation is stopped, and binding abnormality information is generated and sent to the system background. In practical applications, in order to better fix the insulating sheath, the binding belt can be tied on the insulating sheath according to the preset spacing. If the binding is missed, it may cause the insulating sheath to fall off easily. When the ratio of the number of missing bindings of the missing binding belt to the total number of bindings is greater than the ratio threshold, it is determined that the number of binding belts on the insulating sheath is too small, and there may be an abnormality in the binding mechanism. Then, the robot is promptly controlled to stop the binding operation, and the binding abnormality information is sent to the system background. The system background personnel can determine that the binding mechanism is abnormal by the number of missing bindings and the total number of bindings, and eliminate the abnormality of the binding mechanism in time.

[0067] The robot of this embodiment collects operation images through a camera, and collects the tension of the binding belt and the length of the pulled binding belt through a binding data collection module as binding data, and inputs the operation image and the binding data into a pre-trained binding operation detection model to obtain the binding operation result. When the binding operation result is abnormal, the heavy operation position is determined according to the robot position and the operation image, the robot is controlled to move to the heavy operation position, and the binding operation is performed at the heavy operation position. The robot can automatically and timely identify binding abnormalities and heavy operation positions during the binding operation, and automatically re-execute the binding operation, thereby improving the completion and efficiency of the binding operation, improving the firmness of the insulating sheath on the overhead line, and ensuring the safety of the overhead line.

[0068] Furthermore, by counting the number of times the straps are tied at the same heavy operation position, the total number of abnormal tying, the total number of missed tying, and the total number of tying, and calculating the relevant ratios to determine whether to stop the operation, and generating a prompt message from the statistical data to the system background, it can not only control the robot to stop the operation in time when the conditions for stopping the operation are met, but also timely count the relevant data and send it to the background, and the wireless background monitors the statistical data of the personnel.

[0069] Embodiment 3

[0070] Figure 3 This is a schematic diagram of the structure of a lashing operation robot control device provided in the third embodiment of the present invention. Figure 3 As shown, the tying operation robot control device is used to control the robot tying belt, and the robot is provided with a camera and a tying data acquisition module, including:

[0071] The image and position acquisition module 301 is used to collect the operation image through the camera on the robot and determine the position of the robot during the robot performs the binding operation, wherein the operation image is an image collected of the insulating sheath installed by the robot;

[0072] The tying data collection module 302 is used to collect tying data when tying the tying belt through the tying data collection module;

[0073] The lashing operation detection module 303 is used to input the operation image into a pre-trained lashing operation detection model to obtain a lashing operation result;

[0074] A heavy operation position determination module 304 is used to determine a heavy operation position according to the robot position and the operation image when the binding operation result is abnormal, wherein the heavy operation position is the position of the abnormal insulating sheath;

[0075] The heavy operation module 305 is used to control the robot to move to the heavy operation position and perform the binding operation at the heavy operation position.

[0076] Optionally, the lashing data collection module 302 includes:

[0077] The cable tie tension and length acquisition module is used to acquire the tension of the cable tie when tying the cable tie, and the length of the cable tie pulled when the cable tie is finished, through the data acquisition module.

[0078] Optionally, the lashing operation detection model is trained by the following modules:

[0079] A training data acquisition module, used to acquire training images and tying data for training, wherein the training images include images marked with tying probability, and the tying data include the pulling force during tying and the length of the pulled tying tie;

[0080] A data input module is used to randomly extract N training images and corresponding N frames of lashing data and input them into a lashing operation detection model to obtain a lashing operation result for each training image, wherein the lashing operation result includes a predicted probability of lashing in place, and a predicted tension and length;

[0081] The loss rate calculation module is used to calculate the loss rate based on the predicted and marked lashing probability, tension and length;

[0082] A loss rate determination module, used to determine whether the loss rate is less than a preset loss rate threshold;

[0083] A stop training module is used to stop training the lashing operation detection model to obtain a trained lashing operation detection model;

[0084] The parameter adjustment module is used to perform gradient descent on the parameters of the lashing operation detection model according to the loss rate and return to the data input module step.

[0085] Optionally, the loss rate calculation module includes:

[0086] The loss rate calculation unit is used to calculate the loss rate according to the following loss function:

[0087]

[0088] Among them, Good i is the predicted binding probability in the i-th image, L_Good i is the probability of the marked binding in place in the i-th image, Pull i is the predicted pulling force of the lashing operation in the i-th image, L_Pull i is the tension of the lashing operation marked in the i-th image, Len i is the predicted length of the i-th image binding operation, L_Len i is the length of the annotation of the lashing operation in the i-th image.

[0089] Optionally, it also includes:

[0090] a first counting module, used for counting the number of times the robot performs the lashing operation at the heavy operation position;

[0091] A first stopping operation module, used for stopping the binding operation at the heavy operation position when the number of times is greater than a preset number threshold;

[0092] The first prompt information generating module is used to generate prompt information and send the prompt information and the operation image to the system background.

[0093] Optionally, it also includes:

[0094] A second number counting module is used for counting the total number of abnormal lashing operations and the total number of lashing operations when the lashing operation result is abnormal;

[0095] A first ratio calculation module, used for calculating the ratio of the total number of abnormalities to the total number of bindings;

[0096] The second operation stopping module is used to stop the binding operation when the ratio is greater than or equal to the ratio threshold, and generate binding abnormality information and send it to the system background.

[0097] Optionally, the binding operation result includes missing binding belts, and further includes:

[0098] The third times counting module is used to count the times of missed tying of the tying belt and the total times of tying;

[0099] A second ratio calculation module, used for calculating the ratio of the number of missed binding times to the total number of binding times;

[0100] The third operation stopping module is used to stop the binding operation when the ratio is greater than or equal to the ratio threshold, and generate binding abnormality information and send it to the system background.

[0101] The tying operation robot control device provided in the embodiment of the present invention can execute the tying operation robot control method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0102] Embodiment 4

[0103] Figure 4 FIG. 4 is a schematic diagram showing a structure of a robot 40 that can be used to implement an embodiment of the present invention. Figure 4 As shown, the robot 40 includes a camera 46, a binding mechanism 47, at least one processor 41 and a memory, wherein the camera 46, the binding mechanism 47 and the memory are all connected to the at least one processor, wherein the memory includes a read-only memory (ROM) 42, a random access memory (RAM) 43, etc., and the memory stores a computer program that can be executed by at least one processor, and the processor 41 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 42 or the computer program loaded from the storage unit 48 to the random access memory (RAM) 43. In the RAM 43, various programs and data required for the operation of the robot 40 can also be stored. The processor 41, the ROM 42 and the RAM 43 are connected to each other through a bus 44, and an input / output (I / O) interface 45 is also connected to the bus 44.

[0104] The camera 46, the binding mechanism 47, the storage unit 48, and the communication unit 49 in the robot 40 are connected to the I / O interface 45. The storage unit 48 is, for example, a disk, an optical disk, etc. The communication unit 49 is, for example, a network card, a modem, a wireless communication transceiver, etc. The communication unit 49 allows the robot 40 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0105] The processor 41 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 41 executes the various methods and processes described above, such as the lashing operation robot control method.

[0106] In some embodiments, the tying operation robot control method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 48. In some embodiments, part or all of the computer program may be loaded and / or installed on the robot 40 via the ROM 42 and / or the communication unit 49. When the computer program is loaded into the RAM 43 and executed by the processor 41, one or more steps of the tying operation robot control method described above may be performed. Alternatively, in other embodiments, the processor 41 may be configured to execute the tying operation robot control method in any other appropriate manner (e.g., by means of firmware).

[0107] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0108] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0109] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0110] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0111] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for controlling a lashing operation robot, characterized in that: Applied to control the robot tying belt, the robot is provided with a camera and a tying data acquisition module, including: When the robot performs the tying operation, the camera on the robot is used to collect an operation image and determine the position of the robot, wherein the operation image is an image collected of the cable tie tied by the robot; The tying data collection module is used to collect tying data when tying the tying belt; Inputting the operation image and the lashing data into a pre-trained lashing operation detection model to obtain a lashing operation result; When the binding operation result is abnormal, determining a heavy operation position according to the robot position and the operation image, the heavy operation position being the position of the abnormal binding operation; Controlling the robot to move to the heavy operation position and perform tying operation at the heavy operation position; The lashing operation detection model is trained in the following way: Acquire training images and tying data for training, wherein the training images include images marked with tying probability, and the tying data include the pulling force during tying and the length of the pulled tying tie; Randomly extract N training images and corresponding N frames of lashing data and input them into the lashing operation detection model to obtain the lashing operation result of each training image, wherein the lashing operation result includes the predicted probability of lashing in place, and the predicted tension and length; Calculate the loss rate based on the predicted and marked lashing probability, tension and length; Determining whether the loss rate is less than a preset loss rate threshold; If yes, stop training the lashing operation detection model to obtain a trained lashing operation detection model; If not, gradient descent is performed on the parameters of the tying operation detection model according to the loss rate, and the process returns to the step of randomly extracting N training images and corresponding N frames of tying data and inputting them into the tying operation detection model.

2. The control method of the lashing operation robot according to claim 1, characterized in that: The collecting of the tying data of the tying belt by the tying data collecting module includes: The data acquisition module is used to collect the tension of the binding strap and the length of the binding strap pulled at the end of the binding strap.

3. The control method of the lashing operation robot according to claim 1, characterized in that: The calculation of the loss rate based on the predicted and marked lashing probability, tension and length includes: The loss rate is calculated according to the following loss function: Among them, Good i is the predicted binding probability in the i-th image, L_Good i is the probability of the marked binding in place in the i-th image, Pull i is the predicted pulling force of the lashing operation in the i-th image, L_Pull i is the tension of the lashing operation marked in the i-th image, Len i is the predicted length of the i-th image binding operation, L_Len i is the length of the annotation of the lashing operation in the i-th image.

4. The control method of the lashing operation robot according to any one of claims 1 to 3, characterized in that: After controlling the robot to move to the heavy operation position and performing the tying operation at the heavy operation position, the method further includes: Counting the number of times the robot performs lashing operations at the heavy operation position; When the number of times is greater than a preset number threshold, stopping the binding operation at the heavy operation position; Prompt information is generated, and the prompt information and the job image are sent to the system background.

5. The control method of the lashing operation robot according to any one of claims 1 to 3, characterized in that: Also includes: When the lashing operation result is abnormal, counting the total number of abnormal lashing operation results and the total number of lashing operations; Calculating the ratio of the total number of abnormalities to the total number of lashings; When the ratio is greater than or equal to the ratio threshold, the binding operation is stopped, and binding abnormality information is generated and sent to the system background.

6. The control method of the lashing operation robot according to any one of claims 1 to 2, characterized in that: The tying operation result includes missing tying belts, and after controlling the robot to move to the heavy operation position and performing the tying operation at the heavy operation position, the method further includes: Count the number of missed tying times and the total number of tying times; Calculating the ratio of the number of missed tying times to the total number of tying times; When the ratio is greater than or equal to the ratio threshold, the binding operation is stopped, and binding abnormality information is generated and sent to the system background.

7. A lashing operation robot control device, characterized in that: Applied to control the robot tying belt, the robot is provided with a camera and a tying data acquisition module, including: An image and position acquisition module, used to collect an operation image through a camera on the robot and determine the position of the robot during the robot's binding operation, wherein the operation image is an image collected of the insulating sheath installed by the robot; A tying data collection module, used to collect tying data when tying the tying belt through the tying data collection module; A lashing operation detection module, used for inputting the operation image into a pre-trained lashing operation detection model to obtain a lashing operation result; A heavy operation position determination module, used for determining a heavy operation position according to the robot position and the operation image when the binding operation result is abnormal, wherein the heavy operation position is the position of the abnormal insulating sheath; A heavy operation module, used for controlling the robot to move to the heavy operation position and perform tying operations at the heavy operation position; The lashing operation detection model is trained through the following modules: A training data acquisition module, used to acquire training images and tying data for training, wherein the training images include images marked with tying probability, and the tying data include the pulling force during tying and the length of the pulled tying tie; A data input module is used to randomly extract N training images and corresponding N frames of lashing data and input them into a lashing operation detection model to obtain a lashing operation result for each training image, wherein the lashing operation result includes a predicted probability of lashing in place, and a predicted tension and length; The loss rate calculation module is used to calculate the loss rate based on the predicted and marked lashing probability, tension and length; A loss rate determination module, used to determine whether the loss rate is less than a preset loss rate threshold; If yes, a stop training module is used to stop training the lashing operation detection model to obtain a trained lashing operation detection model; If not, the parameter adjustment module is used to perform gradient descent on the parameters of the tying operation detection model according to the loss rate, and return to the step of randomly extracting N training images and corresponding N frames of tying data and inputting them into the tying operation detection model.

8. A tying operation robot, characterized in that: The lashing operation robot comprises: A camera, a binding mechanism, at least one processor and a memory, wherein the camera, the binding mechanism and the memory are all connected to the at least one processor, wherein: The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the binding operation robot control method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the tying operation robot control method according to any one of claims 1 to 6 when executed.

Images