Control method, device and equipment of hot-line work robot and storage medium

Abstract

The invention relates to the technical field of power robots, and provides a control method, device and equipment for a hot-line work robot and a storage medium, and the method comprises the steps: obtaining a work image which at least comprises a wire; performing straight line identification on the operation image, and determining a first straight line corresponding to the electric wire; based on the first straight line, generating a wireframe for frame selection of the wires; obtaining a user selection instruction, wherein the user selection instruction at least comprises color information and a specified task; generating a control instruction according to the user selection instruction, wherein the control instruction instructs the hot-line work robot to perform a specified task; the electric wire in the image is confirmed through linear recognition, then the user selection instruction is obtained through confirmation of the user, and the control instruction is generated to control the hot-line work robot, so that the accurate control instruction is obtained through confirmation of the user while the electric wire recognition efficiency is improved, the operation safety is guaranteed, and the working accuracy is improved. Therefore, hot-line work can be completed within the shortest time, and power failure of a power grid is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of electric power robots, and in particular to a control method, device, equipment and storage medium for a live-working robot. Background Art

[0002] With the rapid development of industry, some dangerous outdoor work has been replaced by robots due to the rapid improvement of robot technology, sharp price drops, labor shortages, rising labor costs and other factors. Among them, distribution network live working robots are robots that can operate in live environments and are mainly used in power systems, oil pipelines, communication systems and other industrial occasions.

[0003] CN116442286A The present application invents a robot operation object positioning system, method, device, robot and medium, which are applied to the field of robot technology, specifically: positioning the current operation object based on the point cloud information and image information of the current operation environment; if the positioning is successful, the robot arm is controlled to operate the current operation object based on the posture information of the current operation object; if the positioning fails, the point cloud information of the current operation environment is sent to the ground station device for display, and the current operation object is positioned based on the image information of the current operation environment and at least two point cloud information of the current operation object selected by the user sent by the ground station device, and the robot arm is controlled to operate the current operation object based on the posture information of the current operation object, thereby meeting the operation requirements of the two scenarios of successful and failed autonomous point selection of the distribution network live operation robot.

[0004] CN116175572A invented a multimodal control method for an outdoor working robot, characterized in that the multimodal control method for the outdoor working robot includes: after starting the outdoor working robot, obtaining image data collected in real time by an image acquisition device as an image to be processed, and obtaining an image of a power cable; using a recognition model pre-trained based on a deep neural network to identify the image to be processed to obtain a predicted working frame, and using a softmax method to obtain a first confidence of the predicted working frame, and using the recognition model to identify the power cable image to obtain a target working frame, and using a softmax method to obtain a second confidence of the target working frame; calculating the matching degree between the predicted working frame and the target working frame based on the first confidence and the second confidence; when the matching degree is greater than or equal to a configuration threshold, automatically controlling the outdoor working robot to move to the target working frame to work; or when the matching degree is less than the configuration threshold, issuing a prompt message, the prompt message is used to prompt the use of a three-proof box to control the outdoor working robot to move to the target working frame to work.

[0005] In the prior art, live-working robots are often controlled manually or through intelligent control using artificial intelligence recognition. The former is complex to operate and requires operators to learn. The latter cannot guarantee 100% accuracy through artificial intelligence recognition, especially when there are crossed wires, recognition errors are prone to occur, which in turn affects the work tasks on the power grid. Summary of the invention

[0006] After long-term practice, it is found that the control of live-line working robots is often done manually or through intelligent control through artificial intelligence recognition; the former is complicated to operate and requires the operator to learn; the latter cannot guarantee 100% accuracy through artificial intelligence recognition, especially when there are crossovers of wires, etc., it is easy to make recognition errors, thus affecting the work tasks of the power grid. The present invention provides a control method, device, equipment and storage medium for a live-line working robot to at least solve the above technical problems existing in the prior art.

[0007] According to a first aspect of the present invention, there is provided a control method for a live-working robot, the method comprising: acquiring a working image, the working image comprising at least an electric wire; performing straight line recognition on the working image to determine a first straight line corresponding to the electric wire; based on the first straight line, generating a wireframe to select the electric wire; wherein different electric wires correspond to wireframes of different colors; acquiring a user selection instruction, the user selection instruction comprising at least color information and a designated task; and generating a control instruction according to the user selection instruction, the control instruction being used to instruct the live-working robot to perform the designated task on the electric wire within the wireframe corresponding to the color information.

[0008] In one embodiment, the performing straight line recognition on the working image to determine the first straight line corresponding to the electric wire includes: preprocessing the working image to obtain a preprocessed image corresponding to the working image; performing edge detection on the preprocessed image to obtain edge points corresponding to the preprocessed image; calculating each edge point to determine the straight line equation corresponding to the edge point; when the number of edge points corresponding to the straight line equation is greater than a specified threshold, determining the straight line corresponding to the straight line equation as the first straight line.

[0009] In one embodiment, based on the first straight line, a wireframe is generated to select the wire, including: determining a wireframe color corresponding to the number of the first straight lines; based on the position information of the first straight line, generating multiple wireframes corresponding to the first straight line position information to select the first straight line; each wireframe corresponds to a wireframe color.

[0010] In one embodiment, there are multiple first straight lines, and there are intersections between the first straight lines; correspondingly, based on the first straight lines, a wireframe is generated to select the wires, including: based on the position information of the first straight lines, determining that the slopes of the two straight lines corresponding to the two straight lines are consistent and the distance is less than a specified threshold, and determining that the two straight lines are crossed straight lines; and selecting the two straight lines with two wireframes of the same color.

[0011] In one embodiment, based on the first straight line, a wireframe is generated to select the wire to be processed, and then the method further includes: obtaining a selected image, and sending the selected image to a display device to generate display information; obtaining cursor information, and when the cursor information indicates that the user clicks on the wireframe in the display information, determining the wire corresponding to the wireframe as the wire to be processed; based on the wire to be processed, determining a processing solution corresponding to the wire to be processed; and generating a user selection instruction based on the wire to be processed and the processing solution.

[0012] In one embodiment, cursor information is obtained, and then the method further includes: when the cursor information indicates that the user has long pressed the wireframe for a specified period of time while displaying information, obtaining the cursor position so that the wireframe moves according to the cursor position; when it is determined that the wireframe has moved into the range of another wireframe, swapping the wireframe colors of the two wireframes; when it is determined that the wireframe has not moved into the range of another wireframe, resetting the wireframe.

[0013] In one embodiment, based on the wire to be processed, determining a processing scheme corresponding to the wire to be processed includes: generating optional task information corresponding to the wire to be processed based on the wire to be processed; determining a user selected task corresponding to the optional task information based on the optional task information; and determining a processing scheme corresponding to the wire to be processed based on the user selected task.

[0014] According to a second aspect of the present invention, a control device for a live-working robot is provided, the device comprising: an acquisition module for acquiring a working image, the working image comprising at least an electric wire; an identification module for performing straight line identification on the working image to determine a first straight line corresponding to the electric wire; a generation module for generating a wireframe based on the first straight line to select the electric wire; wherein different electric wires correspond to wireframes of different colors; the acquisition module is further used to acquire user selection instructions, the user selection instructions comprising at least color information and a specified task; the generation module is further used to generate a control instruction according to the user selection instruction, the control instruction being used to instruct the live-working robot to perform the specified task on the electric wire in the wireframe corresponding to the color information.

[0015] In one embodiment, the performing straight line recognition on the working image to determine the first straight line corresponding to the electric wire includes: preprocessing the working image to obtain a preprocessed image corresponding to the working image; performing edge detection on the preprocessed image to obtain edge points corresponding to the preprocessed image; calculating each edge point to determine the straight line equation corresponding to the edge point; when the number of edge points corresponding to the straight line equation is greater than a specified threshold, determining the straight line corresponding to the straight line equation as the first straight line.

[0016] In one embodiment, based on the first straight line, a wireframe is generated to select the wire, including: determining a wireframe color corresponding to the number of the first straight lines; based on the position information of the first straight line, generating multiple wireframes corresponding to the first straight line position information to select the first straight line; each wireframe corresponds to a wireframe color.

[0017] In one embodiment, based on the first straight line, a wireframe is generated to select the wire, including: there are multiple first straight lines, and there are intersections between the first straight lines; correspondingly, based on the first straight line, a wireframe is generated to select the wire, including: based on the position information of the first straight line, determining that the slopes of the two straight lines corresponding to the two straight lines are consistent and the distance is less than a specified threshold, and determining that the two straight lines are crossed lines; and selecting the two straight lines with two wireframes of the same color.

[0018] In one embodiment, based on the first straight line, a wireframe is generated to select the wire to be processed, and then the method further includes: obtaining a selected image, and sending the selected image to a display device to generate display information; obtaining cursor information, and when the cursor information indicates that the user clicks on the wireframe in the display information, determining the wire corresponding to the wireframe as the wire to be processed; based on the wire to be processed, determining a processing solution corresponding to the wire to be processed; and generating a user selection instruction based on the wire to be processed and the processing solution.

[0019] In one embodiment, cursor information is obtained, and then the method further includes: when the cursor information indicates that the user has long pressed the wireframe for a specified period of time while displaying information, obtaining the cursor position so that the wireframe moves according to the cursor position; when it is determined that the wireframe has moved into the range of another wireframe, swapping the wireframe colors of the two wireframes; when it is determined that the wireframe has not moved into the range of another wireframe, resetting the wireframe.

[0020] In one embodiment, based on the wire to be processed, determining a processing scheme corresponding to the wire to be processed includes: generating optional task information corresponding to the wire to be processed based on the wire to be processed; determining a user selected task corresponding to the optional task information based on the optional task information; and determining a processing scheme corresponding to the wire to be processed based on the user selected task.

[0021] According to a third aspect of the present invention, there is provided an electronic device, comprising:

[0022] at least one processor; and

[0023] a memory communicatively connected to the at least one processor; wherein,

[0024] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor so that the at least one processor can perform the method described in the present invention.

[0025] According to a fourth aspect of the present invention, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method of the present invention.

[0026] The control method, device, equipment and storage medium of the live working robot disclosed in the present invention obtain a working image, wherein the working image at least includes electric wires; perform straight line recognition on the working image to determine a first straight line corresponding to the electric wires; based on the first straight line, generate a wireframe to select the electric wires; wherein different electric wires correspond to wireframes of different colors; obtain user selection instructions, wherein the user selection instructions at least include color information and a specified task; generate control instructions according to the user selection instructions, wherein the control instructions are used to instruct the live working robot to perform the specified task on the electric wires in the wireframe corresponding to the color information; confirm the electric wires in the image through straight line recognition, and then obtain the user selection instructions through user confirmation to generate control instructions to control the live working robot, thereby improving the efficiency of electric wire recognition and obtaining accurate control instructions through user confirmation, ensuring operation safety, and improving work accuracy, so that the live working can be completed in the fastest time and power outages in the power grid can be avoided.

[0027] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, in which:

[0029] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0030] Figure 1 A schematic diagram of the implementation flow of the control method of the live-line working robot according to an embodiment of the present invention is shown;

[0031] Figure 2 A schematic diagram showing the structure of a control device for a live-line working robot according to an embodiment of the present invention is shown;

[0032] Figure 3 A schematic diagram of the structure of an electronic device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0033] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0034] After long-term practice, it is found that the control of live-line working robots is often done manually or through intelligent control through artificial intelligence recognition; the former is complicated to operate and requires the operator to learn; the latter cannot guarantee 100% accuracy through artificial intelligence recognition, especially when there are crossovers of wires, etc., it is easy to make recognition errors, thus affecting the work tasks of the power grid. The present invention provides a control method, device, equipment and storage medium for a live-line working robot to at least solve the above technical problems existing in the prior art.

[0035] The wires in the image are confirmed through straight line recognition, and then the user confirms to obtain the user selection instructions, so as to generate control instructions to control the live working robot, thereby improving the efficiency of wire recognition and obtaining accurate control instructions through user confirmation, ensuring operational safety and improving work accuracy, so that the live working can be completed in the fastest time and avoid power outages in the power grid.

[0036] Figure 1 FIG. 1 is a schematic diagram showing the implementation flow of the control method of the live working robot according to an embodiment of the present invention; please refer to FIG. Figure 1 ;

[0037] According to a first aspect of the present invention, a control method for a live-working robot is provided, the method comprising: step 101, acquiring a working image, the working image comprising at least an electric wire; step 102, performing straight line recognition on the working image, and determining a first straight line corresponding to the electric wire; step 103, generating a wireframe based on the first straight line to select the electric wire; wherein different electric wires correspond to wireframes of different colors; step 104, acquiring a user selection instruction, the user selection instruction comprising at least color information and a specified task; step 105, generating a control instruction according to the user selection instruction, the control instruction being used to instruct the live-working robot to perform a specified task on the electric wire in the wireframe corresponding to the color information.

[0038] In step 101 of the specific embodiment of the present invention, an operation image is obtained, and the operation image at least includes electric wires;

[0039] The working image may be an image obtained by the live working robot through its integrated camera device shooting the area where live working is required; the working image includes at least electric wires, and may also include other power grid components, such as insulators, hardware, etc.

[0040] In step 102 of the specific embodiment of the present invention, straight line recognition is performed on the operation image to determine a first straight line corresponding to the electric wire;

[0041] By performing straight line recognition on the work image, the first straight line corresponding to the wire in the work image is determined, so as to facilitate the subsequent generation of the display image and send it to the customer for confirmation. Specifically, the work image is preprocessed to obtain a preprocessed image corresponding to the work image; preprocessing refers to the preprocessing of the work image before straight line detection, and specifically, the preprocessing at least includes grayscale processing and denoising of the work image. Among them, the denoising process can be performed by Gaussian filtering or median filtering. The denoised preprocessed image is edge detected to obtain edge points corresponding to the preprocessed image; among them, the specific method of edge detection is not limited, and edge detection algorithms such as Sobel operator, Canny edge detector, etc. can be applied to highlight the edges in the image. Through edge detection, several edge points corresponding to the edge of the object can be obtained; each edge point is calculated to determine the straight line equation corresponding to the edge point; specifically, for each edge point, a straight line is set and its corresponding point in the parameter space is calculated. This straight line needs to pass through the edge point, and its slope and intercept will be used as the coordinates of the corresponding point in the parameter space. When the number of edge points corresponding to the line equation is greater than the specified threshold, the line corresponding to the line equation is determined to be the first line. That is, by accumulating the number of times each edge point appears on the line in the parameter space. When the cumulative number of edge points corresponding to a line equation reaches a preset threshold, it is determined that there is a line corresponding to this point. Among them, the specified threshold can be usually set based on experience. For example, it can be set to a certain percentage of the number of points in the image. Suppose we have an image containing 1,000 edge points, we can set the threshold specified threshold to 50, that is, only when the cumulative number of edge points corresponding to a line reaches 50 times, it is identified as a significant line.

[0042] In step 103 of the specific embodiment of the present invention, a wire frame is generated based on the first straight line to select the wire;

[0043] Among them, different wires correspond to wireframes of different colors; after determining the first straight line, the first straight line is framed by the wireframe, and different straight lines in the first straight line are framed by setting different colors to achieve color distinction of different wires, which can enable users to distinguish different wires more clearly, thereby assisting users to make accurate judgments and not easily cause users to make misjudgments due to similar colors. Specifically, determine the wireframe color corresponding to the number of first straight lines; for example, there are 7 first straight lines, corresponding to 7 wires, and seven wireframes of different colors are generated, and the wireframe size corresponds to the first straight line; based on the position information of the first straight line, the position information of the first straight line refers to the position of the first straight line in the image. Generate multiple wireframes corresponding to the position information of the first straight line, and the wireframe size can frame the first straight line. After the first straight line is framed, each wireframe can correspond to a wireframe color, so that users can quickly distinguish by color, especially when the line is more complex, and can quickly and accurately determine the target to be processed. It should be noted that after the straight line is detected, the straight line may cross the image. In this case, the two edge points on the two sides of the straight line can be used as the two ends of the wireframe. The first straight line can be offset to both sides of the straight line by a specified distance, and a wireframe can be generated in conjunction with the two edge points to select the first straight line.

[0044] Among them, by setting wireframes of different colors, when there are multiple first straight lines and there are intersections between the first straight lines, and the wire has a certain curve, two corresponding wireframes will be generated when the same straight line is cut. In order to avoid equipment recognition errors under complex wire conditions, the present invention will select the target that is most likely to be the same wire through two wireframes of the same color, so as to provide users with identification and facilitate users to make the most accurate judgment. Specifically, when a straight line is cut into two straight lines, the judgment of the cutting state, that is, when there are the same pixel points in the position information of one straight line and another straight line, the two straight lines are judged to be crossed. Or, four straight lines are formed with the same pixel points, and there are two similar slopes in the four straight lines. At this time, four straight lines are formed, based on the position information of the first straight line, the straight line slopes corresponding to the cut straight lines are calculated, and the two straight lines with the same straight line slope and a distance less than the specified threshold are judged to be crossed straight lines, that is, the same straight line; correspondingly, the other two straight lines are judged to be the same straight line; and the two straight lines are framed with two wireframes of the same color. If the four straight lines do not have the same slope, determine a straight line, sort the slopes corresponding to the remaining three straight lines, determine the straight line with the closest slope to the straight line as the same straight line, and select the two straight lines with two wireframes of the same color.

[0045] After the straight line is framed, display information can be generated and sent to the display device for the user to select and determine whether the recognition is accurate, and the specific wire to be processed is selected through the wire frame to instruct the live working robot to work. Specifically: after generating a wire frame to frame the wire, a framed image is obtained, and the framed image is sent to the display device to generate display information; the display information is used to display on the display device, and the display device may refer to a device for displaying images that is integrated or wirelessly connected to the live working robot. The display device may be an industrial control equipment display, a flat panel display, a mobile phone display, etc. The user can see the framed image through the display device and operate the framed image through the cursor device, specifically, including obtaining cursor information, which refers to the information obtained after the user manipulates the cursor device. Among them, the cursor device can be a mouse, or a touch screen corresponding to the display screen, that is, the contact information of the finger on the touch screen is obtained and determined as the cursor information. Specifically, when the cursor information indicates that the user clicks on the wire frame in the displayed information, it is determined that the user determines the wire corresponding to the wire frame as the wire to be processed; based on the wire to be processed, the processing scheme corresponding to the wire to be processed is determined; the specific determination of the processing scheme includes: generating optional task information corresponding to the wire to be processed according to the wire to be processed; for example, when one wire is selected, generating matter information corresponding to one wire; for example, the matter information may include: obstacle removal, wire stripping, and wire breaking. Further, the matter information may be generated according to the end tool corresponding to the live working robot, for example, if the end tool is a wire fixing tool, the matter information is the wire fixing operation corresponding to the wire fixing tool; if the end tool is a wire grabber, the matter information may include the wire grabbing operation corresponding to the wire grabber. There are 13 corresponding end tools in total. Based on the optional task information, the user can select the optional task information to determine the user selected task corresponding to the optional task information; according to the user selected task, the processing scheme corresponding to the wire to be processed is determined. Among them, the matter information selected by the user may be one or more, and when the matter information is multiple, the work is performed in sequence according to the user selection. For example, wire grabbing, wire cutting, and wire changing. Generate user selection instructions based on the wires to be processed and the processing plan to provide work instructions to the live-line working robot.

[0046] It should be noted that the present application can also identify the operation image through volo8 and select other components, such as insulators, hardware, etc. The corresponding user can also click on the corresponding wireframe to generate the corresponding control method.

[0047] When there are many crossovers of some wires, resulting in the wires being divided into many segments, or when the wires cross but the slopes are similar, resulting in crossover errors when recognizing straight lines, the user can long-press and drag the wireframe to achieve the swapping of wireframe colors, thereby ensuring the accuracy of recognition, making it more convenient for users to make modifications, and assisting users in accurately generating control instructions for the wires; the cursor information can also represent that the user long-presses a candidate frame, representing that the user needs to move the candidate frame. When the cursor information represents that the user long-presses the wireframe for a specified time when displaying information, the specified time can be set by the staff for the display device to determine whether it is a long press, and the specified time can be 2s; when it is determined that the user long-presses, the cursor position is obtained, and the selected wireframe is moved synchronously according to the cursor position; when it is determined that the wireframe moves to the range of another wireframe, that is, the pixel point of the wireframe enters the range of another wireframe, it is determined that the two wireframes are recognized incorrectly, and the wireframe colors of the two wireframes are swapped; wherein, if it is determined that the wireframe does not move to the range of another wireframe, it is determined that the user has made an operation error, and the wireframe is reset to avoid affecting the original image, thereby improving the user experience. Finally, user selection instructions are generated through the wires to be processed and the processing scheme.

[0048] In steps 104-105 of a specific embodiment of the present invention, a user selection instruction is obtained, and the user selection instruction includes at least color information and a specified task; a control instruction is generated according to the user selection instruction, and the control instruction is used to instruct the live working robot to perform a specified task on the wires in the wire frame corresponding to the color information.

[0049] The color information refers to the color corresponding to the wire frame selected by the user in the wire frame corresponding to the first straight line, that is, the accurate wire to be processed is determined; the designated task is the item information selected by the user in the optional task information, which is information used to instruct the live-line working robot to perform a designated operation. A control instruction is generated according to the user's selection instruction, and the control instruction is used to control the live-line working robot so that the live-line working robot operates according to the control instruction. Specifically, the control instruction is used to instruct the live-line working robot to perform a designated task on the wire in the wire frame corresponding to the color information.

[0050] Figure 2 The schematic diagram of the control device of the live working robot according to the embodiment of the present invention is shown; Figure 2 ;

[0051] According to a second aspect of the present invention, a control device for a live-working robot is provided, the device comprising: an acquisition module 201, for acquiring a working image, the working image comprising at least an electric wire; an identification module 202, for performing straight line identification on the working image, and determining a first straight line corresponding to the electric wire; a generation module 203, for generating a wireframe based on the first straight line to select the electric wire; wherein different electric wires correspond to wireframes of different colors; the acquisition module 201 is further used to acquire user selection instructions, the user selection instructions comprising at least color information and a specified task; the generation module 203 is further used to generate a control instruction according to the user selection instruction, the control instruction being used to instruct the live-working robot to perform a specified task on the electric wire in the wireframe corresponding to the color information.

[0052] In one embodiment, the recognition module 202 is also used to preprocess the job image to obtain a preprocessed image corresponding to the job image; perform edge detection on the preprocessed image to obtain edge points corresponding to the preprocessed image; calculate each edge point to determine the straight line equation corresponding to the edge point; when the number of edge points corresponding to the straight line equation is greater than a specified threshold, determine that the straight line corresponding to the straight line equation is the first straight line.

[0053] In one embodiment, the generation module 203 is also used to determine the wireframe color corresponding to the number of first straight lines; based on the position information of the first straight line, generate multiple wireframes corresponding to the first straight line position information to select the first straight line; each wireframe corresponds to a wireframe color.

[0054] In one embodiment, there are multiple first straight lines, and there are intersections between the first straight lines; the generation module 203 is also used to determine, based on the position information of the first straight lines, that the slopes of the lines corresponding to the two straight lines are consistent and the distance is less than a specified threshold, and determine that the two straight lines are crossed lines; and select the two straight lines with two wireframes of the same color.

[0055] In one embodiment, the generation module 203 is further used to obtain a framed image and send the framed image to a display device to generate display information; obtain cursor information, and when the cursor information indicates that the user clicks on a wireframe in the display information, determine the wire corresponding to the wireframe as the wire to be processed; based on the wire to be processed, determine a processing solution corresponding to the wire to be processed; and generate a user selection instruction based on the wire to be processed and the processing solution.

[0056] In one embodiment, the generation module 203 is also used to obtain the cursor position when the cursor information indicates that the user has long pressed the wireframe for a specified period of time when displaying information, so that the wireframe moves according to the cursor position; when it is determined that the wireframe has moved into the range of another wireframe, the wireframe colors of the two wireframes are swapped; when it is determined that the wireframe has not moved into the range of another wireframe, the wireframe is reset.

[0057] In one embodiment, the generation module 203 is further used to generate optional task information corresponding to the wires to be processed based on the wires to be processed; determine the user selected task corresponding to the optional task information based on the optional task information; and determine the processing solution corresponding to the wires to be processed based on the user selected task.

[0058] According to an embodiment of the present invention, the present invention also provides an electronic device and a readable storage medium.

[0059] Figure 3 A schematic block diagram of an example electronic device 300 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0060] like Figure 3 As shown, the device 300 includes a computing unit 301, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 302 or a computer program loaded from a storage unit 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the device 300 can also be stored. The computing unit 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0061] A number of components in the device 300 are connected to the I / O interface 305, including: an input unit 306, such as a keyboard, a mouse, etc.; an output unit 307, such as various types of displays, speakers, etc.; a storage unit 308, such as a disk, an optical disk, etc.; and a communication unit 309, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 309 allows the device 300 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0062] The computing unit 301 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 301 performs the various methods and processes described above, such as the control method of the live working robot. For example, in some embodiments, the control method of the live working robot may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 300 via the ROM 302 and / or the communication unit 309. When the computer program is loaded into the RAM 303 and executed by the computing unit 301, one or more steps of the control method of the live working robot described above may be executed. Alternatively, in other embodiments, the computing unit 301 may be configured to execute the control method of the live working robot in any other appropriate manner (e.g., by means of firmware).

[0063] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0064] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0065] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0066] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0067] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0068] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0069] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0070] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0071] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A control method for a live working robot, characterized in that: The method comprises: Acquire a work image, wherein the work image at least includes electric wires; Performing straight line recognition on the operation image to determine a first straight line corresponding to the electric wire; Based on the first straight line, a wireframe is generated to select the wire; wherein different wires correspond to wireframes of different colors; Acquire a user selection instruction, wherein the user selection instruction at least includes color information and a specified task; A control instruction is generated according to the user selection instruction, and the control instruction is used to instruct the live working robot to perform the designated task on the wires in the wire frame corresponding to the color information.

2. The method according to claim 1, characterized in that: The performing straight line recognition on the operation image to determine a first straight line corresponding to the electric wire includes: Preprocessing the job image to obtain a preprocessed image corresponding to the job image; Performing edge detection on the preprocessed image to obtain edge points corresponding to the preprocessed image; Calculating each edge point to determine a straight line equation corresponding to the edge point; When the number of edge points corresponding to the straight line equation is greater than a specified threshold, the straight line corresponding to the straight line equation is determined to be the first straight line.

3. The method according to claim 1, characterized in that Based on the first straight line, generating a wire frame to select the wire includes: determining a wireframe color corresponding to the first number of straight lines; Based on the position information of the first straight line, a plurality of wireframes corresponding to the position information of the first straight line are generated to select the first straight line; each of the wireframes corresponds to a wireframe color.

4. The method according to claim 3, characterized in that: There are multiple first straight lines, and there are intersections between the first straight lines; Correspondingly, based on the first straight line, a wireframe is generated to select the wire, including: Based on the position information of the first straight line, determining that the straight line slopes corresponding to the two straight lines are consistent and the distance is less than a specified threshold, and determining that the two straight lines are crossed straight lines; Two pairs of wireframes with the same color are selected by the two straight lines.

5. The method according to claim 4, characterized in that Based on the first straight line, a wireframe is generated to select the wire, and then the method further includes: Obtaining a framed image, and sending the framed image to a display device to generate display information; acquiring cursor information, and when the cursor information indicates that the user clicks on the wire frame in the displayed information, determining the wire corresponding to the wire frame as the wire to be processed; Based on the wire to be processed, determining a processing scheme corresponding to the wire to be processed; A user selection instruction is generated based on the wire to be processed and the processing plan.

6. The method according to claim 5, characterized in that Get cursor information, followed by: When the cursor information indicates that the user has long pressed the wireframe for a specified time period in the displayed information, obtaining the cursor position so that the wireframe moves according to the movement of the cursor position; When it is determined that the wireframe moves into the range of another wireframe, the wireframe colors of the two wireframes are swapped; When it is determined that the wireframe has not moved into the range of another wireframe, the wireframe is reset.

7. The method according to claim 5, characterized in that Based on the wire to be processed, determining a processing scheme corresponding to the wire to be processed includes: According to the wires to be processed, generating optional task information corresponding to the wires to be processed; Based on the optional task information, determining a user selected task corresponding to the optional task information; According to the task selected by the user, a processing solution corresponding to the wire to be processed is determined.

8. A control device for a live working robot, characterized in that: The device comprises: An acquisition module, used for acquiring an operation image, wherein the operation image at least includes electric wires; A recognition module, used for performing straight line recognition on the operation image to determine a first straight line corresponding to the electric wire; A generating module, configured to generate a wireframe based on the first straight line to select the wire; wherein different wires correspond to wireframes of different colors; The acquisition module is further used to acquire a user selection instruction, wherein the user selection instruction at least includes color information and a specified task; The generation module is also used to generate a control instruction according to the user selection instruction, and the control instruction is used to instruct the live working robot to perform the specified task on the wires in the wire frame corresponding to the color information.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-7.

Images