Visual line patrol method, line patrol equipment and storage medium
Through the visual line patrol method, the final area of the line patrol is corrected and the movement direction of the line patrol equipment is adjusted, which solves the problems of high line patrol cost and poor anti-interference ability in complex environments in the prior art, and achieves a low-cost and stable line patrol effect.
Patent Information
- Application Number
- CN202411798480.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-06
- Publication Date
- 2025-04-18
AI Technical Summary
The existing line patrol solutions are costly in complex environments and cannot effectively solve the anti-interference problem of guide lines.
The visual line patrol method is adopted to obtain visual images of the line patrol equipment, identify the guide lines and special patterns, correct the final area of the line patrol line, and adjust the movement direction of the line patrol equipment according to the area.
It realizes a low-cost line patrol solution in complex environments, improving the anti-interference ability of the guide line and the stability of the line patrol.
Smart Images
Figure CN119850718A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of line patrol robots, and in particular to a visual line patrol method, a line patrol device and a storage medium. Background Art
[0002] Line-patrol navigation is a commonly used robot navigation method. Its principle is to set a navigation line on the ground, and the robot identifies the navigation line and moves along it. Traditional line-patrol navigation methods use magnetic guidance, electromagnetic induction and other guidance methods. These methods all involve the laying of magnetic strip guide rails, which require the modification of the existing environment. They are costly and easily affected by the electromagnetic field in the installation environment, which can easily lead to navigation failures.
[0003] Existing line patrol solutions are not universal and cannot meet the needs of specific occasions. For example, in complex environments, existing line patrol solutions are often costly and cannot effectively solve the anti-interference problem of the guide line. Summary of the invention
[0004] The present application provides a visual line patrol method, a line patrol device and a storage medium to solve the problem in the prior art that the existing line patrol solutions are often costly in complex environments and cannot effectively solve the technical problem of anti-interference of the guide line.
[0005] In order to solve the above technical problems, a technical solution adopted in the present application is: to provide a visual line patrol method, which is applied to a line patrol device, and the visual line patrol method includes: obtaining a visual image of the line patrol device during the line patrol process, wherein the visual pattern includes a guide line and a special pattern located on the guide line; based on the visual image, obtaining an area of the special pattern and an initial area of the guide line; based on the area of the special pattern and the initial area of the guide line, obtaining a final area of the guide line; based on the final area of the guide line, correcting the movement direction of the line patrol device.
[0006] In some specific embodiments, the special pattern is different in color and / or material from the guide wire.
[0007] In some specific embodiments, obtaining an area with a special pattern and an initial area with a guide line based on a visual image includes: performing a color gamut conversion on the visual image to obtain a converted image; identifying the color of the guide line from the converted image to obtain a binary image of the initial area including the guide line; and obtaining an area with a special pattern based on the binary image.
[0008] In some specific embodiments, obtaining the area of the special pattern and the initial area of the guide line based on the visual image includes identifying the color of the guide line from the visual image to obtain the initial area of the guide line, and identifying the color of the special pattern from the visual image to obtain the area of the special pattern.
[0009] In some specific embodiments, the final area of the guide line is obtained based on the area of the special pattern and the initial area of the guide line, including: based on the width of the guide line, pixel compensation is performed on the area of the special pattern to obtain the estimated area of the guide line; based on the estimated area of the guide line, the initial area of the guide line and the area of the special pattern, the final area of the guide line is obtained.
[0010] In some specific embodiments, the final area of the guide line is obtained based on the estimated area of the guide line, the initial area of the guide line and the area of the special pattern, including: performing a phase AND operation on the estimated area of the guide line and the initial area of the guide line to obtain the corrected area of the guide line; performing a phase OR operation on the corrected area of the guide line and the area of the special pattern to obtain the final area of the guide line.
[0011] In some specific embodiments, the movement direction of the line patrol device is corrected based on the final area of the guide line, including calculating the center of mass offset of the final area of the guide line in the image coordinate system; obtaining a correction value of the movement direction of the line patrol device according to the center of mass offset of the final area of the guide line, and correcting the movement direction of the line patrol device based on the correction value.
[0012] In some specific embodiments, the correction value of the movement direction of the line patrol device is obtained according to the centroid offset of the final area of the guide line, including calculating the horizontal component of the centroid offset in the image coordinate system, and obtaining the correction value of the movement direction of the line patrol device through PID closed-loop control, or calculating the horizontal component of the centroid offset in the image coordinate system, and calculating the correction value of the movement direction of the line patrol device through the following formula:
[0013]
[0014] Among them, angle_x is the correction value of the movement direction of the line patrol device, delta_x is the horizontal component of the center of mass offset in the image coordinate system, img.width represents the width of the image where the final area of the guide line is located, and K is a correction coefficient less than 1.
[0015] In order to solve the above technical problems, another technical solution adopted in the present application is: to provide a line patrol device, which includes: a memory, a processor and an image acquisition component connected to the processor, wherein the image acquisition component is used to obtain visual images during the line patrol process of the line patrol device, and the processor is used to execute program instructions stored in the memory to implement the visual line patrol method of any of the above embodiments.
[0016] In order to solve the above technical problems, another technical solution adopted by the present application is: providing a computer-readable storage medium on which program instructions are stored, and when the program instructions are executed by a processor, the visual line patrol method of any of the above embodiments is implemented.
[0017] Beneficial effects: Different from the prior art, the visual line patrol method provided by the present application is applied to line patrol equipment. Specifically, the visual line patrol method includes: obtaining a visual image of the line patrol equipment during the line patrol process, wherein the visual pattern includes a guide line and a special pattern located on the guide line; obtaining the area of the special pattern and the initial area of the guide line based on the visual image; obtaining the final area of the guide line based on the area of the special pattern and the initial area of the guide line; and correcting the movement direction of the line patrol equipment based on the final area of the guide line. The visual line patrol method provided by the present application has a simple execution process and low hardware cost. It ensures its uniqueness in the environment through the combination of guide lines and special patterns, can effectively adapt to various complex environments, ensure the stability of line patrol, and has strong practicality. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0019] Figure 1 This is a flow chart of an embodiment of a visual line inspection method provided by the present application;
[0020] Figure 2 yes Figure 1 A schematic flow chart of an embodiment of step S12;
[0021] Figure 3 It is a structural schematic diagram of an embodiment of a visual image collected by a line patrol device during a line patrol process;
[0022] Figure 4 is a schematic structural diagram of an embodiment of a binary image of an initial area containing a guide line obtained based on a conversion image;
[0023] Figure 5 Yes Figure 4 A schematic diagram of the structure of an image obtained by inverting the binary image shown;
[0024] Figure 6 is based on Figure 5 A schematic structural diagram of an embodiment of a region of a special pattern obtained from an image shown;
[0025] Figure 7 yes Figure 1 A schematic flow chart of an embodiment of step S13;
[0026] Figure 8is a structural schematic diagram of an embodiment of an estimated area of a guide line;
[0027] Fig. 9 is a structural schematic diagram of an embodiment of the final region of the guide wire;
[0028] Fig.10 yes Figure 1 A schematic flow chart of an embodiment of step S14;
[0029] Fig.11 It is a schematic diagram of a framework of an embodiment of a line patrol device provided by the present application;
[0030] Fig.12 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium provided by the present application. DETAILED DESCRIPTION
[0031] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. It is to be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only some structures related to the present application are shown in the accompanying drawings, rather than all structures. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.
[0032] The terms "first", "second", etc. in this application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.
[0033] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0034] At present, robots need to perform autonomous line patrol movements in some specific occasions, and line patrol algorithms are one of the important factors to further improve artificial intelligence. However, the line patrol solutions of the prior art are not universal and cannot meet the needs of specific occasions. For example, in the armed escort business, the line patrol and boarding and disembarking processes of the line patrol equipment are involved; the environment at this time is uncontrollable and complex, and the existing line patrol solutions are often costly and cannot effectively solve the anti-interference problem of the guide line. Based on this, the present application provides a visual line patrol method, a line patrol device, and a computer-readable storage medium. The visual line patrol method provided by the present application is applied to the line patrol device, and the line patrol device can be a line patrol vehicle or a line patrol robot, etc. The visual line patrol method provided by the present application is reliable and low-cost, and can solve the problem of interference in the line patrol screen caused by environmental factors. The visual line patrol method provided by the present application will be described in detail below.
[0035] See also Figure 1 As shown, Figure 1 1 is a flow chart of an embodiment of a visual line patrol method provided by the present application. Specifically, the steps of the visual line patrol method may include:
[0036] S11: Acquire a visual image of the line patrol device during the line patrol process, wherein the visual pattern includes a guide line and a special pattern located on the guide line.
[0037] In order to solve the interference problem of line inspection, the present application adds a special pattern to the guide line, that is, adds a special pattern to the guide line. There is no specific requirement for the color of the guide line, as long as there is a color difference with the ground color in the working environment.
[0038] Add a special pattern to the guide line. The special pattern only needs to have a color difference with the guide line, that is, the color of the special pattern in the visual image can be different from that of the guide line. The shape and outline of the special pattern can be set according to the needs. Optionally, the material of the guide line and the material of the special pattern can also be different. The material of the guide line can be matte or frosted to reduce the reflection caused by strong light or ambient light. Other characteristics can be selectively added according to the actual environment, such as waterproof, etc., which can be set according to the needs.
[0039] The present application ensures its uniqueness in the environment through the combination of guide lines and special patterns. The line patrol device is provided with an image acquisition component. During the line patrol process, the image acquisition component acquires the visual image during the line patrol process, wherein the visual pattern includes the guide line and the special pattern located on the guide line. For example, Figure 3 As shown, Figure 3 It is a picture of a visual image collected by the image acquisition component during the line patrol process of the current line patrol device, and a grayscale image is shown here. In other embodiments, the visual image can also be a color image.
[0040] The present application improves the particularity of the line patrol guide line to effectively solve the anti-interference problem of visual line patrol of the line patrol equipment.
[0041] S12: Obtaining a special pattern area and an initial area of a guide line based on the visual image.
[0042] After the image acquisition component of the line patrol device acquires the visual image, it can process the visual image to obtain the area of the special pattern and the initial area of the guide line. The area of the special pattern here refers to the set of pixel coordinate positions occupied by the special pattern in the image. The initial area of the guide line refers to the set of pixel coordinate positions occupied by the guide line in the image.
[0043] Alternatively, the color of the guide line can be directly identified from the visual image according to the recognition algorithm to obtain the initial area of the guide line, and the color of the special pattern can be identified from the visual image to obtain the area of the special pattern. This direct recognition method does not require image conversion, but the recognition amount is large, which affects the recognition efficiency.
[0044] In other embodiments, in order to improve recognition efficiency, such as Figure 2 As shown, Figure 2 yes Figure 1 The flowchart of an embodiment of step S12 in the embodiment of FIG. 1 is as follows. Specifically, the step of obtaining the area of the special pattern and the initial area of the guide line based on the visual image may include:
[0045] S121: Perform color gamut conversion on the visual image to obtain a converted image.
[0046] Since the visual image is the original captured image, when the environment is relatively complex, the number of contours in the original image may be tens of thousands, and the recognition takes a long time. In some embodiments, in order to improve the recognition efficiency, the visual image can be converted into a color gamut to obtain a converted image, and the interference of a large number of environmental elements can be filtered out by the converted image to quickly identify the area of the guide line and the area of the special pattern.
[0047] The converted image can be an HSV (Hue, Saturation, Value) color gamut image. Different colors have different intervals in the HSV color gamut. Based on this, a lot of interference can be removed and the target color can be identified. It is understandable that there are many types of color gamuts, and visual images can also be converted into other color gamut images. For example, visual images can also be converted into YUV (Luminance, Chrominance) color gamut images. The type of converted image can be selected according to the processing algorithm, and no specific limitation is made here.
[0048] S122: Identify the color of the guide line from the converted image to obtain a binary image of the initial area including the guide line.
[0049] After the conversion image is obtained, color recognition can be performed on the conversion image according to the range of the color of the guide line in the color gamut (such as the HSV color gamut) to obtain the initial area of the guide line.
[0050] After obtaining the initial area of the guide line, remove the environmental elements in the conversion image and output a binary image of the initial area containing the guide line. The size of the binary image is the same as that of the conversion image and the visual image. Outputting the binary image of the initial area containing the guide line and then performing image recognition on the binary image can greatly improve the efficiency of image recognition. Figure 4 As shown, Figure 4 It is the result of the color recognition of the guide line under the HSV color gamut. The image is a binary image of the initial area containing the guide line. In the binary image, the color of the guide line is displayed as white, and the display color of the special pattern in the guide line is black. For the convenience of understanding, the following is also explained based on the embodiment corresponding to the binary image. Of course, in other embodiments, it is also possible to directly identify the color of the visual pattern instead of based on the binary image. This is a scope that can be understood by those skilled in the art and will not be specifically elaborated here.
[0051] S123: Obtaining a special pattern area based on the binary image.
[0052] Based on the above binary image, the area of the special pattern can be obtained. In this embodiment, the initial area of the guide line is first determined, and then the area of the special pattern is determined by the initial area of the guide line. Since only the pattern on the guide line will be regarded as a valid special pattern, compared with directly performing color recognition on the special image by converting the image, this method can avoid interference to a greater extent.
[0053] Specifically, the binary image is inverted, that is, the pixel values of 0 are set to 255, and the pixel values of 255 are set to 0, so as to highlight the special pattern from the image. Figure 5 As shown, Figure 5 Is for Figure 4 The pattern result is negated or recognized according to the color of the special pattern. After negation, the color of the special pattern is white and the color of the guide line is black. The outline information of all white blocks in the current picture can be obtained from this picture to determine the position information of the special pattern through this white outline.
[0054] Furthermore, based on obtaining the outline information of all white blocks in the image, the outlines that are too large or too small can be removed, such as removing Figure 5The noise point at the rear end of the guide wire is obviously different in size from the special pattern. Remove the contour whose color does not match the special pattern. For example, in the visual image, the color of the special pattern is red, and the color of the area in the original image corresponding to the white block here is not red, then it is considered that the white block here does not match the color of the special pattern and can be removed.
[0055] Based on the obtained white color block contour information, all color block areas containing special patterns are obtained. Graphic matching or contour matching is performed on all color block areas to obtain special pattern areas. The determined special pattern areas can be referred to Figure 6 shown. Figure 6 The area of the special pattern shown in is formed by the starting point coordinates and the width and height of the special pattern. Since the subsequent calculation only needs to know the width and height of the special pattern, in order to simplify the calculation and improve the recognition efficiency, it is not necessary to return the real outline of the special pattern.
[0056] The above method for obtaining the area of special pattern is simple, and has high recognition accuracy and efficiency.
[0057] It is understandable that color recognition may also be performed directly on the converted image to obtain the area of the special pattern.
[0058] S13: Based on the area of the special pattern and the initial area of the guide line, a final area of the guide line is obtained.
[0059] After obtaining the area of the special pattern, the final area of the guide line can be obtained according to the area of the special pattern and the initial area of the guide line. Here, the final area of the guide line is the determined area of the guide line, which has higher accuracy than the initial area of the guide line.
[0060] Specifically, in some embodiments, Figure 7 As shown, Figure 7 yes Figure 1 The flowchart of an embodiment of step S13 in the embodiment of FIG. 1 is as follows. Specifically, based on the area of the special pattern and the initial area of the guide line, the step of obtaining the final area of the guide line may include:
[0061] S131: Based on the width of the guide line, pixel compensation is performed on the area of the special pattern to obtain an estimated area of the guide line.
[0062] The area with special patterns is compensated for pixels on the left and right. This compensation value can be adjusted according to the width of the guide line. The height is determined by the position of the farthest end of the special pattern (the height of the special pattern). No compensation is done in the height direction. Since the part far away from the image acquisition component is unreliable by default and the part close to the image acquisition component is reliable, the nearby area (the area below the special pattern) can also be included. There may or may not be lines nearby. If there are, they will be included in the subsequent calculations. If not, they can be automatically removed in the subsequent bitwise AND operation. In the above manner, the estimated area of the guide line can be finally obtained, that is, the approximate area of the guide line can be obtained. For example, if Figure 8 As shown, Figure 6 The estimated area of the guide line obtained by pixel compensation in the area of the special pattern is Figure 8 In this way, the area of the guide line is estimated according to the area of the special pattern, and the position of the guide line can be corrected. (This compensation method obtains the estimated area of the guide line based only on the width and height of the special pattern, which corresponds to the above description of "since subsequent calculations only need to know the width and height of the special pattern").
[0063] The pixel compensation method can be a simple widening, such as widening the area of the special pattern by 30 pixels on each side. The compensation value can be determined according to the size of the guide line. The compensation here is to include all the guide lines.
[0064] S132: Based on the estimated area of the guide line, the initial area of the guide line and the area of the special pattern, a final area of the guide line is obtained.
[0065] After obtaining the estimated area of the guide line, the initial area of the guide line, and the area of the special pattern on the guide line, the final area of the guide line can be obtained based on the above information.
[0066] Specifically, in some embodiments, based on the estimated area of the guide line, the initial area of the guide line, and the area of the special pattern, the step of obtaining the final area of the guide line may include: performing a phase AND operation on the estimated area of the guide line and the initial area of the guide line to obtain a revised area of the guide line. This step is to reduce noise and exclude other interferences beside the guide line. If the color block is not in the estimated area of the guide line, it can be directly excluded. For example, Figure 8 The estimated area of the guide line in Figure 4 The initial area of the guide line in the image is subjected to a phase AND operation to remove interference and remove color blocks that are not in the estimated area to obtain a corrected area of the guide line.
[0067] After the above-mentioned AND operation, the special pattern area will be missing. Since the special pattern area is also on the guide line, in order to fill in the color block of the special pattern, the corrected area of the guide line and the special pattern area are then ORed to obtain the final area of the guide line. Figure 6 The special pattern area in the phase OR operation is performed to finally obtain the final area of the guide line (such as Fig. 9 shown).
[0068] In the above embodiment, the position of the obtained guide line is adjusted through the area of the special pattern. This method can reduce the interference of environmental factors and make the final area of the obtained guide line more accurate.
[0069] It is understandable that the above-mentioned phase-and operation may not be performed to supplement the missing special pattern area. The moving direction of the line patrol device is corrected directly according to the contour information of the correction area of the guide line obtained above by taking an average method. This method can also guide the line patrol device, but the calculation process may be more complicated because it involves the boundary judgment of the contour.
[0070] S14: Based on the final area of the guide line, the moving direction of the line patrol device is corrected.
[0071] After the final area of the guide line is obtained, that is, the determined position of the guide line is obtained, the movement direction of the line patrol device can be corrected based on the final area of the guide line to control the line patrol device to move along the guide line.
[0072] In some specific embodiments, Fig.10 As shown, based on the final area of the guide line, the step of correcting the movement direction of the line patrol device may include:
[0073] S141: Calculate the centroid offset of the final area of the guide line in the image coordinate system.
[0074] After obtaining the final area of the guide line, an image coordinate system is established, and the center of mass offset of the final area of the guide line in the image coordinate system is calculated. For example, obtain the center of mass coordinates of the final area of the guide line in the image coordinate system, and the center coordinates of the image where the final area of the guide line is located in the image coordinate system, calculate the difference between the above center of mass coordinates and the center coordinates to obtain the center of mass offset of the final area of the guide line. Among them, the image coordinate system refers to the upper left corner of the image as the origin, the extension line to the right of the origin is the positive x direction, and the downward direction of the origin is the positive y direction. The size of the images displayed in each step of this application is the same, such as Figure 3-Figure 6 as well as Figure 8 and Fig. 9 The same size.
[0075] S142: Obtaining a correction value of the movement direction of the line patrol device according to the centroid offset of the final area of the guide line, and correcting the movement direction of the line patrol device based on the correction value.
[0076] After obtaining the centroid offset of the final area of the guide line, the correction value of the movement direction of the line patrol device can be calculated according to the centroid offset of the final area of the guide line, and then the movement direction of the line patrol device is corrected based on the correction value of the movement direction of the line patrol device.
[0077] The purpose of calculating the centroid offset of the final area of the guide line is to obtain the correction value of the movement direction of the line patrol device, so as to correct the guide line to the predetermined position, such as the center of the screen. The larger the centroid offset, the larger the direction correction value. For example, if the offset is greater than zero, the direction is adjusted to the right, and if it is less than zero, the direction is adjusted to the left, and finally the line patrol device is controlled to move along the guide line.
[0078] Optionally, in some embodiments, after calculating the horizontal component (x component) of the centroid offset of the final area of the guide line in the image coordinate system, a correction value of the movement direction of the line patrol device can be obtained through PID (Proportional Integral Derivative) closed-loop control. PID control is a field that can be understood by those skilled in the art and will not be described in detail here.
[0079] Optionally, in other embodiments, the correction value of the movement direction of the line patrol device can be obtained by the ratio of the horizontal component (x component) of the centroid offset of the final area of the guide line to the screen width. Specifically, after calculating the horizontal component of the centroid offset in the image coordinate system, the correction value of the movement direction of the line patrol device is calculated by the following formula:
[0080]
[0081] Among them, angle_x is the correction value of the movement direction of the line patrol device, delta_x is the horizontal component of the above-mentioned center of mass offset in the image coordinate system, img.width represents the width of the image where the final area of the guide line is located, and K is a correction coefficient less than 1.
[0082] After obtaining the correction value of the moving direction of the line patrol device, the line patrol device is controlled to move along the guide line through the speed value and the correction value of the moving direction of the line patrol device.
[0083] The line patrol method of the line patrol device provided in the present application solves the anti-interference problem in the line patrol movement of the line patrol device at a relatively low cost. The uniqueness of the line patrol device in the environment is ensured by the combination of the guide line and the special pattern. Finally, the algorithm is used for processing to correct the initial area of the guide line to obtain a more accurate final area of the guide line. Then, the movement direction of the line patrol device is controlled according to the final area of the guide line. This method can effectively adapt to various complex environments and ensure the stability of the line patrol.
[0084] The line patrol method provided by the present application has the following advantages: (1) The visual line patrol method provided by the present application is low-cost. The hardware part of the line patrol motion control only adds a guide line with a special pattern and an image acquisition component. No additional hardware is required, which controls the hardware cost to a large extent; (2) The anti-interference ability is strong. The guide line can be made of matte or frosted material, which can effectively reduce the interference of external ambient light; (3) The guide line is designed with a special pattern to ensure its uniqueness in the environment, which can effectively prevent similar interference objects from appearing in the environment and affecting the line patrol function, thereby improving the anti-interference ability of the line patrol; (4) The environmental adaptability is strong. By adding lights to the line patrol equipment and adjusting the material of the guide line, it can be used in situations where the ambient light is too strong or too weak, and there is rain or dust.
[0085] This application also provides a line patrol device, please refer to Fig.11 As shown, Fig.11 1 is a schematic diagram of a framework of an embodiment of a line patrol device provided by the present application. Specifically, the line patrol device 200 includes: an image acquisition component 21, a memory 22 and a processor 23 coupled to each other. The image acquisition component 21 is used to capture visual images during the line patrol process of the line patrol device, and the processor 23 is connected to the image acquisition component 21 and the memory 22.
[0086] The processor 23 is used to execute the program instructions stored in the memory 22 to implement the steps in the visual line patrol method of the line patrol device of any of the above embodiments. For the steps of the visual line patrol method, please refer to the introduction of any of the above embodiments, which will not be repeated here.
[0087] The line patrol device 200 of the present application can solve the problem of line patrol screen interference caused by environmental factors when patrolling the line, and the cost is low.
[0088] This application also provides a computer-readable storage medium, see Fig.12 As shown, Fig.12 The computer-readable storage medium 30 stores program data 31 that can be executed by a processor, and the program data 31 is used to implement the steps in the visual line patrol method of the line patrol device of any of the above embodiments.
[0089] The program data 31 may be stored in the computer-readable storage medium 30 in the form of a software product, including a number of instructions for enabling a device or a processor to execute all or part of the steps of the methods of various embodiments of the present application.
[0090] The computer readable storage medium 30 is a medium in a computer memory for storing a certain discontinuous physical quantity. The computer readable storage medium 30 includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program data 31 codes.
[0091] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.
Claims
1. A visual line inspection method, characterized in that: Applied to line patrol equipment, the visual line patrol method includes: Acquire a visual image of the line patrol device during the line patrol process, wherein the visual pattern includes a guide line and a special pattern located on the guide line; Obtaining the area of the special pattern and the initial area of the guide line based on the visual image; Based on the area of the special pattern and the initial area of the guide line, a final area of the guide line is obtained; Based on the final area of the guide line, the moving direction of the line patrol device is corrected.
2. The visual line inspection method according to claim 1, characterized in that: The special pattern is different in color and / or material from the guide wire.
3. The visual line inspection method according to claim 2, characterized in that: The obtaining of the area of the special pattern and the initial area of the guide line based on the visual image comprises: Performing color gamut conversion on the visual image to obtain a converted image; Identifying the color of the guide line from the converted image to obtain a binary image of an initial area including the guide line; The area of the special pattern is obtained based on the binary image.
4. The visual line inspection method according to claim 2, characterized in that: The area of the special pattern and the initial area of the guide line are obtained based on the visual image, include, The color of the guide line is identified from the visual image to obtain an initial area of the guide line, and the color of the special pattern is identified from the visual image to obtain an area of the special pattern.
5. The visual line inspection method according to claim 1, characterized in that: The final area of the guide line is obtained based on the area of the special pattern and the initial area of the guide line, include, Based on the width of the guide line, pixel compensation is performed on the area of the special pattern to obtain an estimated area of the guide line; Based on the estimated area of the guide line, the initial area of the guide line and the area of the special pattern, a final area of the guide line is obtained.
6. The visual line inspection method according to claim 5, characterized in that: The obtaining of the final area of the guide line based on the estimated area of the guide line, the initial area of the guide line and the area of the special pattern comprises: Performing an AND operation on the estimated area of the guide line and the initial area of the guide line to obtain a revised area of the guide line; The final area of the guide line is obtained by performing a phase OR operation on the corrected area of the guide line and the area of the special pattern.
7. The visual line inspection method according to claim 1, characterized in that: The correcting of the moving direction of the line patrol device based on the final area of the guide line includes: Calculating the centroid offset of the final region of the guide line in the image coordinate system; According to the centroid offset of the final area of the guide line, a correction value of the moving direction of the line patrol device is obtained, so as to correct the moving direction of the line patrol device based on the correction value.
8. The visual line inspection method according to claim 7, characterized in that: The correction value of the moving direction of the line patrol device is obtained according to the centroid offset of the final area of the guide line, including: Calculate the horizontal component of the center of mass offset in the image coordinate system, and obtain a correction value of the movement direction of the line patrol device through PID closed-loop control, or, The horizontal component of the centroid offset in the image coordinate system is calculated, and the correction value of the movement direction of the line patrol device is calculated by the following formula: Among them, the angle_x is the correction value of the movement direction of the line patrol device, the delta_x is the horizontal component of the center of mass offset in the image coordinate system, the img.width represents the width of the image where the final area of the guide line is located, and K is a correction coefficient less than 1.
9. A line patrol device, characterized in that: The line patrol device includes: a memory, a processor, and an image acquisition component connected to the processor, wherein the image acquisition component is used to obtain visual images of the line patrol device during the line patrol process, and the processor is used to execute program instructions stored in the memory to implement the visual line patrol method described in any one of claims 1-8.
10. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the visual line inspection method according to any one of claims 1 to 8 is implemented.