Automatic line tracking method, terminal device and computer readable storage medium

Abstract

The application is suitable for the technical field of intelligent control, and particularly relates to an automatic line-patrolling method, a terminal device and a computer readable storage medium. The method comprises the following steps: acquiring a first shooting image of a road in front of a trolley; performing image segmentation processing on a track line in the first shooting image to obtain a mask image; detecting first line-patrolling information of the road in front of the trolley according to the mask image; and controlling the trolley to move according to the first line-patrolling information. In the embodiment of the application, the mask image of the track line is extracted from the shooting image of the road in front of the trolley, and then the track line is detected according to the mask image of the track line. Since the mask image is not easily affected by light conditions, and background noise is filtered out in the mask image, the detection precision can be effectively improved by detecting the track line through the mask image, and the automatic navigation capability of the line-patrolling trolley is improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of intelligent control, and particularly relates to an automatic line tracking method, a terminal device and a computer readable storage medium. BACKGROUND

[0002] A line tracking trolley (also known as a tracking robot) is an intelligent trolley that automatically travels along a specific route. Line tracking (also known as tracking) is a key task in the automatic travel process of the line tracking trolley. The principle of line tracking is to detect a track line on the ground through a sensor and move along the track line.

[0003] In the current line tracking method, the color of the track line is usually compared with the background color, or a fixed template is relied on, that is, the track line in the actual scene is matched with the fixed template to detect the track line. In some complex scenes, such as large changes in light or complex background scenes, the above method shows certain limitations and cannot accurately detect the track line, thereby affecting the automatic driving of the line tracking trolley. SUMMARY

[0004] The embodiments of the present application provide an automatic line tracking method, a terminal device and a computer readable storage medium, which can effectively improve the detection accuracy of road information, thereby helping to improve the automatic line tracking ability of line tracking elimination.

[0005] In a first aspect, the embodiments of the present application provide an automatic line tracking method, comprising:

[0006] obtaining a first photographed image of a road in front of a trolley;

[0007] performing image segmentation processing on a track line in the first photographed image to obtain a mask image;

[0008] detecting first line tracking information of the road in front of the trolley according to the mask image;

[0009] controlling the trolley to move according to the first line tracking information.

[0010] In the embodiments of the present application, the mask image of the track line is extracted from the photographed image of the road in front of the trolley, and then the track line is detected according to the mask image of the track line. Since the mask image is not easily affected by the light condition, and the background noise is filtered out in the mask image, the detection accuracy can be effectively improved by detecting the track line through the mask image, which is beneficial to improve the automatic navigation ability of the line tracking trolley.

[0011] In a possible implementation manner of the first aspect, the first line tracking information comprises a type of a road intersection.

[0012] The detecting the first line tracking information of the road in front of the trolley according to the mask image comprises:

[0013] detecting whether the road intersection is included in the mask image;

[0014] if the road intersection is included in the mask image, detecting a type of the road intersection according to the mask image.

[0015] In the above manner, whether the road intersection exists is determined first, and then the type of the road intersection is detected in the case where the road intersection exists. In the case where the road intersection does not exist, the subsequent process can be reduced. In this way, algorithm redundancy can be reduced, and thus the detection efficiency can be improved.

[0016] In a possible implementation of the first aspect, the detecting the type of the road intersection according to the mask image comprises:

[0017] In the first line inspection mode, detecting a number of times of changes of pixel values in a preset range of the road intersection included in the mask image;

[0018] determining the type of the road intersection according to the number of times of changes.

[0019] In a possible implementation of the first aspect, the detecting the number of times of changes of pixel values in the preset range of the road intersection included in the mask image comprises:

[0020] obtaining a rectangular frame including the road intersection; wherein an area in the rectangular frame is in the preset range;

[0021] counting the number of times of changes of pixel values at an intersection of the rectangular frame and a track line, to obtain the number of times of changes.

[0022] In the above manner, the number of times of changes of pixel values in the preset range of the road intersection is detected, which is equivalent to detecting the number of boundaries of the track of the road intersection, so as to determine the type of the road intersection.

[0023] In a possible implementation of the first aspect, the detecting the type of the road intersection according to the mask image comprises:

[0024] In the second line inspection mode, performing region division on the mask image according to track lines in the mask image, to obtain at least one connected region;

[0025] calculating a minimum circumscribed circle of the track line in each connected region;

[0026] detecting the type of the road intersection according to a number of connected regions in the mask image and a radius of each minimum circumscribed circle.

[0027] In a possible implementation manner of the first aspect, the region division of the mask image according to the track line in the mask image comprises:

[0028] The mask image is regionally divided according to the track line in the mask image to obtain at least one candidate region;

[0029] The area of each candidate region is calculated;

[0030] The candidate region with an area less than a preset value is filtered out to obtain at least one connected region.

[0031] In the foregoing manner, the number of connected regions and the radius of the minimum circumscribed circle are combined, so that the probability of false detection can be effectively reduced, and the detection accuracy of the intersection type is effectively improved.

[0032] In a possible implementation manner of the first aspect, the first line-sweeping information comprises a line-sweeping start point and a line-sweeping end point;

[0033] The first line-sweeping information of the road in front of the small vehicle is detected from the mask image, comprising:

[0034] A first boundary point of the track line is searched in a first row of pixel points of the mask image; the first row of pixel points is located at the bottom of the mask image;

[0035] A second boundary point of the track line is searched in a second row of pixel points of the mask image; the second row of pixel points is above the first row of pixel points, and the second row of pixel points is spaced apart from the first row of pixel points by d rows of pixels;

[0036] The line-sweeping start point is calculated according to the first boundary point;

[0037] The line-sweeping end point is calculated according to the second boundary point.

[0038] In a possible implementation manner of the first aspect, the movement of the small vehicle is controlled according to the first line-sweeping information, comprising:

[0039] Second line-sweeping information is acquired; the second line-sweeping information is line-sweeping information detected from at least one frame of photographed image before the first photographed image;

[0040] If the first line-sweeping information is inconsistent with the second line-sweeping information, the movement of the small vehicle is controlled according to the second line-sweeping information.

[0041] In the foregoing manner, the influence of single-frame false detection on line-sweeping can be reduced, and the detection accuracy is effectively improved.

[0042] In a second aspect, the embodiments of the present application provide an automatic line inspection device, comprising:

[0043] An acquisition unit is configured to acquire a first image of a road in front of the trolley;

[0044] A segmentation unit is configured to perform image segmentation processing on a track line in the first image to obtain a mask image;

[0045] A detection unit is configured to detect first line inspection information of the road in front of the trolley according to the mask image;

[0046] A control unit is configured to control movement of the trolley according to the first line inspection information.

[0047] In a third aspect, the embodiments of the present application provide a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the automatic line inspection method according to any one of the first aspect.

[0048] In a fourth aspect, the embodiments of the present application provide a computer readable storage medium, which stores a computer program, and the computer program is executable by a processor to implement the automatic line inspection method according to any one of the first aspect.

[0049] In a fifth aspect, the embodiments of the present application provide a computer program product, which, when executed on a terminal device, causes the terminal device to perform the automatic line inspection method according to any one of the first aspect.

[0050] It can be understood that the beneficial effects of the second aspect to the fifth aspect can be referred to the related description of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0052] Figure 1 is a flowchart of the automatic line inspection method provided by the embodiments of the present application;

[0053] Figure 2 is a node diagram provided by the embodiments of the present application;

[0054] Figure 3 is a schematic diagram of a track line provided by the embodiments of the present application;

[0055] Figure 4 FIG. 11 is a schematic diagram of a road intersection in a single-track line inspection mode according to an embodiment of the present application;

[0056] Figure 5 FIG. 12 is a schematic diagram of a detection of a road intersection according to an embodiment of the present application;

[0057] Figure 6 FIG. 13 is a schematic diagram of a communication area according to an embodiment of the present application;

[0058] Figure 7 FIG. 14 is a schematic diagram of a line inspection starting point and a line inspection ending point according to an embodiment of the present application;

[0059] Figure 8 FIG. 15 is a schematic diagram of a line inspection process according to an embodiment of the present application;

[0060] Figure 9 FIG. 16 is a structural block diagram of an automatic line inspection device according to an embodiment of the present application;

[0061] Figure 10 FIG. 17 is a structural schematic diagram of a terminal device according to an embodiment of the present application. DETAILED DESCRIPTION

[0062] In the following description, for the purposes of explanation and not limitation, specific details are set forth, such as particular system configurations, techniques, etc., in order to provide a thorough understanding of the embodiments of the application. However, it will be apparent to those skilled in the art that the application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the application with unnecessary detail.

[0063] It should be understood that the term "comprises / comprising" when used in this specification and associated claims, indicates the presence of the stated features, integers, steps, operations, elements, and / or components but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0064] It should also be understood that the term "and / or" when used in this specification and associated claims, means any one and / or any combination of the associated listed items can be present.

[0065] As used in the description of the application and the appended claims, the term "if' can be interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting" depending on the context. Similarly, the phrase "if it is determined" or "if [a described condition or event] is detected" can be interpreted to mean "upon determining" or "in response to determining" or "upon detecting [the described condition or event]" or "in response to detecting [the described condition or event]".

[0066] In addition, in the description of the application and the appended claims, the terms "first", "second", "third", etc. are used only to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

[0067] Reference in the specification to "one embodiment" or "some embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrases "in one embodiment", "in some embodiments", "in other embodiments", "in additional embodiments", etc. in various places in the specification are not necessarily all referring to the same embodiment, although they can. They are used to indicate that, in some embodiments, the feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment of the application, unless otherwise specifically indicated.

[0068] A track following trolley (also known as a track following robot) is an intelligent trolley that automatically travels along a specific route. Track following is a key task in the automatic driving process of the track following trolley. The track following principle is to detect the track line on the ground through a sensor and move along the track line.

[0069] In current track following methods, the track line color is usually compared with the background color, or a fixed template is relied on, that is, the track line in the actual scene is matched with the fixed template to detect the track line. In some complex scenes, such as large changes in light or complex background scenes, the above methods show certain limitations and cannot accurately detect the track line, thereby affecting the automatic driving of the track following trolley.

[0070] For example, in the template matching method, the actual intersection in the scene is matched through a predefined fixed template. However, this method is sensitive to the shape, size and illumination conditions of the intersection and cannot cope with non-standard intersections, complex multi-fork intersections and the like, especially in scenes with large changes in light or complex backgrounds, the template matching method often fails.

[0071] For another example, in the edge detection method, track information is extracted through edge detection. This method is difficult to ensure the stability and accuracy of detection in the case of changes in illumination conditions (such as shadows, backlight or different ambient light), processing of track edge blur or complex background noise.

[0072] Based on this, the embodiment of the present application provides an automatic line patrol method. In the embodiment of the present application, a mask image of a track line is extracted from a photographed image of a road in front of the trolley, and then the track line is detected according to the mask image of the track line. Since the mask image is not easily affected by the light condition, and the background noise is filtered out in the mask image, the detection precision can be effectively improved by detecting the track line through the mask image, which is beneficial to improve the automatic navigation ability of the line patrol trolley.

[0073] Referring to Figure 1 is a flowchart of an automatic line patrol method provided by the embodiment of the present application. As an example but not limitation, the method can include the following steps:

[0074] S101, a first photographed image of a road in front of the trolley is acquired.

[0075] The automatic line patrol method in the embodiment of the present application can be applied to a controller (or a processor, etc.) of the trolley.

[0076] The camera is installed on the trolley body, and the photographed image of the road in front of the trolley can be acquired through the camera. Correspondingly, the controller of the trolley acquires the photographed image collected by the camera through interaction with the camera.

[0077] The camera can be photographed according to a preset period. For example, the camera can be photographed once every 1s, or once every 5s, etc. In actual application, the preset period of the camera can be determined according to the moving speed of the trolley. For example, the moving speed of the trolley is 1m / s, and the preset period of the camera can be set to 1s.

[0078] It can be understood that the preset period of the camera can be consistent with the control period of the controller. For example, the camera collects a photographed image every 1s, and correspondingly, the controller acquires a photographed image every 1s, and executes the automatic line patrol method once according to the acquired photographed image.

[0079] S102, image segmentation processing is performed on the track line in the first photographed image to obtain a mask image.

[0080] In the embodiment of the present application, the image segmentation algorithm can be used to detect and segment the track line in the first photographed image to obtain a segmented image, and then the segmented image is binarized, such as setting the pixels of the track line to 1 and the pixels of the non-track line to 0, or setting the pixels of the track line to 0 and the pixels of the non-track line to 1, to obtain the mask image.

[0081] In one embodiment, the first photographed image can be input into a trained segmentation model to output the mask image. The segmentation model can perform image segmentation processing on the input image and binarize the segmented image.

[0082] Optionally, the segmentation model can adopt a neural network model.

[0083] For example, the manner of training the segmentation model can include: collecting a large number of sample images, each sample image corresponding to a reference mask image; inputting the sample image into the segmentation model to output a predicted mask image; calculating the difference between the predicted mask image and the reference mask image to obtain a loss value of the segmentation model; if the loss value is less than a preset value, the current segmentation model is determined as the trained segmentation model; if the loss value is greater than or equal to the preset value, the parameters of the segmentation model are adjusted according to the loss value, and the segmentation model is continuously trained until the loss value of the segmentation model is less than the preset value.

[0084] In the above manner, since the segmentation model is pre-trained, the efficiency of image segmentation processing can be effectively improved, and the efficiency of automatic line inspection can be improved. In addition, the trained segmentation model has high precision, so that a more accurate segmentation result can be obtained by using the trained segmentation model, and reliable data basis is provided for subsequent automatic line inspection.

[0085] In some implementations, the segmentation model can also be trained to detect the position of the intersection. Correspondingly, when the photographed image includes a road intersection, the segmentation model can output a mask image and mark the position of the road intersection in the mask image.

[0086] It can be understood that if the segmentation model includes the function of detecting the position of the intersection, the reference position of the road intersection in the sample image needs to be marked in advance during the training process. When calculating the loss value of the segmentation model, the difference between the predicted mask image and the reference mask image needs to be calculated, and the difference between the predicted position of the road intersection output by the segmentation model and the reference position also needs to be calculated.

[0087] For example, referring to Figure 2 , a node schematic diagram provided by an embodiment of the present application is shown. As an example but not as a limitation, in the mask image shown in Figure 2 , the black part represents the track line, and the white part is the area of the non-track line. Among them, the road intersection is marked with a rectangular frame (hatched frame), that is, the node.

[0088] It should be noted that Figure 2 only for the marking example of the road intersection, in actual application, the position of the road intersection can also be marked by a point or a circular frame, and the present application does not make specific limitation on this.

[0089] S103, detecting first line inspection information of the road in front of the trolley according to the mask image.

[0090] In an embodiment, the first line inspection information can include a type of the road intersection. Correspondingly, S103 includes:

[0091] detecting whether the road intersection is included in the mask image;

[0092] if the road intersection is included in the mask image, detecting the type of the road intersection according to the mask image.

[0093] Optionally, as described in the above embodiment, the segmentation model can output the position information of the road intersection. Correspondingly, if the segmentation model outputs the position information of the road intersection, it is determined that the mask image includes the road intersection.

[0094] Optionally, if the segmentation model does not have the detection function of the road intersection, it can be determined whether the black regions in the mask image overlap; if there is overlap, it is determined that the mask image includes the road intersection.

[0095] In the above manner, it is first determined whether the road intersection exists, and then the type of the road intersection is detected in the case that the road intersection exists; in the case that the road intersection does not exist, the subsequent process can be reduced. In this way, it is helpful to reduce algorithm redundancy, thereby improving the detection efficiency.

[0096] Generally, the line inspection mode includes single-track line inspection and double-track line inspection. For example, refer to Figure 3 is a schematic diagram of a track line provided by an embodiment of the present application. As shown in Figure 3 (a), it is a mask image in a single-track line inspection mode, in which the black region in the mask image represents the track. As shown in Figure 3 (b), it is a mask image obtained in a double-track line inspection mode, in which the black region in the mask image represents the boundary of the track, and the part between the two black lines represents the track.

[0097] The intersection detection manner in the two line inspection modes is different, which will be introduced respectively.

[0098] In the single-track line inspection mode (i.e., the first line inspection mode), in an implementation manner, the detection manner of the type of the road intersection can include:

[0099] detecting the number of changes of pixel values in a preset range including the road intersection in the mask image;

[0100] determining the type of the road intersection according to the number of changes.

[0101] The type of the road intersection can include a straight line, a T-shaped intersection, a cross-shaped intersection, etc.

[0102] For example, refer to Figure 4This is a schematic diagram of an intersection under the single-track line-following mode provided in an embodiment of this application. It is intended as an example and not a limitation. Figure 4 As shown in (a) above, this is a straight intersection. Figure 4 As shown in (b) in the diagram, this is a crossroads. Figure 4 As shown in (c), this is a T-junction.

[0103] from Figure 4 As can be seen, in the single-track line-following mode, the black area in the mask image represents the track. The number of track boundaries varies for different types of road intersections. The pixel values ​​at the track boundaries in the mask image change abruptly. Based on this, in the above method, by detecting the number of times the pixel values ​​change within a preset range of the road intersection, it is equivalent to detecting the number of track boundaries of the road intersection, thereby realizing the determination of the intersection type.

[0104] Optionally, the methods for detecting the number of changes may include:

[0105] Obtain a rectangular frame including the road intersection; wherein the area within the rectangular frame is within a preset range;

[0106] The number of times the pixel value changes at the intersection of the rectangle and the track line is counted, and the number of changes is obtained.

[0107] For example, see Figure 5 This is a schematic diagram of road intersection detection provided in an embodiment of this application. It is intended as an example and not a limitation. Figure 5 As shown in (a), this is a schematic diagram of a straight intersection. The rectangle 51 includes the road intersection, and there are 4 intersection points 52 between the rectangle 51 and the track line. At each intersection point 52, a pixel change occurs, that is, the number of changes is 4.

[0108] like Figure 5 As shown in (b) in the diagram, this is a schematic diagram of a crossroads. The rectangle 51 includes the road intersection, and there are 8 intersection points 52 between the rectangle 51 and the track line. At each intersection point 52, a pixel change occurs, meaning the number of changes is 8.

[0109] like Figure 5 As shown in (c), this is a schematic diagram of a T-junction. The rectangle 51 includes the road intersection, and there are 6 intersection points 52 between the rectangle 51 and the track line. At each intersection point 52, a pixel change occurs, meaning the number of changes is 6.

[0110] It should be noted that in practical applications, circular frames, elliptical frames, or diamond frames can also be used to determine the preset range, and this application embodiment does not specifically limit this.

[0111] Here, a change in pixel value refers to either changing from 1 to 0 or changing from 0 to 1.

[0112] Optionally, the number of changes of pixel values in the horizontal direction of the image can be counted by the following formula:

[0113]

[0114] The number of changes of pixel values in the vertical direction of the image can be counted by the following formula:

[0115]

[0116] wherein the top-left vertex coordinate of the rectangular frame is (rect0x, rect0y), and the bottom-right vertex coordinate of the rectangular frame is (rext1x, rext1y). The mask() represents the pixel value of the coordinate point in the brackets. The "^" is the exclusive or operation. The 1{} represents taking the value 1 when the condition in the brackets is met.

[0117] In the second line scanning mode, in one implementation, the detection method of the intersection type can include:

[0118] According to the track lines in the mask image, the mask image is regionally divided to obtain at least one connected region;

[0119] The minimum circumscribed circle of the track lines in each connected region is calculated;

[0120] According to the number of connected regions in the mask image and the radius of each minimum circumscribed circle, the intersection type of the road intersection is detected.

[0121] It can be understood that a region G on a plane is a simply connected region if any simple closed curve is drawn in the region G, and the interior of the closed curve belongs to G.

[0122] For example, referring to Figure 6 is a schematic diagram of a connected region provided by an embodiment of the present application. As shown in Figure 6 (a), four dashed curves divide the mask image into four connected regions. As shown in Figure 6 (b), three dashed curves divide the mask image into three connected regions.

[0123] Optionally, the mask image can be input into a trained region detection model to output the connected region.

[0124] Optionally, the detection method of the connected region can include:

[0125] According to the track lines in the mask image, the mask image is regionally divided to obtain at least one candidate region;

[0126] The area of each candidate region is calculated;

[0127] The candidate region with an area less than the preset value is filtered out, to obtain at least one connected region.

[0128] In some cases, the connected region may be incorrectly divided due to false detection. By filtering out the candidate region with a small area in the above manner, the probability of false detection can be reduced, thereby helping to improve the detection accuracy.

[0129] Optionally, the function in Opencv can be used to calculate the minimum circumscribed rectangle of the track line in each connected region. The function can output the center and radius of the minimum circumscribed rectangle.

[0130] In one implementation, detecting the type of the road intersection according to the number of connected regions in the mask image and the radius of each minimum circumscribed circle can include:

[0131] If the number of connected regions is less than or equal to 2, it is determined that the type of the intersection is a straight intersection.

[0132] If the number of connected regions is greater than or equal to 4, it is determined that the type of the intersection is a cross intersection.

[0133] If the number of connected regions is 3, the type of the intersection is detected according to the radius of each minimum circumscribed circle. Specifically, if among the minimum circumscribed circles of the 3 connected regions, there is one minimum circumscribed circle whose radius differs from the radius of the other two minimum circumscribed circles by more than a preset value, it is determined that the type of the intersection is a T-shaped intersection; if the difference between the radii of the minimum circumscribed circles of the 3 connected regions is less than the preset value, it is determined that the type of the intersection is a cross intersection.

[0134] It can be understood that when 3 connected regions are detected, it can be a T-shaped intersection, or it can be that one boundary of the cross intersection does not appear in the image due to the shooting angle. Therefore, in this case, the radius of the minimum circumscribed circle needs to be further determined.

[0135] As shown in (a) of FIG. 10, it is a case of a cross intersection, in which the radii of the 4 minimum circumscribed circles usually differ little. As shown in (b) of FIG. 10, it is a case of a T-shaped intersection, in which the radii of the minimum circumscribed circles of the two connected regions on the right are small, and the radius of the minimum circumscribed circle of the connected region on the left is large. Figure 6 Figure 6

[0136] In this way, the number of connected regions and the radius of the minimum circumscribed circle are combined to effectively reduce the probability of false detection, thereby effectively improving the detection accuracy of the type of the intersection.

[0137] In another embodiment, the first line-sweeping information includes a line-sweeping starting point and a line-sweeping ending point. Correspondingly, S103 can include:

[0138] ​​searching a first boundary point of the track line in first row of pixels of the mask image; wherein the first row of pixels is located at the bottom of the mask image;

[0139] searching a second boundary point of the track line in second row of pixels of the mask image; wherein the second row of pixels is above the first row of pixels, and the second row of pixels is spaced apart from the first row of pixels by d rows of pixels;

[0140] calculating a track line starting point according to the first boundary point;

[0141] calculating a track line ending point according to the second boundary point.

[0142] Optionally, the first row of pixels can be the pixels of the lowest row of the mask image.

[0143] Exemplarily, referring to Figure 7 , a schematic diagram of the track line starting point and the track line ending point provided by the embodiment of the present application is shown. As shown in Figure 7 , the track line starting point is a point in the pixels of the lowest row of the mask image, and the track line ending point is a point in the pixels of the dth row from the bottom of the mask image.

[0144] Optionally, the midpoint of the first boundary point can be determined as the track line starting point, and the midpoint of the second boundary point can be determined as the track line ending point. Specifically, the track line starting point and the track line ending point can be calculated according to the following formula:

[0145]

[0146]

[0147] wherein lineStart is the track line starting point, lineEnd is the track line ending point, leftBottom is the left boundary point in the first row of pixels, rightBottom is the right boundary point in the first row of pixels, leftTop is the left boundary point in the second row of pixels, and rightTop is the right boundary point in the second row of pixels.

[0148] wherein the boundary point can be detected according to the jump of the pixel value.

[0149] S104, controlling the movement of the trolley according to the first track line information.

[0150] In one embodiment, S104 can include:

[0151] obtaining second track line information; wherein the second track line information is track line information detected according to at least one frame of shooting image before the first shooting image;

[0152] if the first track line information is inconsistent with the second track line information, then controlling the movement of the trolley according to the second track line information.

[0153] For example, the detection results of the last 5 frames are cached, and each time a new result comes, it is cached at the end of the queue, and the oldest result is removed. If the result of the current frame is consistent with the previous 4 frames, the detection result (track information) of the current frame is directly output; if it is not consistent, the detection result of the previous frame is output.

[0154] In the above manner, the influence of single-frame false detection on track inspection can be reduced, and the detection accuracy is effectively improved.

[0155] In the embodiment of the application, the mask image of the track line is extracted from the photographed image of the road in front of the trolley, and then the track line is detected according to the mask image of the track line. Since the mask image is not easily affected by the lighting conditions, and the background noise is filtered out in the mask image, the detection accuracy can be effectively improved by detecting the track line through the mask image, which is beneficial to improve the automatic navigation ability of the track inspection trolley.

[0156] Referring to Figure 8 , it is a schematic diagram of a track inspection process provided by the embodiment of the application. As an example but not limitation, as shown in Figure 8 , the track inspection process can include the following steps:

[0157] S801, when starting an automatic track inspection task, the controller of the trolley acquires a photographed image of a current frame.

[0158] The step of acquiring the photographed image can refer to the description in the embodiment of S101, which will not be repeated here.

[0159] Optionally, before acquiring the photographed image, track information and vehicle type information can be acquired. The track information includes single-track track inspection mode and double-track track inspection mode.

[0160] The vehicle type information can include vehicle type and corresponding preset threshold. The preset threshold can be used to fix the track inspection start point and the track inspection end point. For example, for two adjacent image frames, if the distance from the track inspection start point and the track inspection end point calculated for each image frame to the center point of the road intersection is less than the preset threshold, the track inspection start point and the track inspection end point of the previous image frame are adopted. In other words, when the track inspection start point and the track inspection end point are close to the road intersection, the track inspection start point and the track inspection end point can not need to be updated.

[0161] S802, image segmentation processing is performed according to the photographed image to obtain a mask image of the track line.

[0162] Step S802 is the same as S102, and the specific description can refer to the description in the embodiment of S102, which will not be repeated here.

[0163] S803, it is judged whether the mask in the mask image is empty.

[0164] If the mask is empty, the next frame of the photographed image is acquired. If the mask is not empty, S804 is executed.

[0165] It can be understood that the mask being empty means that the mask image does not include the pixel points used to represent the track line.

[0166] S804 is executed to determine whether there is an intersection (road intersection) in the mask image.

[0167] If there is no intersection, S807 is executed, that is, the movement of the vehicle is controlled according to the track line information of the previous frame; if there is an intersection, S805 is executed.

[0168] S805 is executed to detect the track line information.

[0169] S805 is the same as S103, and the description of the embodiment of S103 can be referred to, which will not be repeated here.

[0170] S806 is executed to determine whether the track line information is abnormal.

[0171] If the track line information is normal, S807 is executed; if the track line information is abnormal, S808 is executed.

[0172] S807 is executed to control the movement of the vehicle according to the track line information of the current frame.

[0173] S808 is executed to control the movement of the vehicle according to the track line information of the previous frame.

[0174] S806-S808 are the same as S104, and the description of the embodiment of S104 can be referred to, which will not be repeated here.

[0175] In the embodiment of the application, the mask image of the track line is extracted from the photographed image of the road in front of the vehicle, and then the track line is detected according to the mask image of the track line. Since the mask image is not easily affected by the light condition, and the background noise is filtered out in the mask image, the detection precision can be effectively improved by detecting the track line through the mask image, which is beneficial to improve the automatic navigation ability of the track line vehicle. In addition, different detection methods are used to detect the intersection type in different track line modes, which is more targeted and is helpful to improve the detection accuracy of the intersection, thereby improving the control precision of the automatic track line.

[0176] It should be understood that the sequence numbers of the steps in the above embodiments do not mean the order of execution, and the execution order of the processes should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the application.

[0177] corresponding to the automatic track line method described in the above embodiments, Figure 9Figure 1 is a structural block diagram of an automatic line tracking device provided by an embodiment of the present application. For ease of illustration, only parts related to the embodiments of the present application are shown.

[0178] With reference to Figure 9 The device 9 comprises:

[0179] An acquisition unit 91 is configured to acquire a first captured image of a road ahead of the trolley.

[0180] A segmentation unit 92 is configured to perform image segmentation processing on a track line in the first captured image to obtain a mask image.

[0181] A detection unit 93 is configured to detect first line tracking information of the road ahead of the trolley according to the mask image.

[0182] A control unit 94 is configured to control movement of the trolley according to the first line tracking information.

[0183] Optionally, the first line tracking information comprises a type of a road intersection; and the detection unit 93 is further configured to:

[0184] detect whether the mask image includes the road intersection;

[0185] if the mask image includes the road intersection, detect a type of the road intersection according to the mask image.

[0186] Optionally, the detection unit 93 is further configured to:

[0187] in a first line tracking mode, detect a number of times of changes of pixel values in a preset range of the mask image including the road intersection;

[0188] determine the type of the road intersection according to the number of times of changes.

[0189] Optionally, the detection unit 93 is further configured to:

[0190] acquire a rectangular frame including the road intersection; wherein an area in the rectangular frame is within the preset range;

[0191] count a number of times of changes of pixel values at an intersection of the rectangular frame and the track line to obtain the number of times of changes.

[0192] Optionally, the detection unit 93 is further configured to:

[0193] in a second line tracking mode, divide the mask image into at least one connected region according to the track line in the mask image;

[0194] calculate a minimum circumscribed circle of the track line in each connected region;

[0195] Detect a type of the road intersection according to a number of the connected regions in the mask image and a radius of each of the minimum circumscribed circles.

[0196] Optionally, the detection unit 93 is further configured to:

[0197] According to the track line in the mask image, the mask image is regionally divided to obtain at least one candidate region.

[0198] Calculate an area of each of the candidate regions.

[0199] Filter out the candidate regions with an area less than a preset value to obtain at least one of the connected regions.

[0200] Optionally, the first track information includes a track starting point and a track ending point; and the detection unit 93 is further configured to:

[0201] Search for a first boundary point of the track line in the first row of pixel points of the mask image; wherein the first row of pixel points is located at the bottom of the mask image.

[0202] Search for a second boundary point of the track line in the second row of pixel points of the mask image; wherein the second row of pixel points is above the first row of pixel points, and the second row of pixel points is spaced apart from the first row of pixel points by d rows of pixels.

[0203] Calculate the track starting point according to the first boundary point.

[0204] Calculate the track ending point according to the second boundary point.

[0205] Optionally, the control unit 94 is further configured to:

[0206] Obtain second track information; wherein the second track information is track information detected according to at least one frame of the captured images before the first captured image.

[0207] If the first track information is inconsistent with the second track information, control the trolley to move according to the second track information.

[0208] It should be noted that the information interaction, execution process and the like between the above-described devices / units are based on the same concept as the method embodiments of the present application, and the specific functions and the technical effects brought by the same can be referred to the method embodiments part, which will not be described here in detail.

[0209] In addition, Figure 9 The automatic track following device shown can be a software unit, a hardware unit or a software and hardware combined unit built in an existing terminal device, can be integrated into the terminal device as an independent pendant, or can exist as an independent terminal device.

[0210] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to complete all or part of the above-described functions. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit. In addition, the specific name of each functional unit and module is only for convenient distinction, and does not limit the protection scope of the present application. The specific working process of the unit and module in the system can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0211] Figure 10 is a structural schematic diagram of a terminal device provided by the embodiment of the present application. As shown in Figure 10 the terminal device 10 of the embodiment includes at least one processor 100 (only one processor is shown in the figure), a memory 101, and a computer program 102 stored in the memory 101 and executable on the at least one processor 100, wherein the processor 100 executes the computer program 102 to implement the steps in any of the above-mentioned automatic line patrol methods. Figure 10 The terminal device can be a desktop computer, a notebook computer, a palm computer, a cloud server and the like. The terminal device can include, but is not limited to, a processor and a memory. Those skilled in the art can understand that

[0212] only an example of the terminal device 10, and does not constitute a limitation on the terminal device 10, and can include more or fewer components than shown, or combine certain components, or different components, for example, can also include input and output devices, network access devices and the like. Figure 10

[0213] ​The processor 100 can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0214] The memory 101 can be an internal storage unit of the terminal device 10 in some embodiments, for example, a hard disk or a memory of the terminal device 10. The memory 101 can also be an external storage device of the terminal device 10 in other embodiments, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 101 can include both an internal storage unit and an external storage device of the terminal device 10. The memory 101 is used to store an operating system, an application program, a boot loader, data, and other programs, for example, program codes of the computer program, etc. The memory 101 can also be used to temporarily store data that has been output or will be output.

[0215] The embodiments of the present application further provide a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the steps in the above-mentioned various method embodiments.

[0216] The embodiments of the present application provide a computer program product. When the computer program product is run on a terminal device, the terminal device executes the steps in the above-mentioned various method embodiments.

[0217] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the present application can implement all or part of the processes in the above-mentioned embodiment methods through a computer program to instruct relevant hardware to complete, and the computer program can be stored in a computer readable storage medium. When the computer program is executed by a processor, the steps of each method embodiment described above can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable medium at least includes any entity or device capable of carrying the computer program code to the device / terminal equipment, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc. In some jurisdictions, according to legislation and patent practice, the computer readable medium can not be an electrical carrier signal and a telecommunication signal.

[0218] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0219] Those of ordinary skill in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Professionals can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0220] In the embodiments provided by the present application, it should be understood that the disclosed apparatus / terminal equipment and method can be implemented in other ways. For example, the above-described apparatus / terminal equipment embodiments are only schematic. For example, the division of the modules or units is only a logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection between each displayed or discussed unit can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0221] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may also be distributed to multiple network units. Part or all of the units can be selected to achieve the purpose of the embodiment scheme according to actual needs.

[0222] The above embodiments are only used to illustrate the technical solutions of the present application, but not limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. An automatic line-following method, characterized in that, include: Acquire the first image of the road in front of the car; The track lines in the first captured image are segmented to obtain a mask image; The first line-following information of the road ahead of the vehicle is detected based on the mask image; The movement of the trolley is controlled based on the first line inspection information; The first patrol information includes the intersection type of the road intersection; The first line-following information for detecting the road ahead of the vehicle based on the mask image includes: Detect whether the mask image includes a road intersection; If the mask image includes a road intersection, then the intersection type is detected based on the mask image; wherein, detecting the intersection type based on the mask image includes: In the first line-following mode, the number of times the pixel values ​​in the mask image, including the road intersection, change is detected; The intersection type of the road intersection is determined based on the number of changes.

2. The automatic line following method as described in claim 1, characterized in that, The detection of the number of times pixel values ​​change within a preset range, including the road intersection, in the mask image includes: Obtain a rectangular frame including the road intersection; wherein the area within the rectangular frame is within the preset range; The number of times the pixel value changes at the intersection of the rectangle and the track line is counted to obtain the number of changes.

3. The automatic line following method as described in claim 1, characterized in that, The step of detecting the intersection type of the road intersection based on the mask image includes: In the second line-following mode, the mask image is divided into regions based on the track lines in the mask image to obtain at least one connected region; Calculate the minimum circumcircle of the orbital lines in each of the connected regions; The intersection type of the road intersection is detected based on the number of connected regions in the mask image and the radius of each minimum circumcircle.

4. The automatic line following method as described in claim 3, characterized in that, The step of dividing the mask image into regions based on the trajectory lines in the mask image to obtain at least one connected region includes: The mask image is divided into regions based on the trajectory lines in the mask image to obtain at least one candidate region; Calculate the area of ​​each candidate region; Candidate regions with areas smaller than a preset value are filtered out to obtain at least one connected region.

5. The automatic line following method as described in claim 1, characterized in that, The first patrol information includes the patrol start point and the patrol end point; The first line-following information for detecting the road ahead of the vehicle based on the mask image includes: The first boundary point of the trajectory line is searched in the first row of pixels of the mask image; wherein the first row of pixels is located at the bottom of the mask image; The second boundary point of the track line is searched in the second row of pixels of the mask image; wherein the second row of pixels is above the first row of pixels, and the second row of pixels is spaced d rows of pixels apart from the first row of pixels; The starting point of the line patrol is calculated based on the first boundary point; The endpoint of the line patrol is calculated based on the second boundary point.

6. The automatic line following method according to any one of claims 1 to 5, characterized in that, The step of controlling the movement of the trolley based on the first line inspection information includes: Acquire second line-following information; wherein, the second line-following information is line-following information detected based on at least one frame of captured image prior to the first captured image; If the first line patrol information is inconsistent with the second line patrol information, the trolley is controlled to move according to the second line patrol information.

7. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 6.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

Images