Method and device for determining deviation of moving trajectory

Abstract

The present invention provides a method and device for determining the offset of a moving trajectory, including: acquiring a first image and a second image, the first image and the second image being images acquired at different times by an image acquisition device installed on a target object during the movement of the target object in a preset area, the first image including a first virtual marking line, and the second image including a second virtual marking line; determining whether the moving trajectory of the target object is offset according to the first virtual marking line and the second virtual marking line; and determining the offset angle and offset distance of the target object according to the first virtual marking line and the second virtual marking line when it is determined that the moving trajectory of the target object is offset. The present invention solves the problem of trajectory offset error caused by trajectory offset during the robot's driving.

Description

Technical Field

[0001] The present invention relates to the field of communications, and in particular to a method and device for determining the offset of a moving track. Background Art

[0002] With the continuous development of science and technology, robots are becoming more and more common in daily scenes, such as service robots and security robots. Take firefighting robots as an example. Firefighting robots are special robots that replace firefighters to enter or approach dangerous environments such as fire scenes to perform tasks such as firefighting, rescue, and reconnaissance. With the continuous improvement of the requirements for the actual application of firefighting robots, intelligent functions have become an inevitable feature of firefighting robots.

[0003] When a robot travels a long distance, it often has a large driving deviation due to the lack of reference for the surrounding environment. For example, when a robot travels a long distance in a straight line, due to the lack of reference for the surrounding environment, the moving trajectory may deviate to the left or right, causing the driving trajectory to deviate from the route.

[0004] In the related art, there is currently no effective solution to the problem of trajectory deviation error caused by trajectory deviation during the robot's driving. Summary of the invention

[0005] The embodiments of the present invention provide a method and device for determining the offset of a moving trajectory, so as to at least solve the problem of trajectory offset error caused by trajectory offset during the movement of a robot in the related art.

[0006] According to one embodiment of the present invention, a method for determining an offset of a moving trajectory is provided, comprising: acquiring a first image and a second image, wherein the first image and the second image are images acquired at different times by an image acquisition device installed on a target object during the process of the target object moving in a preset area, the first image includes a first virtual marking line, and the second image includes a second virtual marking line; determining whether the moving trajectory of the target object is offset according to the first virtual marking line and the second virtual marking line; and when it is determined that the moving trajectory of the target object is offset, determining an offset angle and an offset distance of the target object according to the first virtual marking line and the second virtual marking line.

[0007] Optionally, determining whether the moving trajectory of the target object is offset based on the first virtual marking line and the second virtual marking line includes: determining a first slope of the first virtual marking line and a second slope of the second virtual marking line; and determining that the moving trajectory of the target object is offset when it is determined that the difference between the first slope and the second slope is greater than or equal to a first preset threshold.

[0008] Optionally, determining the first slope of the first virtual marking line and the second slope of the second virtual marking line includes: selecting at least two groups of pixel point sets in the virtual marking line, wherein the vertical coordinates of the pixels in each group of pixel point sets are the same; determining the midpoint coordinates of each group of pixel point sets; and determining the slope of the virtual marking line based on at least two midpoint coordinates, wherein the virtual marking line is the first virtual marking line or the second virtual marking line.

[0009] Optionally, the method also includes: determining the coordinates of a first intersection point of a first straight line and the first virtual marking line, wherein the first straight line is a straight line formed by a first marking point and a second marking point set on the target object; determining the coordinates of a second intersection point of a second straight line and the first virtual marking line, wherein the second straight line is a straight line formed by a third marking point and a fourth marking point set on the target object; and determining a first moving angle of the target object based on the coordinates of the first intersection point and the coordinates of the second intersection point.

[0010] Optionally, the method also includes: determining the coordinates of a third intersection of the first straight line and the second virtual marking line; determining the coordinates of a fourth intersection of the second straight line and the second virtual marking line; and determining a second moving angle of the target object based on the coordinates of the third intersection and the coordinates of the fourth intersection.

[0011] Optionally, determining whether the moving trajectory of the target object is offset according to the first virtual marking line and the second virtual marking line includes: determining that the moving trajectory of the target object is offset when it is determined that the difference between the first moving angle and the second moving angle is greater than or equal to a second preset threshold; determining the offset angle of the target object according to the first virtual marking line and the second virtual marking line includes: determining that the difference between the first moving angle and the second moving angle is the offset angle of the target object.

[0012] Optionally, the method also includes: when the first virtual marking line and the second virtual marking line are located on the left side of the target object, if the first moving angle is greater than the second moving angle, it is determined that the target object is offset to the left; if the first moving angle is less than the second moving angle, it is determined that the target object is offset to the right; when the first virtual marking line and the second virtual marking line are located on the right side of the target object, if the first moving angle is greater than the second moving angle, it is determined that the target object is offset to the right; if the first moving angle is less than the second moving angle, it is determined that the target object is offset to the left.

[0013] Optionally, determining whether the moving trajectory of the target object is offset according to the first virtual marking line and the second virtual marking line includes: determining a first distance between the first marking point and the first intersection, and a second distance between the first marking point and the third intersection; when it is determined that the difference between the first distance and the second distance is greater than or equal to a third preset threshold, determining that the moving trajectory of the target object is offset; determining the offset distance of the target object according to the first virtual marking line and the second virtual marking line includes: determining the difference between the first distance and the second distance as the offset distance of the target object.

[0014] Optionally, the method also includes: when the first virtual marking line and the second virtual marking line are located on the left side of the target object, if the first distance is greater than the second distance, it is determined that the target object is offset to the left; if the first distance is less than the second distance, it is determined that the target object is offset to the right; when the first virtual marking line and the second virtual marking line are located on the right side of the target object, if the first distance is greater than the second distance, it is determined that the target object is offset to the right; if the first distance is less than the second distance, it is determined that the target object is offset to the left.

[0015] Optionally, determining whether the moving trajectory of the target object is offset according to the first virtual marking line and the second virtual marking line includes: determining a third distance between the second marking point and the second intersection, and a fourth distance between the second marking point and the fourth intersection; when it is determined that the difference between the third distance and the fourth distance is greater than or equal to a fourth preset threshold, determining that the moving trajectory of the target object is offset; determining the offset distance of the target object according to the first virtual marking line and the second virtual marking line includes: determining the difference between the third distance and the fourth distance as the offset distance of the target object.

[0016] Optionally, the method also includes: when the first virtual marking line and the second virtual marking line are located on the left side of the target object, if the third distance is greater than the fourth distance, determining that the target object is offset to the left; if the third distance is less than the fourth distance, determining that the target object is offset to the right; when the first virtual marking line and the second virtual marking line are located on the right side of the target object, if the third distance is greater than the fourth distance, determining that the target object is offset to the right; if the third distance is less than the fourth distance, determining that the target object is offset to the left.

[0017] According to another embodiment of the present invention, a device for determining an offset of a moving trajectory is provided, comprising: an acquisition module, configured to acquire a first image and a second image, wherein the first image and the second image are images acquired at different times by an image acquisition device installed on the target object during the movement of the target object in a preset area, the first image includes a first virtual marking line, and the second image includes a second virtual marking line; a first determination module, configured to determine whether the moving trajectory of the target object is offset according to the first virtual marking line and the second virtual marking line; and a second determination module, configured to determine an offset angle and an offset distance of the target object according to the first virtual marking line and the second virtual marking line when it is determined that the moving trajectory of the target object is offset.

[0018] According to yet another embodiment of the present invention, a storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any one of the above method embodiments when running.

[0019] According to yet another embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0020] Through the present invention, since the first image and the second image are captured at different times by the image acquisition device installed on the target object, the first image includes a first virtual marking line, and the second image includes a second virtual marking line; it is determined whether the moving trajectory of the target object is offset according to the first virtual marking line and the second virtual marking line; when it is determined that the moving trajectory of the target object is offset, the offset angle and offset distance of the target object are determined based on the first virtual marking line and the second virtual marking line. The purpose of using the virtual marking line as a reference for judging the surrounding environment and preventing the trajectory from being offset during the robot's driving process is achieved. Therefore, the problem of trajectory offset error caused by the trajectory offset during the robot's driving process can be solved, and the effect of preventing the trajectory from being offset during the robot's driving process and reducing the trajectory offset error is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0022] Figure 1 It is a hardware structure block diagram of a mobile terminal of a method for determining a movement trajectory offset according to an embodiment of the present invention;

[0023] Figure 2 is a flow chart of determining the offset of a moving trajectory according to an embodiment of the present invention;

[0024] Figure 3 is a schematic diagram of a mobile robot according to an embodiment of the present invention;

[0025] Figure 4 is a partially enlarged schematic diagram of determining the slope of a virtual marking line by using a median slope optimization algorithm according to an optional embodiment of the present invention;

[0026] Figure 5 is a schematic diagram of image collection by a robot according to an embodiment of the present invention;

[0027] Figure 6 4 is a structural block diagram of a device for determining a movement trajectory offset according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0030] The method embodiment provided in the first embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal of a method for determining a movement trajectory offset according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal 10 may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0031] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for determining the offset of the moving trajectory in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0032] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by the communication provider of the mobile terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0033] In this embodiment, a method for determining the offset of a moving track running on the above-mentioned mobile terminal is provided. Figure 2 is a flow chart of determining the offset of a moving track according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0034] Step S202, acquiring a first image and a second image, wherein the first image and the second image are images acquired at different times by an image acquisition device installed on the target object during the movement of the target object in a preset area, the first image includes a first virtual marking line, and the second image includes a second virtual marking line;

[0035] Step S204, determining whether the moving track of the target object is offset according to the first virtual marking line and the second virtual marking line;

[0036] Step S206: When it is determined that the moving track of the target object is offset, determine the offset angle and offset distance of the target object according to the first virtual marking line and the second virtual marking line.

[0037] As an optional embodiment, the target object may be a mobile robot, such as Figure 3 The figure shows a schematic diagram of a mobile robot according to an embodiment of the present invention. An image acquisition device such as a camera is provided on the robot, and images such as a first image and a second image can be acquired by the image acquisition device during the movement of the robot, and these images can form a video. In this embodiment, the first image and the second image can be images acquired at different times during the movement of the robot in the moving area, or can be images acquired at different moving times. The moving time can be determined according to the actual situation, for example, it can be a few seconds or a few minutes, and the specific situation can be determined according to the actual situation. In this embodiment, the moving area can be a road in various scenes, for example, it can be a road. The first virtual marking line and the second virtual marking line are images obtained by photographing a reference object in the moving area, and the reference object in the moving area can be a marking line on the road. In this embodiment, image acquisition can be performed with reference objects in the moving area, and the virtual marking lines in the acquired images can be used to determine whether the moving trajectory of the robot is offset, and the angle and distance of the offset.

[0038] Through the above steps, the first image and the second image are collected at different times by the image acquisition device installed on the target object, the first image includes the first virtual marking line, and the second image includes the second virtual marking line; determine whether the moving trajectory of the target object is offset according to the first virtual marking line and the second virtual marking line; when it is determined that the moving trajectory of the target object is offset, determine the offset angle and offset distance of the target object based on the first virtual marking line and the second virtual marking line. The purpose of using the virtual marking line as a reference for judging the surrounding environment and preventing the trajectory from being offset during the robot's driving process is achieved. Therefore, the problem of trajectory offset error caused by the trajectory offset during the robot's driving process can be solved, and the effect of preventing the trajectory from being offset during the robot's driving process and reducing the trajectory offset error is achieved.

[0039] Optionally, the execution subject of the above steps may be a terminal, etc., but is not limited thereto.

[0040] Optionally, determining whether the moving trajectory of the target object is offset based on the first virtual marking line and the second virtual marking line includes: determining a first slope of the first virtual marking line and a second slope of the second virtual marking line; and determining that the moving trajectory of the target object is offset when it is determined that the difference between the first slope and the second slope is greater than or equal to a first preset threshold.

[0041] As an optional embodiment, the reference object in the moving area can be located on the left side of the mobile robot, or on the right side of the mobile robot, or on both the left and right sides of the mobile robot. Whether the moving trajectory of the robot is offset can be determined by the median slope algorithm. Taking the reference object in the moving area as an example, the image of the reference object captured at the first moment during the movement of the robot is the first virtual marking line, and the image of the reference object captured at the second moment is the second virtual marking line. The first virtual marking line and the second virtual marking line can both be left virtual marking lines or right virtual marking lines. Such as the schematic diagram of the robot collecting images. The first slope of the first virtual marking line and the second slope of the second virtual marking line can be determined by the median slope algorithm. If the difference between the first slope and the second slope is greater than a preset threshold, it is determined that the moving trajectory of the robot is offset. The preset threshold can be determined according to the actual situation, for example, it can be 0.1, 0.2, 0.3, etc. The larger the difference between the first slope and the second slope, the greater the degree of offset of the moving trajectory of the mobile robot, and vice versa.

[0042] Optionally, determining the first slope of the first virtual marking line and the second slope of the second virtual marking line includes: selecting at least two groups of pixel point sets in the virtual marking line, wherein the vertical coordinates of the pixels in each group of pixel point sets are the same; determining the midpoint coordinates of each group of pixel point sets; and determining the slope of the virtual marking line based on at least two midpoint coordinates in at least two groups of pixel point sets, wherein the virtual marking line is the first virtual marking line or the second virtual marking line.

[0043] As an optional implementation, the median slope algorithm may include the following steps:

[0044] Step 1: Analyze the coordinate information of the pixel points in the virtual marker line (the virtual marker line is the first virtual marker line or the second virtual marker line), and divide the pixel points with the same ordinate into a pixel point set according to the coordinate of the pixel point in the virtual coordinate axis of the image. The virtual marker line may include multiple groups of pixel point sets, and the ordinates of the pixels in each pixel point set are equal. These pixel point sets may be sorted in order from small to large according to the order of the ordinates, and identified by serial numbers, and the set with a larger ordinate may correspond to a larger serial number.

[0045] Step 2: In the multiple pixel point sets corresponding to the virtual marking line, take the midpoint data of each sequence in each pixel point set. If the number of coordinates in the current pixel point set is odd, take the midpoint coordinate. For example, if the current pixel point set is {(2, 5), (4, 5), (6, 5)}, then (4, 5) is the midpoint coordinate. If the number of coordinates in the current pixel point set is even, take the average of the two values ​​before and after the midpoint. For example, if the current pixel point set is {(2, 5), (4, 5), (6, 5), (8, 5)}, then the average of (4, 5) and (6, 5) (5, 5) is the midpoint coordinate.

[0046] Step 3: After obtaining each group of median data, calculate the slope between two consecutive groups of medians to obtain the slope of the virtual marker line.

[0047] In order to improve the calculation accuracy, the optimization algorithm will be run, taking the virtual marking line on the left side of the robot as an example. Based on the actual situation that the overall slope of the virtual marking line on the left side must be positive, the data with negative values ​​in the slopes between the groups calculated in the previous step will be eliminated; similarly, in the data on the right side, the data with positive slope data will be eliminated according to the actual situation.

[0048] The mean of the slopes of each group after data optimization is calculated by the following formula, where n is the number of all groups, m is the number of eliminated data, and k is the slope.

[0049]

[0050] The slope of the first virtual marker line is The slope of the second virtual marker line is

[0051] Step 3: If the difference in slope change between the first virtual marker line and the second virtual marker line is less than the threshold Δ, rerun the above program, continue to collect images, and judge the slope change of the virtual marker line in the image. The threshold can be determined according to the actual situation. For example, the value range of Δ can be 0-0.015. Figure 4 It is a partially enlarged schematic diagram of determining the slope of a virtual marking line by using a median slope optimization algorithm according to an optional embodiment of the present invention.

[0052] Optionally, the method also includes: determining the coordinates of a first intersection point of a first straight line and the first virtual marking line, wherein the first straight line is a straight line formed by a first marking point and a second marking point set on the target object; determining the coordinates of a second intersection point of a second straight line and the first virtual marking line, wherein the second straight line is a straight line formed by a third marking point and a fourth marking point set on the target object; and determining a first moving angle of the target object based on the coordinates of the first intersection point and the coordinates of the second intersection point.

[0053] As an optional implementation, Figure 3 The mobile robot shown is provided with marking points A, B, C, and D. Figure 3 Point B in the image may correspond to the first marking point, point C may correspond to the second marking point, point A may correspond to the third marking point, and point D may correspond to the fourth marking point. By identifying the preset marking points A, B, C, and D on the body of the fire-fighting robot from the image collected by the camera, it can be determined whether the driving trajectory of the robot is offset.

[0054] exist Figure 5 In the image captured by the camera shown in FIG. 1 , the line connecting point B and point C is the first straight line, and the line connecting point A and point D is the second straight line. Assume that the first virtual marking line and the second virtual marking line are the left virtual marking lines. If the intersection of the first straight line and the first virtual marking line is point H(x H ,y H ), the intersection of the second straight line and the first virtual marking line is point F(x F ,y F ) According to the formula, we can get the value based on H(x H ,y H )、F(x F ,y F ) point coordinates to obtain the angle θ between the first virtual marking line and the virtual horizontal axis i , and θ i The first moving angle of the target object.

[0055]

[0056] Optionally, the method also includes: determining the coordinates of a third intersection of the first straight line and the second virtual marking line; determining the coordinates of a fourth intersection of the second straight line and the second virtual marking line; and determining a second moving angle of the target object based on the coordinates of the third intersection and the coordinates of the fourth intersection.

[0057] As an optional implementation, assume that the intersection point of the first straight line and the second virtual marking line is H(x H' ,y H' ), the intersection of the second straight line and the first virtual marking line is F'(x F' ,y F' ), the angle θ between the second virtual marking line and the virtual horizontal axis can be obtained by calculating the coordinate values ​​of points H' and F' according to the formula i+1 , and θ i+1 As the second moving angle of the target object.

[0058]

[0059] Optionally, determining whether the moving trajectory of the target object is offset according to the first virtual marking line and the second virtual marking line includes: determining that the moving trajectory of the target object is offset when it is determined that the difference between the first moving angle and the second moving angle is greater than or equal to a second preset threshold; determining the offset angle of the target object according to the first virtual marking line and the second virtual marking line includes: determining that the difference between the first moving angle and the second moving angle is the offset angle of the target object.

[0060] As an optional implementation, a threshold can be set and the i With θ i+1 The difference between θ and θ determines whether the robot deviates from the driving route. The threshold value can be determined according to the actual situation, for example, it can be 5 degrees, 10 degrees, etc. The specific value can be set arbitrarily. i With θ i+1 The difference is determined as the offset angle of the robot's moving trajectory.

[0061] Optionally, the method also includes: when the first virtual marking line and the second virtual marking line are located on the left side of the target object, if the first moving angle is greater than the second moving angle, it is determined that the target object is offset to the left; if the first moving angle is less than the second moving angle, it is determined that the target object is offset to the right; when the first virtual marking line and the second virtual marking line are located on the right side of the target object, if the first moving angle is greater than the second moving angle, it is determined that the target object is offset to the right; if the first moving angle is less than the second moving angle, it is determined that the target object is offset to the left.

[0062] As an optional implementation, according to θ i With θ i+1 The size of determines the offset direction of the robot's moving trajectory. Specifically, if the marking line located on the left side of the robot is used as a reference. If the first moving angle in the image captured at the first moment is greater than the second moving angle obtained by capturing at the second moment, it is determined that the robot is offset to the left, otherwise it is determined that the robot is offset to the right. If the marking line located on the right side of the robot is used as a reference. If the first moving angle in the image captured at the first moment is greater than the second moving angle obtained by capturing at the second moment, it is determined that the robot is offset to the right, otherwise it is determined that the robot is offset to the left, wherein the first moment is the previous moment of the second moment.

[0063] Optionally, determining whether the moving trajectory of the target object is offset according to the first virtual marking line and the second virtual marking line includes: determining a first distance between the first marking point and the first intersection, and a second distance between the first marking point and the third intersection; when it is determined that the difference between the first distance and the second distance is greater than or equal to a third preset threshold, determining that the moving trajectory of the target object is offset; determining the offset distance of the target object according to the first virtual marking line and the second virtual marking line includes: determining the difference between the first distance and the second distance as the offset distance of the target object.

[0064] As an optional implementation, the first image is an image taken before the second image. Assume that the distance between H and B is α i , the distance between H' and B is α i+1 , can be obtained by α i With α i+1 If α i With α i+1 The difference between the values ​​of α and α is greater than a third preset threshold, and it is determined that the movement trajectory of the robot has shifted. The value of the third preset threshold can be determined according to the actual situation, for example, it can be 10 cm, 20 cm, etc. The deviation value of the movement trajectory of the robot can be determined as α i With αi+1 The difference.

[0065] Optionally, the method also includes: when the first virtual marking line and the second virtual marking line are located on the left side of the target object, if the first distance is greater than the second distance, it is determined that the target object is offset to the left; if the first distance is less than the second distance, it is determined that the target object is offset to the right; when the first virtual marking line and the second virtual marking line are located on the right side of the target object, if the first distance is greater than the second distance, it is determined that the target object is offset to the right; if the first distance is less than the second distance, it is determined that the target object is offset to the left.

[0066] As an optional implementation, according to α i With α i+1 The size between determines the moving direction of the robot. Specifically, if the virtual marking line is on the left side of the robot, α i Greater than α i+1 , the robot shifts to the left, otherwise it shifts to the right. If the virtual marker line is on the right side of the robot, α i Greater than α i+1 , the robot shifts to the right, otherwise it shifts to the left.

[0067] Optionally, determining whether the moving trajectory of the target object is offset according to the first virtual marking line and the second virtual marking line includes: determining a third distance between the second marking point and the second intersection, and a fourth distance between the second marking point and the fourth intersection; when it is determined that the difference between the third distance and the fourth distance is greater than or equal to a fourth preset threshold, determining that the moving trajectory of the target object is offset; determining the offset distance of the target object according to the first virtual marking line and the second virtual marking line includes: determining the difference between the third distance and the fourth distance as the offset distance of the target object.

[0068] As an optional implementation, the distance between F and A is β i , the distance between F' and A is β i+1 , can be obtained through β i With β i+1 The distance between them determines whether the robot's moving trajectory is offset. If β i With β i+1 If the difference between the values ​​of β and β is greater than a fourth preset threshold, it is determined that the movement trajectory of the robot is offset. The value of the fourth preset threshold can be determined according to the actual situation, for example, it can be 15 cm, 25 cm, etc. The offset value of the movement trajectory of the robot can be β i With β i+1 The difference.

[0069] Optionally, the method also includes: when the first virtual marking line and the second virtual marking line are located on the left side of the target object, if the third distance is greater than the fourth distance, determining that the target object is offset to the left; if the third distance is less than the fourth distance, determining that the target object is offset to the right; when the first virtual marking line and the second virtual marking line are located on the right side of the target object, if the third distance is greater than the fourth distance, determining that the target object is offset to the right; if the third distance is less than the fourth distance, determining that the target object is offset to the left.

[0070] According to β i With β i+1 Specifically, if the virtual marking line is on the left side of the robot, β i Greater than β i+1 The robot deviates to the left, and vice versa. If the virtual marker line is on the right side of the robot, β i Greater than β i+1 The robot deviates to the right, otherwise it deviates to the left.

[0071] As an optional implementation, the present application is described below through a specific example:

[0072] When the robot enters automatic straight-line driving, the binocular camera is first reset, the camera position is adjusted to be horizontal and parallel to the ground, and the lens direction is coaxial with the vehicle body's travel direction; the firefighting robot mainboard controls the driver to drive the motor to start moving forward automatically, and at the same time begins to collect environmental information during the driving process through the camera. In this embodiment, three trajectory deviation judgments are included, and the accuracy of the trajectory deviation can be guaranteed through three judgments, including:

[0073] 1. Use the median slope algorithm to make the first trajectory deviation judgment:

[0074] Analyze the image information collected by the camera, identify the road virtual marking line information, and determine the road virtual marking line as the left virtual marking line and the right virtual marking line according to the differences in area, continuity, etc. The identified road virtual marking lines can be two areas with the same or similar colors and continuous distribution, which contain a limited number of pixels. Track deviation judgment can be performed only by the left virtual marking line or the right virtual marking line, or by using both the left virtual marking line and the right virtual marking line at the same time.

[0075] Process the coordinate point information corresponding to the pixel points of the virtual marking line on the left. According to the coordinates of the pixel points in the virtual coordinate axis of the image, the pixels with the same vertical coordinate are first grouped into one set, and these sets are numbered from 1 to n in order from bottom to top; the coordinate information points on the right are also grouped and processed in sequence;

[0076] In each set of coordinate point data points, take the midpoint data of each set of data series. If the number of current set data is an odd number, take the midpoint value. If the number of current set data is an even number, take the average of the two values ​​before and after the midpoint.

[0077] After obtaining the median data of each group, calculate the slope between two consecutive medians. In order to improve the calculation accuracy, the optimization algorithm will be run. Taking the virtual marking line on the left side of the road as an example, based on the actual situation that the overall slope of the virtual marking line on the left side must be positive, the data with negative values ​​in the slopes between the groups calculated in the previous step will be eliminated; similarly, in the data on the right side, according to the actual situation, the data with positive slope data will be eliminated;

[0078] The mean of the slopes of each group after data optimization is calculated by the following formula, where n is the number of all groups and m is the number of eliminated data.

[0079]

[0080] Get the average slope of the left and right virtual markers at two consecutive moments, respectively in, The average slope of the virtual marker on the left at the previous moment, The average slope of the virtual marker on the right at the previous moment, The average slope of the virtual marker on the left at the next moment, The average slope of the virtual marker on the right at the next moment.

[0081] If the slope change rates of the left and right sides at two moments are both less than the calibration value Δ, the robot's trajectory deviation at the next moment is determined again using the median slope algorithm. Otherwise, the dual standard calibration algorithm is run, and the Δ value range is: 0-0.015.

[0082] 2. Use the dual-standard verification algorithm to make a second trajectory deviation judgment:

[0083] Identify the preset marking points A, B, C, and D on the body of the fire-fighting robot from the image collected by the camera, where points B and C are located at the frontmost positions on the left and right sides of the body sheet metal of the fire-fighting robot, respectively, and point A is located behind the parallel line of the body of point B, and point D is located behind the parallel line of the body of point C. The distance between points A and B = the distance between points C and D = L, and the value of L can be set arbitrarily, for example, it can be 0.3-0.5m;

[0084] Identify road markings and road boundaries, see Figure 5 The left and right virtual marking lines are obtained in the image. Solve the intersection points H and I of the vehicle reference points B and C on the same horizontal axis with the virtual marking lines on both sides of the road, and at the same time, the lengths of the three line segments HB, BC and CI in the captured image can be solved, where HB and CI represent the distance from the left side of the fire robot body to the virtual marking line of the road and the distance from the right side of the body to the virtual marking line of the road, respectively, and BC represents the distance between the marking points on the left and right sides of the fire robot body sheet metal. The length of BC is known and can be in the range of 0.8-1.2m;

[0085] Solve the intersection points F and G of the reference points A and D on the same horizontal axis with the virtual road marking lines on both sides, and make the midpoint J of FG. The coordinates of J are fixed to the midpoint coordinates of FG at the first sampling; fit the connecting lines HF and GI; fit the vehicle connecting lines HF and GI, which will be used to solve the intersection point of the extended lines of the virtual road marking lines. HF and GI are two preset parallel lines in reality, but in the image collected by the camera, they will intersect at one point due to the imaging principle; make the intersection point of the extended lines of HF and GI, and solve the coordinates E(x E ,y E );

[0086] Based on the coordinate values ​​of points H and F, the angle θ between the virtual marking line on the left road and the virtual horizontal axis at this moment is calculated. i , where i can be the same as representing the image collected by the robot for the i-th time.

[0087]

[0088] Collect the angle of the virtual marking line on the left side of the road at the current time point and update the database {θ1,θ2...θ i}, where θ1, θ2, θ i The corresponding coordinates can be calculated at different time points;

[0089] Based on the coordinate values ​​of points G and I, the angle δ between the virtual marking line of the right road and the virtual horizontal axis at this moment is calculated. i :

[0090]

[0091] Collect the angle of the virtual road marking line on the left side at the current time point, and update the database {δ1,δ2...δi}; collect the distances αi and βi from the front of the fire robot body to the virtual road marking line at the current moment, and update the database {α1,α2...αi} and {β1,β2...βi};

[0092] Get the angle θ between the left and right road virtual marking lines and the virtual longitudinal axis at the current sampling time i , δ i , the distances αi and β between the front end of the robot and the virtual road marking lines i And the angle θ between the left and right road virtual marking lines at the previous sampling moment and the virtual longitudinal axis at this moment i-1 , δ i-1 , the distance between the front and sides of the robot body and the virtual road marking line α i-1 and β i-1 .

[0093] Perform the first verification judgment to determine whether the angle change rate of the left and right virtual marking lines at two consecutive moments is less than the calibration value at the same time. If so, return to the running median slope algorithm, otherwise perform the second verification judgment;

[0094] Perform a second check to determine whether the distance change rate of the left and right virtual marking lines at two consecutive moments is less than the calibration value at the same time. If so, return to the running median slope algorithm, otherwise display the vehicle deviation direction, deviation distance, and deviation angle; the deviation direction is determined based on comparing the α i Value greater than α i-1 If the value is set, the fire robot will deviate to the right, otherwise it will deviate to the left.

[0095] Calculate the deviation distance: If the firefighting robot deviates to the left, the deviation distance Δd = (α i -α i-1 )*(L BC / l BC ), where α i Indicates the distance between the left side of the front end of the fire robot and the virtual road marking line at the current sampling moment, α i-1 Indicates the distance between the front left side of the fire robot and the virtual road marking line at the previous sampling moment, L BC Indicates the actual distance of the reference mark point BC on the fire robot sheet metal, l BC represents the distance of the reference mark point BC on the sheet metal of the fire-fighting robot in the sampled image; if the fire-fighting robot deviates to the right, the deviation distance Δd = (β i -β i-1 )*(L BC / l BC ), where β i represents the distance between the front right side of the robot body and the virtual road marking line at the current sampling moment, β i-1 Indicates the distance between the front right side of the fire robot and the virtual road marking line at the previous sampling moment, L BC Indicates the actual distance of the reference mark point BC on the fire robot sheet metal, l BCRepresents the distance of the reference mark point BC on the sheet metal of the fire robot in the sampled image;

[0096] Calculate the deviation angle, then the deviation angle Δθ is equal to l EJ Angle with the longitudinal axis;

[0097] If a shutdown signal or manual control signal is detected, the program will be stopped; otherwise, the image information will be collected again and the program will be run.

[0098] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0099] In this embodiment, a device for determining the offset of a moving trajectory is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments, and will not be repeated hereafter. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0100] Figure 6 is a structural block diagram of a device for determining a displacement of a moving track according to an embodiment of the present invention. Figure 6 As shown, the device includes: an acquisition module 62, used to acquire a first image and a second image, wherein the first image and the second image are images acquired by an image acquisition device installed on the target object at different times during the process of the target object moving in a preset area, the first image includes a first virtual marking line, and the second image includes a second virtual marking line; a first determination module 64, used to determine whether the moving trajectory of the target object is offset according to the first virtual marking line and the second virtual marking line; a second determination module 66, used to determine the offset angle and offset distance of the target object according to the first virtual marking line and the second virtual marking line when it is determined that the moving trajectory of the target object is offset.

[0101] Optionally, the above-mentioned device is also used to determine whether the moving trajectory of the target object is offset according to the first virtual marking line and the second virtual marking line in the following manner: determine the first slope of the first virtual marking line and the second slope of the second virtual marking line; when it is determined that the difference between the first slope and the second slope is greater than or equal to a first preset threshold, determine that the moving trajectory of the target object is offset.

[0102] Optionally, the above-mentioned device is also used to achieve the determination of the first slope of the first virtual marking line and the second slope of the second virtual marking line in the following manner: selecting at least two groups of pixel point sets in the virtual marking line, wherein the vertical coordinates of the pixels in each group of pixel point sets are the same; determining the midpoint coordinates of each group of pixel point sets; determining the slope of the virtual marking line based on at least two midpoint coordinates, wherein the virtual marking line is the first virtual marking line or the second virtual marking line.

[0103] Optionally, the above-mentioned device is also used to determine the coordinates of a first intersection point of a first straight line and the first virtual marking line, wherein the first straight line is a straight line formed by a first marking point and a second marking point set on the target object; determine the coordinates of a second intersection point of a second straight line and the first virtual marking line, wherein the second straight line is a straight line formed by a third marking point and a fourth marking point set on the target object; and determine a first moving angle of the target object based on the coordinates of the first intersection point and the coordinates of the second intersection point.

[0104] Optionally, the above-mentioned device is also used to determine the coordinates of the third intersection of the first straight line and the second virtual marking line; determine the coordinates of the fourth intersection of the second straight line and the second virtual marking line; and determine the second moving angle of the target object based on the coordinates of the third intersection and the coordinates of the fourth intersection.

[0105] Optionally, the above-mentioned device is also used to determine whether the moving trajectory of the target object is offset according to the first virtual marking line and the second virtual marking line in the following manner: when it is determined that the difference between the first moving angle and the second moving angle is greater than or equal to a second preset threshold, it is determined that the moving trajectory of the target object is offset; determine the offset angle of the target object according to the first virtual marking line and the second virtual marking line: determine the difference between the first moving angle and the second moving angle as the offset angle of the target object.

[0106] Optionally, the above-mentioned device is also used to, when the first virtual marking line and the second virtual marking line are located on the left side of the target object, if the first moving angle is greater than the second moving angle, determine that the target object is shifted to the left, and if the first moving angle is less than the second moving angle, determine that the target object is shifted to the right; when the first virtual marking line and the second virtual marking line are located on the right side of the target object, if the first moving angle is greater than the second moving angle, determine that the target object is shifted to the right, and if the first moving angle is less than the second moving angle, determine that the target object is shifted to the left.

[0107] Optionally, the above-mentioned device is also used to implement the determination of whether the moving trajectory of the target object is offset according to the first virtual marking line and the second virtual marking line in the following manner: determine the first distance between the first marking point and the first intersection, and the second distance between the first marking point and the third intersection; when it is determined that the difference between the first distance and the second distance is greater than or equal to a third preset threshold, determine that the moving trajectory of the target object is offset; determine the offset distance of the target object according to the first virtual marking line and the second virtual marking line: determine the difference between the first distance and the second distance as the offset distance of the target object.

[0108] Optionally, the above-mentioned device is also used to, when the first virtual marking line and the second virtual marking line are located on the left side of the target object, if the first distance is greater than the second distance, determine that the target object is offset to the left, and if the first distance is less than the second distance, determine that the target object is offset to the right; when the first virtual marking line and the second virtual marking line are located on the right side of the target object, if the first distance is greater than the second distance, determine that the target object is offset to the right, and if the first distance is less than the second distance, determine that the target object is offset to the left.

[0109] Optionally, the above-mentioned device is also used to implement the determination of whether the moving trajectory of the target object is offset according to the first virtual marking line and the second virtual marking line in the following manner: determine the third distance between the second marking point and the second intersection, and the fourth distance between the second marking point and the fourth intersection; when it is determined that the difference between the third distance and the fourth distance is greater than or equal to a fourth preset threshold, determine that the moving trajectory of the target object is offset; determine the offset distance of the target object according to the first virtual marking line and the second virtual marking line: determine the difference between the third distance and the fourth distance as the offset distance of the target object.

[0110] Optionally, the above-mentioned device is also used to, when the first virtual marking line and the second virtual marking line are located on the left side of the target object, if the third distance is greater than the fourth distance, determine that the target object is offset to the left, and if the third distance is less than the fourth distance, determine that the target object is offset to the right; when the first virtual marking line and the second virtual marking line are located on the right side of the target object, if the third distance is greater than the fourth distance, determine that the target object is offset to the right, and if the third distance is less than the fourth distance, determine that the target object is offset to the left.

[0111] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0112] An embodiment of the present invention further provides a storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0113] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0114] S1, acquiring a first image and a second image, wherein the first image and the second image are images acquired at different times by an image acquisition device installed on the target object during the movement of the target object in a preset area, the first image includes a first virtual marking line, and the second image includes a second virtual marking line;

[0115] S2, determining whether the moving track of the target object is offset according to the first virtual marking line and the second virtual marking line;

[0116] S3: When it is determined that the moving track of the target object is offset, determine the offset angle and the offset distance of the target object according to the first virtual marking line and the second virtual marking line.

[0117] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

[0118] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0119] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0120] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0121] S1, acquiring a first image and a second image, wherein the first image and the second image are images acquired at different times by an image acquisition device installed on the target object during the movement of the target object in a preset area, the first image includes a first virtual marking line, and the second image includes a second virtual marking line;

[0122] S2, determining whether the moving track of the target object is offset according to the first virtual marking line and the second virtual marking line;

[0123] S3: When it is determined that the moving track of the target object is offset, determine the offset angle and the offset distance of the target object according to the first virtual marking line and the second virtual marking line.

[0124] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0125] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0126] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for determining the offset of a moving trajectory, characterized in that: include: Acquire a first image and a second image, wherein the first image and the second image are images acquired at different times by an image acquisition device installed on the target object during the movement of the target object in a preset area, the first image includes a first virtual marking line, the second image includes a second virtual marking line, and the first virtual marking line and the second virtual marking line are images obtained by photographing a reference object; determining whether the moving track of the target object is offset according to the first virtual marking line and the second virtual marking line; When it is determined that the moving track of the target object is offset, determining the offset angle and the offset distance of the target object according to the first virtual marking line and the second virtual marking line; Wherein, determining whether the moving track of the target object is offset according to the first virtual marking line and the second virtual marking line includes: Selecting at least two groups of pixel point sets from the virtual marking line, wherein the vertical coordinates of the pixel points in each group of pixel point sets are the same, wherein the virtual marking line is the first virtual marking line or the second virtual marking line; Determine the midpoint coordinates of each set of pixel points; Determine the slope between at least two of the midpoint coordinates to obtain a plurality of slopes; filter the plurality of slopes according to the positional relationship between the reference object and the target object to obtain the slope of the virtual marking line; Determining the slope of the first virtual marking line as a first slope, and determining the slope of the second virtual marking line as a second slope; It is determined whether the moving track of the target object is offset according to the first slope and the second slope.

2. The method according to claim 1, characterized in that The determining, according to the first slope and the second slope, whether the moving track of the target object is offset includes: When it is determined that the difference between the first slope and the second slope is greater than or equal to a first preset threshold, it is determined that the movement track of the target object is offset.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: Determine the coordinates of a first intersection point of a first straight line and the first virtual marking line, wherein the first straight line is a straight line formed by a first marking point and a second marking point set on the target object; Determine the coordinates of a second intersection point of a second straight line and the first virtual marking line, wherein the second straight line is a straight line formed by a third marking point and a fourth marking point set on the target object; A first moving angle of the target object is determined according to the coordinates of the first intersection point and the coordinates of the second intersection point.

4. The method according to claim 3, characterized in that The method further comprises: determining the coordinates of a third intersection point of the first straight line and the second virtual marking line; determining the coordinates of a fourth intersection point of the second straight line and the second virtual marking line; A second moving angle of the target object is determined according to the coordinates of the third intersection point and the coordinates of the fourth intersection point.

5. The method according to claim 4, characterized in that Determining whether the moving track of the target object is offset according to the first virtual marking line and the second virtual marking line includes: determining that the moving track of the target object is offset when it is determined that the difference between the first moving angle and the second moving angle is greater than or equal to a second preset threshold; Determining the offset angle of the target object according to the first virtual marking line and the second virtual marking line includes: determining a difference between the first moving angle and the second moving angle as the offset angle of the target object.

6. The method according to claim 5, characterized in that The method further comprises: In the case where the first virtual marking line and the second virtual marking line are located on the left side of the target object, if the first moving angle is greater than the second moving angle, it is determined that the target object is shifted to the left; if the first moving angle is less than the second moving angle, it is determined that the target object is shifted to the right; When the first virtual marking line and the second virtual marking line are located on the right side of the target object, if the first moving angle is greater than the second moving angle, it is determined that the target object is shifted to the right; if the first moving angle is less than the second moving angle, it is determined that the target object is shifted to the left.

7. The method according to claim 4, characterized in that The determining whether the moving track of the target object is offset according to the first virtual marking line and the second virtual marking line includes: determining a first distance between the first marking point and the first intersection point, and a second distance between the first marking point and the third intersection point; and determining that the moving track of the target object is offset when it is determined that the difference between the first distance and the second distance is greater than or equal to a third preset threshold value; Determining the offset distance of the target object according to the first virtual marking line and the second virtual marking line includes: determining a difference between the first distance and the second distance as the offset distance of the target object.

8. The method according to claim 7, characterized in that The method further comprises: In the case where the first virtual marking line and the second virtual marking line are located on the left side of the target object, if the first distance is greater than the second distance, it is determined that the target object is offset to the left; if the first distance is less than the second distance, it is determined that the target object is offset to the right; When the first virtual marking line and the second virtual marking line are located on the right side of the target object, if the first distance is greater than the second distance, it is determined that the target object is offset to the right; if the first distance is less than the second distance, it is determined that the target object is offset to the left.

9. The method according to claim 4, characterized in that The determining whether the moving track of the target object is offset according to the first virtual marking line and the second virtual marking line includes: determining a third distance between the second marking point and the second intersection point, and a fourth distance between the second marking point and the fourth intersection point; and determining that the moving track of the target object is offset when it is determined that the difference between the third distance and the fourth distance is greater than or equal to a fourth preset threshold value; Determining the offset distance of the target object according to the first virtual marking line and the second virtual marking line includes: determining a difference between the third distance and the fourth distance as the offset distance of the target object.

10. The method according to claim 9, characterized in that The method further comprises: In the case where the first virtual marking line and the second virtual marking line are located on the left side of the target object, if the third distance is greater than the fourth distance, it is determined that the target object is offset to the left; if the third distance is less than the fourth distance, it is determined that the target object is offset to the right; When the first virtual marking line and the second virtual marking line are located on the right side of the target object, if the third distance is greater than the fourth distance, it is determined that the target object is offset to the right; if the third distance is less than the fourth distance, it is determined that the target object is offset to the left.

11. A device for determining the deviation of a moving trajectory, characterized in that: include: an acquisition module, configured to acquire a first image and a second image, wherein the first image and the second image are images acquired at different times by an image acquisition device installed on the target object during the movement of the target object in a preset area, the first image includes a first virtual marking line, the second image includes a second virtual marking line, and the first virtual marking line and the second virtual marking line are images obtained by photographing a reference object; A first determining module, configured to determine whether a moving track of the target object is offset according to the first virtual marking line and the second virtual marking line; a second determining module, configured to determine, when it is determined that the moving track of the target object is offset, an offset angle and an offset distance of the target object according to the first virtual marking line and the second virtual marking line; The device is also used to select at least two groups of pixel point sets in the virtual marking line, wherein the vertical coordinates of the pixels in each group of pixel point sets are the same, wherein the virtual marking line is the first virtual marking line or the second virtual marking line; determine the midpoint coordinates of each group of pixel point sets; determine the slope between at least two of the midpoint coordinates to obtain multiple slopes; filter the multiple slopes according to the positional relationship between the reference object and the target object to obtain the slope of the virtual marking line; determine the slope of the first virtual marking line as the first slope, and determine the slope of the second virtual marking line as the second slope; and determine whether the moving trajectory of the target object is offset according to the first slope and the second slope.

12. A storage medium, characterized in that: The storage medium stores a computer program, wherein the program can be executed by a terminal device or a computer to execute the method described in any one of claims 1 to 10.

13. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 10.

Images