A dynamic capture and analysis system based on video images

Abstract

The present invention relates to the technical field of image processing, and discloses a dynamic capture and analysis system based on video images, including: an image preprocessing module, which captures markers in the image information based on the image information; a vehicle speed calculation module, which calculates the vehicle speed based on a second marker; a dynamic calculation module, which calculates the lifting trajectory and steering trajectory of the vehicle based on the image information, and calculates the actual path length of the vehicle based on the lifting trajectory and steering trajectory of the vehicle; a dynamic navigation module, which determines the current position of the vehicle on the navigation route based on the dynamic calculation module; a dynamic start engine, which determines whether to start a position correction module to correct the current position of the vehicle based on the information of the dynamic navigation module and the image preprocessing module; the present invention provides a system for navigating a vehicle based on machine vision.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and more specifically, to a dynamic capture and analysis system based on video images. Background Art

[0002] The vehicle image systems in the prior art are generally only used for purposes such as obstacle recognition and traffic light recognition, as an auxiliary for intelligent driving, and cannot participate in the vehicle navigation work. Summary of the Invention

[0003] The present invention provides a dynamic capture and analysis system based on video images, which solves the technical problem that the vehicle image system in the prior art cannot participate in the vehicle navigation work.

[0004] According to one aspect of the present invention, there is provided a dynamic capture and analysis system based on video images, including:

[0005] An image acquisition module, which is used to connect to an image acquisition device and receive the image information collected by the image acquisition device;

[0006] An image preprocessing module, which captures the markers in the image information based on the image information;

[0007] The markers include a first marker and a second marker, where the first marker is the connection point of two roads; the second marker is the ground marker of the road;

[0008] A vehicle speed calculation module, which calculates the vehicle speed based on the second marker, and the calculation method includes:

[0009] Extract the image information containing the same second marker, and screen out the image information with a time information difference of a set first value as the first image pair;

[0010] Judge the first distance between the second marker and the image acquisition device from the image information with a smaller time information value in the first image pair, judge the second distance between the second marker and the image acquisition device from the image information with a larger time information value in the first image pair, and calculate the vehicle speed based on the difference between the first distance and the second distance and the time information difference of the two graphic information in the first image pair;

[0011] A dynamic calculation module, which calculates the lifting trajectory and turning trajectory of the vehicle based on the image information, and calculates the actual path length of the vehicle based on the lifting trajectory and turning trajectory of the vehicle;

[0012] A navigation map generation module, which generates a navigation route for the vehicle based on the map information;

[0013] The dynamic navigation module determines the current position of the vehicle on the navigation route based on the dynamic calculation module, and marks the last passing point that the vehicle has recently passed as the first passing point, and marks the passing point that the vehicle is about to pass as the second passing point;

[0014] Dynamically start the engine, which determines whether to start the position correction module based on the information of the dynamic navigation module and the image preprocessing module. Starting the position correction module requires meeting the following conditions: when the image preprocessing module processes the image information to obtain the third identifier, the dynamic navigation module determines that the current position of the vehicle on the navigation route has not reached the passing point;

[0015] The position correction module generates all navigation paths that can connect to the first passing point from the starting point as the first path set, extracts the previous passing point of the first passing point from the first path set to generate the first point set, extracts all adjacent passing points of the first passing point, and deletes the passing points belonging to the first point set to generate the second point set;

[0016] Calculate the first probability parameter of the second point set, and the calculation formula is as follows:

[0017]

[0018] where i is the i-th passing point in the second point set, s is the number of passing points in the second point set, and q is the number of passing points in the first point set;

[0019] Generate navigation paths that can connect to the starting point for the passing points of the second point set;

[0020] Calculate the second probability parameter for the passing points of the second point set. The calculation formula for the second probability parameter of the i-th passing point in the second point set is as follows:

[0021]

[0022] where, N represents the set of the previous passing points of the navigation paths that can connect to the starting point of the i-th passing point in the second point set, and X j represents the j-th passing point in N; As the conditional probability, it is extracted and calculated based on the historical driving data trajectory;

[0023] The calculation of is an iterative formula;

[0024] Calculate the third probability parameter of the passing points of the second point set. The calculation formula for the second probability parameter of the i-th passing point in the second point set is as follows:

[0025] Extract waypoints from the second point set where the second probability parameter is greater than the set first threshold. If no waypoints are extracted, change the current vehicle position to the position of the second waypoint.

[0026] If one waypoint is extracted, change the current vehicle position to that waypoint.

[0027] If more than two waypoints are extracted, sort the extracted waypoints in descending order based on the magnitude of the second probability parameter, and select the waypoint ranked first in the ranking sequence. Change the current vehicle position to that waypoint.

[0028] Further, the image acquisition device includes a binocular stereo camera.

[0029] Further, the image acquisition device includes an ordinary camera.

[0030] Further, the earlier the acquisition time of the image information, the smaller the value of its time information.

[0031] Further, the navigation route includes waypoints, and the waypoints are connected by roads.

[0032] Further, the dynamic calculation module includes:

[0033] A lifting calculation module that extracts the two closest images with the same time information to generate a second image pair. The second image pair is from a binocular stereo camera. The focal length of the binocular stereo camera is f, the baseline width is b, and the height is h.

[0034] Generate depth information z = f * b / d, where x and y are image plane coordinates and z is the corresponding depth.

[0035] z A is the depth of the ground point A corresponding to the imaging point A (x A , y A ), and z A = h * f / y A ;

[0036] The calculation formula for the slope is as follows:

[0037]

[0038] where z is the depth information, z A is the depth of the ground point A corresponding to the imaging point A, y A is a coordinate of the imaging point A, and f is the focal length of the binocular stereo camera;

[0039] z - z A being positive indicates a downhill slope, and z - z A being negative indicates an uphill slope.

[0040] Furthermore, the dynamic calculation module includes:

[0041] A steering calculation module that calculates the steering trajectory of the vehicle based on a sensor connected to the vehicle's steering wheel.

[0042] Furthermore, if the current position of the vehicle does not change to the position of the second waypoint, the navigation map generation module regenerates the vehicle's navigation route.

[0043] Furthermore, the position correction module is connected to a historical database that stores historical driving data, including navigation paths and waypoints.

[0044] The beneficial effects of the present invention are as follows:

[0045] The system of the present invention dynamically identifies and determines the current position of the vehicle based on image information, and can navigate the vehicle based on the image system and map information;

[0046] And when there is a conflict between the image information and the predicted current position of the vehicle, the position is automatically corrected through probability calculation, solving the problem of position deviation in image information judgment. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a schematic diagram of the modules of a dynamic capture and analysis system based on video images of the present invention;

[0048] Figure 2 is a schematic diagram of the modules of the dynamic calculation module of the present invention.

[0049] In the figure: Image acquisition module 101, Image preprocessing module 102, Vehicle speed calculation module 103, Dynamic calculation module 104, Navigation map generation module 105, Dynamic navigation module 106, Dynamic start engine 107, Position correction module 108, Lift calculation module 201, Steering calculation module 202. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0050] Now, the subject matter described herein will be discussed with reference to example embodiments. It should be understood that discussing these embodiments is only to enable those skilled in the art to better understand and thus implement the subject matter described herein, and is not a limitation on the scope of protection, applicability, or examples set forth in the claims. The functions and arrangements of the elements discussed can be changed without departing from the scope of protection of the content of this specification. Each example can omit, substitute, or add various processes or components as needed. Additionally, the features described relative to some examples can also be combined in other examples.

[0051] Example 1

[0052] AsFigure 1 , Figure 2 As shown in Figure 2 , a dynamic capture and analysis system based on video images includes:

[0053] An image acquisition module 101, which is used to connect to an image acquisition device and receive the image information collected by the image acquisition device;

[0054] In an embodiment of the present invention, the image acquisition device includes a binocular stereo camera and an ordinary camera.

[0055] An image preprocessing module 102, which captures the markers in the image information based on the image information;

[0056] The markers include a first marker and a second marker, where the first marker is the connection point of two roads;

[0057] The second marker is the ground marker of the road;

[0058] A vehicle speed calculation module 103, which calculates the vehicle speed based on the second marker, and the calculation method includes:

[0059] Extract the image information containing the same second marker, and screen out the image information with a time information difference of a set first value as the first image pair;

[0060] Judge the first distance between the second marker and the image acquisition device from the image information with a smaller time information value in the first image pair, judge the second distance between the second marker and the image acquisition device from the image information with a larger time information value in the first image pair, and calculate the vehicle speed based on the difference between the first distance and the second distance and the time information difference of the two graphic information in the first image pair;

[0061] The value of the time information represents the sequence of the time when the image information is collected. The earlier the collection time, the smaller the value of its time information.

[0062] A dynamic calculation module 104, which calculates the lifting trajectory and steering trajectory of the vehicle based on the image information, and calculates the actual path length of the vehicle based on the lifting trajectory and steering trajectory of the vehicle;

[0063] The lifting trajectory actually refers to the set of information about the uphill and downhill of the vehicle, and the steering trajectory refers to the set of information about the vehicle's steering. Corresponding to the time axis, based on the vehicle speed calculation module 103 and the dynamic calculation module 104, the vehicle speed information, slope information and steering information at the time points on the time axis are obtained, and the actual path length of the vehicle on the navigation map can be calculated.

[0064] In an embodiment of the present invention, the dynamic calculation module 104 includes:

[0065] The lifting calculation module 201 extracts the two most recent images with the same time information to generate a second image pair. The second image pair is from a binocular stereo camera. The binocular stereo camera has a focal length of f, a baseline width of b, and a height of h;

[0066] Generate depth information z = f * b / d, where x and y are image plane coordinates and z is the corresponding depth;

[0067] z A is the depth of the ground point A corresponding to the imaging point A (x A , y A ), and z A = h * f / y A ;

[0068] The calculation formula for the slope is as follows:

[0069]

[0070] where z is the depth information, z A is the depth of the ground point A corresponding to the imaging point A, y A is a coordinate of the imaging point A, and f is the focal length of the binocular stereo camera;

[0071] z - z A being positive indicates a downhill slope, and z - z A being negative indicates an uphill slope;

[0072] The steering calculation module 202 calculates the steering trajectory of the vehicle based on a sensor connected to the vehicle's steering wheel.

[0073] The navigation map generation module 105 generates a navigation route for the vehicle based on map information. The navigation route includes waypoints, and the waypoints are connected by roads;

[0074] The dynamic navigation module 106 determines the current position of the vehicle on the navigation route based on the dynamic calculation module 104, and marks the most recently passed waypoint of the vehicle as the first waypoint and the upcoming waypoint to be passed by the vehicle as the second waypoint;

[0075] The dynamic engine start 107 determines whether to start the position correction module 108 based on the information of the dynamic navigation module 106 and the image pre - processing module 102. Starting the position correction module 108 requires meeting the following conditions: when the image pre - processing module 102 processes the image information to obtain the third marker, the dynamic navigation module 106 determines that the current position of the vehicle on the navigation route has not reached the waypoint;

[0076] The appearance of the third identifier indicates the arrival of the waypoint. When there is a deviation from the judgment of the dynamic navigation module 106 under the above conditions, the position correction module 108 is activated at this time.

[0077] The position correction module 108 generates all navigation paths that can connect to the first waypoint from the starting point as the first path set, extracts the previous waypoint of the first waypoint from the first path set to generate the first point set, extracts all adjacent waypoints of the first waypoint, and deletes the waypoints belonging to the first point set to generate the second point set;

[0078] The starting point represents the first waypoint of the navigation path.

[0079] Calculate the first probability parameter of the second point set, and the calculation formula is as follows:

[0080]

[0081] where i is the i-th waypoint in the second point set, s is the number of waypoints in the second point set, and q is the number of waypoints in the first point set;

[0082] Generate navigation paths that can connect to the starting point for the waypoints in the second point set;

[0083] Calculate the second probability parameter for the waypoints in the second point set. The calculation formula for the second probability parameter of the i-th waypoint in the second point set is as follows:

[0084]

[0085] where N represents the set of the previous waypoints of the navigation paths that can connect to the starting point of the i-th waypoint in the second point set, and X j represents the j-th waypoint in N; As the conditional probability, it is calculated and obtained based on the extraction from the historical driving data trajectory;

[0086] The calculation of is an iterative formula;

[0087] In an embodiment of the present invention, the position correction module 108 is connected to a historical database, and the historical database stores historical driving data, and the historical driving data includes navigation paths and waypoints.

[0088] Calculate the third probability parameter of the waypoints in the second point set. The calculation formula for the second probability parameter of the i-th waypoint in the second point set is as follows:

[0089] Extract the waypoints in the second point set whose second probability parameter is greater than the set first threshold. If no waypoints are extracted, change the current position of the vehicle to the position of the second waypoint;

[0090] If a waypoint is extracted, change the current position of the vehicle to that waypoint;

[0091] If more than two waypoints are extracted, sort the extracted waypoints from largest to smallest based on the magnitude of the second probability parameter, and select the waypoint ranked first in the ranking sequence, and change the current position of the vehicle to that waypoint.

[0092] If the changed current position of the vehicle is not the position of the second waypoint, the navigation map generation module 105 regenerates the navigation route of the vehicle.

[0093] The above has described the embodiments of this embodiment, but this embodiment is not limited to the above specific implementation manners. The above specific implementation manners are merely illustrative rather than restrictive. Under the inspiration of this embodiment, those of ordinary skill in the art can also make many forms without departing from the purpose of this embodiment and the scope protected by the claims, and all belong to the protection scope of this embodiment.

Claims

1. A dynamic capture and analysis system based on video images, characterized in that Including: An image acquisition module, which is used to connect to an image acquisition device and receive the image information collected by the image acquisition device; An image preprocessing module, which captures the markers in the image information based on the image information; The markers include a first marker and a second marker, where the first marker is the connection point of two roads; The second marker is the ground marker of the road; A vehicle speed calculation module, which calculates the vehicle speed based on the second marker, and the calculation method includes: Extracting the image information containing the same second marker, and screening out the image information with a time information difference of a set first value from it as the first image pair; Judging the first distance between the second marker and the image acquisition device from the image information with a smaller time information value in the first image pair, judging the second distance between the second marker and the image acquisition device from the image information with a larger time information value in the first image pair, and calculating the vehicle speed based on the difference between the first distance and the second distance and the time information difference of the two graphic information in the first image pair; A dynamic calculation module, which calculates the lifting trajectory and steering trajectory of the vehicle based on the image information, and calculates the actual path length of the vehicle based on the lifting trajectory and steering trajectory of the vehicle; A navigation map generation module, which generates a navigation route for the vehicle based on the map information; A dynamic navigation module, which judges the current position of the vehicle on the navigation route based on the dynamic calculation module, and marks the nearest passing point passed by the vehicle as the first passing point, and marks the passing point that the vehicle is about to pass as the second passing point; A dynamic start engine, which judges whether to start the position correction module based on the information of the dynamic navigation module and the image preprocessing module. To start the position correction module, the following conditions need to be met: when the image preprocessing module processes the image information to obtain the third marker, the dynamic navigation module judges that the current position of the vehicle on the navigation route has not reached the passing point; A position correction module, which generates all navigation paths that can connect to the first passing point from the starting point as the first path set, extracts the previous passing point of the first passing point from the first path set to generate the first point set, extracts all adjacent passing points of the first passing point, and deletes the passing points belonging to the first point set to generate the second point set; Calculating the first probability parameter of the second point set, and the calculation formula is as follows: Where i is the i-th passing point in the second point set, s is the number of passing points in the second point set, and q is the number of passing points in the first point set; Generating navigation paths that can connect to the starting point for the passing points in the second point set; Calculating the second probability parameter for the passing points in the second point set, and the calculation formula for the second probability parameter of the i-th passing point in the second point set is as follows: Among them, N represents the set of the previous waypoints of the navigation path that can connect to the starting point for the i-th waypoint of the second point set, and Xj represents the j-th waypoint in N; P(X i |X j ) as the conditional probability is calculated based on the extraction from the historical driving data trajectory; m ji (X i ) is calculated by an iterative formula; Calculate the third probability parameter of the waypoints of the second point set. The calculation formula for the third probability parameter of the i-th waypoint of the second point set is as follows: Extracting the passing points in the second point set whose third probability parameter is greater than the set first threshold. If no passing point is extracted, changing the current position of the vehicle to the position of the second passing point; If one passing point is extracted, changing the current position of the vehicle to that passing point; If two or more passing points are extracted, sorting the extracted passing points from largest to smallest based on the magnitude of the third probability parameter, taking the passing point with the first ranking sequence, and changing the current position of the vehicle to that passing point.

2. The dynamic capture and analysis system based on video images according to claim 1, characterized in that, The image acquisition device includes a binocular stereo camera.

3. A dynamic capture and analysis system based on video images according to claim 1 or 2, characterized in that, The image acquisition device includes an ordinary camera.

4. A dynamic capture and analysis system based on video images according to claim 1, characterized in that, The earlier the acquisition time of the image information, the smaller the value of its time information.

5. A dynamic capture and analysis system based on video images according to claim 1, characterized in that, The navigation route includes waypoints, and the waypoints are connected by roads.

6. The dynamic capture and analysis system based on video images according to claim 1, characterized in that, The dynamic calculation module includes: The lifting calculation module extracts the two images with the same time information that are the closest to generate a second image pair. The second image pair is from the binocular stereo camera. The focal length of the binocular stereo camera is f, the baseline width is b, and the height is h. Generate the depth information z = f * b / d, where x and y are the image plane coordinates and z is the corresponding depth. z A For the depth z of the ground point corresponding to the imaging point A(x A , y A ), A z = h * f / y A ; The calculation formula for the slope is as follows: where y A is a coordinate of imaging point A; z-z A A positive value indicates a downhill slope, z-z A A negative value indicates an uphill slope.

7. An analysis system for dynamic capture based on video images according to claim 1 or 6, characterized in that The dynamic calculation module includes: The steering calculation module calculates the steering trajectory of the vehicle based on the sensor connected to the vehicle's steering wheel.

8. A dynamic capture and analysis system based on video images according to claim 1, characterized in that, If the current position of the vehicle does not change to the position of the second waypoint, the navigation map generation module regenerates the navigation route of the vehicle.

9. The dynamic capture and analysis system based on video images according to claim 1, wherein, The position correction module is connected to a historical database, and historical driving data is stored in the historical database. The historical driving data includes the navigation path and waypoints.

Images