A method and system for identifying the moving direction of an object at an intersection
Through image processing technology, the motion trajectory points of vehicles and pedestrians at intersections are identified, and the motion direction is judged by algorithms, which solves the problem of inaccurate recognition of vehicle and pedestrians in the prior art, and improves the efficiency of traffic management and the smoothness of roads.
Patent Information
- Application Number
- CN201910907356.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-09-24
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2039-09-24
AI Technical Summary
The existing method of obtaining traffic information at intersections is difficult to accurately identify the movement direction of vehicles and pedestrians, which affects the formulation of signal light strategies and traffic management efficiency.
By obtaining multiple images of the intersection, identifying the moving trajectory points of the target object, and analyzing the positional relationship of the trajectory points according to the preset algorithm, determining the moving direction of the target object, including straight, left or right. This method uses the relative positional relationship between the trajectory vector and the reference system, and combines image processing and algorithm to determine the driving direction of the vehicle.
It realizes accurate identification of the movement direction of vehicles and pedestrians at intersections, provides traffic information, supports more scientific signal light strategy formulation and traffic management, and improves road smoothness and safety.
Smart Images

Figure CN112634320B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to a method for identifying the moving direction of an object at an intersection. Background Art
[0002] Traffic flow information at intersections can be used to formulate traffic light strategies at intersections. Reasonable traffic light strategies can achieve the goals of reducing congestion and blockage, ensuring smooth urban roads, and avoiding traffic accidents. With the continuous development of computer technology, image processing, pattern recognition and other technologies, traffic flow or detection methods based on video image processing have been increasingly widely used. The present application provides a method for identifying the movement direction of objects such as vehicles and pedestrians based on video image processing, so as to identify the movement direction of objects such as vehicles at intersections, grasp the movement trajectory of the target object, or facilitate the statistics of movement information of different movement directions. Summary of the invention
[0003] One aspect of the present invention provides a method for identifying the direction of movement of an object at an intersection, the method comprising: acquiring multiple images of the intersection, at least one of the multiple images including a target object; obtaining multiple trajectory points on the target object's motion trajectory based on the coordinate position of the target object in the multiple images; and determining the direction of movement of the target object by analyzing the positional relationship of the multiple trajectory points according to a preset algorithm.
[0004] In some embodiments, the plurality of images are taken from video data of the intersection.
[0005] In some embodiments, the movement direction includes at least one of going straight, turning left, or turning right.
[0006] In some embodiments, analyzing the positional relationship of the multiple trajectory points according to a preset algorithm to determine the direction of movement of the target object includes: generating one or more trajectory vectors based on the multiple trajectory points; determining at least one reference system; and determining the direction of movement of the target object based on the relative positional relationship between the one or more trajectory vectors and at least one reference system.
[0007] In some embodiments, the reference system includes a reference vector; analyzing the positional relationship of the multiple trajectory points according to a preset algorithm to determine the driving direction of the target vehicle includes: generating one or more trajectory point vectors based on the multiple trajectory points; calculating the angles related to the one or more trajectory point vectors and the reference vector to obtain one or more trajectory point vector angles; the one or more trajectory point vector angles include the angles between one or more trajectory point vectors and the reference vector or include the difference between the maximum angle and the minimum angle between the one or more trajectory point vectors and the reference vector; comparing the one or more trajectory point vector angles with a preset angle threshold; in response to the one or more trajectory point angles being less than the preset angle threshold, determining that the target object is moving straight.
[0008] In some embodiments, the reference vector reflects an extension direction of at least one incoming road of the target object in the plurality of images; the target object enters the intersection via the incoming road.
[0009] In some embodiments, the one or more trajectory point vectors include vectors pointing from a specified trajectory point in the plurality of trajectory points to one or more other trajectory points.
[0010] In some embodiments, the designated trajectory point is a starting trajectory point.
[0011] In some embodiments, the reference system includes a first reference coordinate system; in response to the angle between the one or more trajectory points being not less than a preset angle threshold, the following steps are performed: generating a start-end point vector based on the multiple trajectory points; the quadrant of the start-end point vector in the first reference coordinate system is the quadrant in which the start-end point vector is located when the start-end point vector is translated so that the starting point of the start-end point vector coincides with the origin of the first reference coordinate system; and, determining the concavity of the motion trajectory of the target object relative to the straight line where the start-end point vector is located based on the multiple trajectory points; and determining whether the target object is turning left or right based on the quadrant in which the start-end point vector is located and the concavity of the motion trajectory.
[0012] In some embodiments, the reference system includes a first reference coordinate system; analyzing the positional relationship of the multiple trajectory points according to a preset algorithm to determine the direction of movement of the vehicle includes: generating a start-end point vector based on the multiple trajectory points; determining the quadrant of the start-end point trajectory vector in the first reference coordinate system; the quadrant of the start-end point vector in the first reference coordinate system is the quadrant in which the start-end point vector is located when the start-end point vector is translated so that the starting point of the start-end point vector coincides with the origin of the first reference coordinate system; and, determining the convexity of the motion trajectory of the target object relative to the straight line where the start-end point vector is located based on the multiple trajectory points; based on the quadrant in which the start-end point vector is located and the convexity of the motion trajectory, determining whether the target object is turning left or right.
[0013] In some embodiments, at least one coordinate axis of the first reference coordinate system reflects an extension direction of at least one road in the plurality of images.
[0014] In some embodiments, at least one coordinate axis of the first reference coordinate system reflects an extension direction of at least one incoming road of the target object in the plurality of images; the target object enters the intersection via the incoming road.
[0015] In some embodiments, obtaining multiple trajectory points on the target object's motion trajectory based on the target object's coordinate position in multiple images also includes: mapping the target object's coordinate position in multiple images to a target coordinate system to obtain the multiple trajectory points; an extension direction of at least one road in the multiple images is parallel to one of its coordinate axes in the target coordinate system.
[0016] In some embodiments, obtaining multiple trajectory points on the target object's motion trajectory based on the target object's coordinate position in multiple images also includes: obtaining multiple sampling points and their coordinate data in at least one image among the multiple images; obtaining multiple reference points and their coordinate data in the target coordinate system; establishing a coordinate transformation matrix based on the coordinate data of the multiple sampling points and the coordinate data of the multiple reference points; and performing operations on the coordinate position of the target object in the multiple images and the coordinate transformation matrix to obtain the multiple trajectory points.
[0017] In some embodiments, acquiring multiple sampling points and their coordinate data in at least one image among the multiple images includes: receiving multiple annotation points drawn by a user in at least one image among the multiple images; the multiple annotation points are not collinear; generating a standard rectangle based on the multiple annotation points; and determining the vertices of the standard rectangle and their coordinate data in at least one image among the multiple images as the multiple sampling points and their coordinate data.
[0018] In some embodiments, the plurality of annotation points are located on a road marking line in at least one image among the plurality of images, and the plurality of annotation points form a quadrilateral.
[0019] Another aspect of the present invention provides a system for identifying the direction of vehicle movement at an intersection, the system comprising: an acquisition module, used to acquire multiple images of the intersection, at least one of the multiple images including a target object; a trajectory point determination module, used to obtain multiple trajectory points on the target object's driving trajectory based on the coordinate position of the target object in the multiple images; and a processing module, used to analyze the positional relationship of the multiple trajectory points according to a preset algorithm to determine the direction of movement of the target object.
[0020] Another aspect of the present invention provides a device for identifying the direction of movement of a vehicle at an intersection, characterized in that the device includes at least one processor and at least one memory; the at least one memory is used to store computer instructions; the at least one processor is used to execute at least part of the computer instructions to implement any operation described in any one of the methods of any embodiment in this specification.
[0021] Another aspect of the present invention provides a computer storable medium, wherein the storage medium stores computer instructions, and when at least part of the computer instructions are executed by a processor, the operations described in any one of the methods of any embodiment in this specification are implemented. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] The present invention will be further described in the form of exemplary embodiments, which will be described in detail by way of the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:
[0023] Figure 1 is an exemplary system block diagram according to some embodiments of the present invention.
[0024] Figure 2 is an exemplary flow chart of a method for identifying a vehicle's driving direction at an intersection according to some embodiments of the present invention.
[0025] Figure 3 is an exemplary sub-flow chart of coordinate transformation according to some embodiments of the present invention.
[0026] Figure 4 is a schematic diagram of obtaining sampling points and their coordinate data according to some embodiments of the present invention.
[0027] Figure 5 is an exemplary sub-flowchart of determining straight travel according to some embodiments of the present invention.
[0028] Figure 6 is an exemplary sub-flowchart of determining a turning direction according to some embodiments of the present invention.
[0029] Figure 7 It is a schematic diagram of judgment rules for determining a turning direction according to some embodiments of the present invention. DETAILED DESCRIPTION
[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of the present invention. For ordinary technicians in this field, the present invention can also be applied to other similar scenarios based on these drawings without creative work. Unless it is obvious from the language environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.
[0031] It should be understood that the "system", "device", "unit" and / or "module" used herein are a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.
[0032] As stated in the present invention and claims, unless the context clearly indicates an exception,
[0033] The words "a", "an", "a kind" and / or "the" do not refer to the singular, but may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list, and the method or device may also include other steps or elements.
[0034] The present invention uses a flow chart to illustrate the operations performed by the system according to an embodiment of the present invention. It should be understood that the preceding or following operations are not necessarily performed precisely in order. On the contrary, the various steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or one or more operations may be removed from these processes.
[0035] One or more embodiments of the present application can be applied to the detection of the direction of motion of target objects in different venues. The venues to which the present application can be applied include, but are not limited to, road intersections, specific road areas, specific venue entrances and exits, etc. For example, crossroads, T-junctions, three-way intersections, multiple intersections, main road / auxiliary road entrances and exits, parking lot entrances and exits, service area entrances and exits, etc. The target object in the present application can be a vehicle, for example, a motor vehicle or a non-motor vehicle. The motor vehicle includes, but is not limited to, private cars, taxis, trucks, buses, etc. The non-motor vehicle includes, but is not limited to, electric vehicles for the elderly, two-wheeled battery vehicles, tricycles, bicycles, etc. In some embodiments, the target object can also be other devices that can travel on the road, such as unmanned vehicles, mobile robots that achieve specific tasks (inspection, surveying, etc.). In some embodiments, the target object can also be other movable objects such as pedestrians and animals. One or more embodiments of the present application can determine traffic flow information based on the direction of travel of the vehicle, and the traffic flow information includes, but is not limited to, the direction of travel of vehicles at each intersection of the venue, the number of vehicles entering each intersection of the venue, the number of vehicles in each direction of travel at each intersection of the venue, and time information related to the traffic flow of the venue. It should be understood that the application scenarios of the systems and methods of one or more embodiments of this specification are only some examples of one or more embodiments of this specification. For ordinary technicians in this field, one or more embodiments of this specification can also be applied to other similar scenarios based on these drawings without creative work. For example, regional intersections in parking lots. For another example, detection and statistics of pedestrian trajectories at intersections.
[0036] Figure 1 1 is an exemplary system block diagram according to some embodiments of the present invention. In some embodiments, a system 100 for identifying the moving direction of an object at an intersection may include an acquisition module 102 , a trajectory point determination module 104 , and a processing module 106 .
[0037] The acquisition module 102 may be used to acquire multiple images of an intersection. In some embodiments, at least one of the multiple images of the intersection includes a target object.
[0038] The trajectory point determination module 104 may be used to determine the coordinate position of the target object in a plurality of images, and then obtain a plurality of trajectory points on the motion trajectory of the target object.
[0039] The processing module 106 may be configured to analyze the positional relationship of the plurality of trajectory points according to a preset algorithm to determine the direction of motion of the target object. In some embodiments, the direction of motion includes at least one of going straight, turning left, or turning right. In some embodiments, analyzing the positional relationship of the plurality of trajectory points according to a preset algorithm to determine the direction of motion of the target object includes: generating one or more trajectory vectors based on the plurality of trajectory points; determining at least one reference system; and determining the direction of motion of the target object based on the relative positional relationship between the one or more trajectory vectors and the at least one reference system.
[0040] It should be understood that Figure 1 The system and its modules shown can be implemented in various ways. For example, in some embodiments, the system and its modules can be implemented by hardware, software, or a combination of software and hardware. Among them, the hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or a dedicated design hardware. Those skilled in the art will understand that the above methods and systems can be implemented using computer executable instructions and / or included in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The system and its modules of one or more embodiments of this specification can not only be implemented by hardware circuits such as ultra-large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., but can also be implemented by software such as executed by various types of processors, and can also be implemented by a combination of the above hardware circuits and software (e.g., firmware).
[0041] Next, the present application takes the target vehicle driving direction detection as an example to elaborate the technical solution of the present application, but it should not be understood as a limitation of the present application. For example, in some embodiments, the target vehicle can be replaced by a pedestrian, a movable device, etc.
[0042] Figure 2 FIG. 1 is an exemplary flow chart of a method for identifying a vehicle's driving direction at an intersection according to some embodiments of the present invention. Figure 2 As shown, a method for identifying a vehicle driving direction at an intersection may include:
[0043] Step 202 , acquiring multiple images of the intersection, at least one of the multiple images includes the target vehicle; in some embodiments, step 202 may be implemented by the acquisition module 102 .
[0044] In some embodiments, the intersection may be an intersection of roads, such as a crossroads, a T-junction, a three-way intersection, a multi-road intersection, etc. In some embodiments, the image may be a photo taken by an image acquisition device at regular intervals, or may be a picture obtained by filtering and cutting a video taken by an image acquisition device. The image acquisition device includes but is not limited to an ordinary camera, a 360° panoramic camera, a fisheye camera, a telephoto camera, a 3D fixed-focus camera, a light field zoom camera, etc. The image acquisition device may be multiple or one.
[0045] In some embodiments, an image acquisition device may be provided at each intersection direction of the intersection, and each image acquisition device may acquire an image of the intersection where it is located. Each image acquisition device may be installed above the intersection where it is located to acquire an image of the intersection with a straight lane; or it may be installed on one side of each intersection to capture a complete image of the intersection, and the acquired image may have a certain degree of deformation, for example, tilt deformation.
[0046] In some embodiments, an image acquisition device may be provided at the intersection, and the image acquisition device may acquire a complete image of the entire intersection. The image acquisition device may be a 360° panoramic camera installed at any position of the intersection, or may be any camera installed above the area near the intersection and capable of capturing a complete image of the intersection, such as an ordinary camera. In some embodiments, the image acquired by the image acquisition device may have certain deformations, such as tilt deformation, "near larger, far smaller" distortion, etc. In some embodiments, the image acquired by the image acquisition device may be non-deformed, for example, the image acquisition device is located directly above or close to directly above the intersection, and the camera is facing vertically downward. At this time, the viewing angle of the image acquired by the device is a top view or close to a top view, and the lanes in the image are consistent with or close to the real lanes. While keeping the camera facing vertically downward, the camera's rotation angle may be adjusted so that the lanes in the acquired image are parallel or perpendicular to the edge of the image.
[0047] In some embodiments, the acquisition module may acquire multiple images of the intersection through an image acquisition device, and the multiple images may be photos taken by the image acquisition device at a certain interval, or may be image frames taken by the image acquisition device at a certain interval. At least one of the multiple images includes a target object. In some embodiments, the multiple images may be selected from video data of the intersection. The video data may be taken by an image acquisition device at an intersection. The multiple images may be image frames in the video, for example, images of each frame in the video, or for example, images of every 5 frames in the video. In some embodiments, the multiple images may be images after filtering a certain number of starting frames and ending frames, for example, images after filtering 50 frames at the beginning and the end.
[0048] Step 204 , determining the coordinate position of the target vehicle in the multiple images, and then obtaining multiple track points on the target vehicle's driving track; in some embodiments, step 204 can be implemented by the track point determination module 104 .
[0049] In some embodiments, the coordinate position may be the position of the target vehicle in the initial coordinate system. The initial coordinate system may be a rectangular coordinate system determined based on the image. The origin of the initial coordinate system may be the center of the intersection in the image, any vertex of the image, or any point in the image; the x-axis and y-axis of the initial coordinate system may be respectively along one of the road directions of the intersection in the image, or may be respectively parallel to any two intersecting borders of the image.
[0050] In some embodiments, a machine learning model may be provided to identify the coordinate position of the target vehicle in each image. In some embodiments, image processing means may also be used to identify the target vehicle in the image, such as by a threshold segmentation algorithm, and further determine the coordinate position of at least one pixel point in the area where the target vehicle is located in the image, for example, determine the center pixel point of the area where the target vehicle is located, determine the coordinate position of the center pixel point in the image as the coordinate position of the target vehicle in the image, or comprehensively determine the coordinate position of the target vehicle in the image based on multiple pixel points in the area where the target vehicle is located.
[0051] In some embodiments, the target vehicle's driving trajectory may be a trajectory directly obtained from images of consecutive frames in the video data. In some embodiments, the target vehicle's driving trajectory may also be formed by fitting the target vehicle's position points in multiple images. The fitting may obtain the trajectory equation f(x) of the target vehicle's driving trajectory. The fitting includes linear fitting and nonlinear fitting, and the fitting algorithm includes but is not limited to the least squares algorithm, the image thinning algorithm, the parameter curve model algorithm, and the like. Specifically, when the fitting is linear fitting, the fitting algorithm may be a linear least squares problem algorithm; when the fitting is nonlinear fitting, the fitting algorithm may be a nonlinear least squares problem algorithm, such as a Gauss Newton algorithm, a Levenberg-Marquardt algorithm, and the like.
[0052] In some embodiments, the track points may be position points of the target vehicle in the multiple images. For example, the coordinate positions of the target object in the multiple images may be directly determined as the multiple track points. In some embodiments, the track points may also be any points on the track equation f(x) of the target vehicle's driving track.
[0053] In some embodiments, the track points are track points after the position points of the target vehicle in the multiple images are filtered. The filtering includes but is not limited to filtering the track points at the edge of the image and filtering the track points that deviate far from the driving track. For example, if the quality of the last several frames of the target vehicle before driving out of the image (or shooting area) is poor, they can be directly eliminated.
[0054] In some embodiments, the filtered image edge trajectory points may be preset trajectory points on the image with a certain starting frame number and ending frame number, for example, 50 frames are removed from the starting frame number and the ending frame number of multiple images. In some embodiments, the filtered image edge trajectory points may also be trajectory points with poor edge quality of the filtered image based on an algorithm. In some embodiments, the coordinate position of the target vehicle in multiple images may be judged, and when the distance from the horizontal coordinate or vertical coordinate of a certain coordinate position to a certain boundary of the image (such as the first row, the last row, the first column or the last column of the image) is less than a set threshold, the coordinate position is deleted and is not used as one of the trajectory points. The set threshold may be 30 pixels, 50 pixels, 60 pixels, etc.
[0055] In some embodiments, filtering the trajectory points that deviate far from the driving trajectory may be manually deleting the trajectory points with large deviations, and the number of deletions may be manually set, for example, deleting 3 trajectory points with large deviations. In some embodiments, filtering the trajectory points that deviate far from the driving trajectory may also be automatically filtering the trajectory points that deviate far based on an algorithm. The algorithm may be a data smoothing algorithm based on the least squares method, for example, a three-point linear smoothing algorithm, a five-point linear smoothing algorithm, a seven-point linear smoothing algorithm, etc. In some embodiments, the algorithm may also be a preset filtering algorithm. Specifically, in some embodiments, the filtering algorithm is to filter the N trajectory point vectors with the largest angle with the reference vector, where N≥1; in some embodiments, the N trajectory point vectors with the largest angle with the reference vector and the M trajectory point vectors with the smallest angle with the reference vector may also be filtered out at the same time, where N≥1, M≥1. For example, N=3, M=3. The reference vector may be a vector established in the initial coordinate system that can be used to determine the direction or angle of the trajectory point vector, for example, the reference vector may be a vector along the extension direction of the road that the vehicle enters. The trajectory point vector may be a plurality of vectors formed by connecting other trajectory points to a specified trajectory point among a plurality of trajectory points. The designated track point may be a starting track point. The starting track point may be the earliest track point in time sequence among a plurality of track points of the vehicle.
[0056] In some embodiments, the coordinate position of the target vehicle in the multiple images can be mapped to a target coordinate system to obtain the multiple track points; the extension direction of at least one road in the multiple images is parallel to a certain coordinate axis in the target coordinate system.
[0057] In some embodiments, the mapping may be an affine transformation of the multiple images. The affine transformation is a linear transformation from a two-dimensional coordinate to a two-dimensional coordinate. The affine transformation may include translation, scaling, flipping, rotation, and shearing. The affine transformation may maintain the "straightness" of the two-dimensional image, that is, the affine transformation may make the original image straighter, so as to improve the accuracy of determining the trajectory points in some embodiments of this specification. For detailed steps on coordinate position mapping, please refer to the description in this specification. Figure 3 and Figure 4 Related content.
[0058] Step 206 , analyzing the positional relationship of the plurality of trajectory points according to a preset algorithm to determine the driving direction of the vehicle. In some embodiments, step 206 may be implemented by the processing module 106 .
[0059] The preset algorithm is an algorithm used to determine the vehicle's driving direction based on the positional relationship of multiple trajectory points, including but not limited to a trajectory point fitting algorithm, a trajectory point filtering algorithm, a coordinate conversion algorithm, a vector translation algorithm, an algorithm for determining the target vehicle's driving direction, an algorithm for determining whether the target vehicle is going straight, an algorithm for determining whether the target vehicle is turning left or right, and the thresholds involved in each algorithm.
[0060] In some embodiments, the preset algorithm can be adjusted according to the determination result. For example, the actual trajectory of some vehicles can be obtained, and according to the difference between the driving direction determination result obtained by using the preset algorithm and the actual driving direction of the vehicle, the algorithm and its threshold for determining the driving direction of the target vehicle can be adjusted to improve the accuracy of the algorithm.
[0061] In some embodiments, the driving direction includes at least one of going straight, turning left or turning right. In some embodiments, the driving direction also includes a specific oncoming direction and an outgoing direction, and the driving direction of the vehicle can be determined to be going straight, turning left or turning right according to the oncoming direction and the outgoing direction. For example, if the outgoing direction of the vehicle is on the right side of the oncoming direction, the driving direction of the vehicle is turning right; if the outgoing direction of the vehicle is on the left side of the oncoming direction, the driving direction of the vehicle is turning left; if the outgoing direction of the vehicle is in front of the oncoming direction, the driving direction of the vehicle is going straight.
[0062] In some embodiments, analyzing the positional relationship of the multiple trajectory points according to a preset algorithm to determine the driving direction of the vehicle includes: generating one or more trajectory vectors based on the multiple trajectory points; at least one reference system; and determining the driving direction of the target vehicle based on the relative positional relationship between the one or more trajectory vectors and at least one reference system.
[0063] In some embodiments, the multiple trajectory points include at least a starting trajectory point and an ending trajectory point. The starting trajectory point may be the earliest point in the time sequence among the multiple trajectory points, and the ending trajectory point may be the latest point in the time sequence among the multiple trajectory points. The timing of the trajectory points may correspond to the acquisition time of the image corresponding to the point. The trajectory vector may include a trajectory point vector and / or a starting and ending point vector. The trajectory point vector may be a plurality of vectors formed by connecting a specified trajectory point among the multiple trajectory points with other trajectory points. In some embodiments, the specified trajectory point may be a starting trajectory point. The starting and ending point vector may be a vector pointing from the starting trajectory point to the ending trajectory point.
[0064] In some embodiments, the reference system may be a reference coordinate system or a reference vector. The objects include, but are not limited to, roads at intersections, intersection centers, lane lines, and the like. The reference system may be determined based on the objects. For example, a first reference coordinate system may be determined based on the roads at intersections and intersection centers. Specifically, the origin of the first reference coordinate system may be determined based on the intersection center (e.g., the two coincide), and the x-axis and y-axis of the first reference coordinate system may be determined based on the direction of the road at the intersection (e.g., the two are parallel). For another example, the direction of the reference vector may be determined based on the lane line (e.g., the two are parallel or perpendicular). In some embodiments, the first reference coordinate system may be established based on the roads at intersections and intersection centers in the initial coordinate system, or may be established based on the roads at intersections and intersection centers in the target coordinate system.
[0065] In some embodiments, the driving direction of the target vehicle includes at least one of going straight, turning left or turning right. The driving direction also includes a specific oncoming direction and an outgoing direction, and the driving direction of the vehicle can be determined as going straight, turning left or turning right according to the oncoming direction and the outgoing direction.
[0066] In some embodiments, analyzing the positional relationship of the multiple trajectory points according to a preset algorithm to determine the driving direction of the vehicle may also include: pre-dividing n intersection areas, n≥1; detecting the intersection area where the target vehicle's starting trajectory point is located, marking the intersection area as 1, and marking other intersection areas as 2, 3, ..., n in a counterclockwise order; detecting the mark value of the intersection area where the target vehicle's trajectory end point is located; and calculating the mark value of the intersection where the trajectory end point is located and the mark value of the intersection area where the starting trajectory point is located to determine the driving direction of the target vehicle.
[0067] Specifically, taking an intersection as an example, the four entrance roads of the intersection are divided into four intersection areas in advance; the intersection area where the starting trajectory point of the target vehicle is located is detected, and the intersection area is marked as ID0=1, and other intersection areas are marked as 2, 3, and 4 in a counterclockwise order; the mark value ID1 of the intersection area where the trajectory end point of the target vehicle is located is detected; the mark value ID1 of the intersection where the trajectory end point is located and the mark value ID0 of the intersection area where the starting trajectory point is located are calculated as follows, and the driving direction of the target vehicle is determined according to the corresponding rules:
[0068] ID1-ID0=1, confirm that the vehicle turns right;
[0069] ID1-ID0=2, confirm that the vehicle is going straight;
[0070] ID1-ID0=3, confirm that the vehicle turns left;
[0071] In some embodiments, different preset algorithms may be used to determine the straight driving and turning of the vehicle.
[0072] In some embodiments, when determining whether the vehicle is traveling straight, the reference system includes a reference vector; analyzing the positional relationship of the multiple trajectory points according to a preset algorithm to determine the driving direction of the target vehicle includes: generating one or more trajectory point vectors based on the multiple trajectory points; calculating the angle associated with the one or more trajectory point vectors to obtain one or more trajectory point vector angles; comparing the one or more trajectory point vector angles with a preset angle threshold; in response to the one or more trajectory point angles being less than the preset angle threshold, determining that the target vehicle is traveling straight.
[0073] In some embodiments, when determining whether a vehicle is going straight, the reference system includes a first reference coordinate system; analyzing the positional relationship of the multiple trajectory points according to a preset algorithm to determine the driving direction of the target vehicle may also include: generating coordinates of a starting trajectory point and an end point of the trajectory; respectively calculating the difference between the x and y coordinate values of the starting trajectory point and the end point of the trajectory to obtain two differences; calculating the two differences and comparing the calculation result with a preset threshold; in response to the calculation result being greater than the preset threshold, determining that the target vehicle is going straight.
[0074] Specifically, the origin of the first reference coordinate system may be the center of the intersection, and the x-axis and y-axis of the first reference coordinate system may be respectively along the two intersecting road directions of the intersection. The coordinates of the starting trajectory point are (x0, y0), and the coordinates of the trajectory end point are (x1, y1). The difference is the difference Δx between the x coordinates of the starting trajectory point and the trajectory end point, and the difference Δy between the y coordinates, and the calculation formula is as follows:
[0075] Δx=x1-x0
[0076] Δy=y1-y0
[0077] The two differences may be calculated according to the following formula:
[0078] Δ=|Δx|-|Δy|
[0079] Where |Δx| is the absolute value of the difference Δx between the x-coordinates of the starting trajectory point and the trajectory end point, and |Δy| is the absolute value of the difference Δy between the y-coordinates of the starting trajectory point and the trajectory end point.
[0080] The preset threshold value may be a value determined based on the width of the intersection. For example, in the first reference coordinate system, the width of the intersection is 5, and the preset threshold value may be 5. When the calculated result Δ is greater than 5, it is determined that the target vehicle is traveling straight.
[0081] For more information about the algorithm for determining whether the target vehicle is going straight, please refer to Figure 5 Related instructions.
[0082] In some embodiments, after determining that the target vehicle is going straight, the method further includes counting the number of vehicles going straight. The number may be the number of vehicles going straight within a certain period of time, for example, the number of vehicles going straight within an hour. The number may also be the number within a certain period of time, for example, the number of vehicles going straight between 7 am and 9 am.
[0083] In some embodiments, when judging whether a vehicle turns left or right, the reference system includes a first reference coordinate system; generating a start-end point vector based on the multiple trajectory points; translating the start-end point vector so that the starting point of the start-end point vector coincides with the origin of the first reference coordinate system; determining the quadrant of the start-end point vector in the first reference coordinate system; and, determining the concavity of the target vehicle's driving trajectory relative to the straight line where the start-end point vector is located based on the multiple trajectory points; and determining whether the target vehicle is turning left or right based on the quadrant where the start-end point vector is located and the concavity of the driving trajectory.
[0084] In some embodiments, the vehicle turning direction may be determined after the vehicle is judged to be going straight. For example, after determining that the target vehicle is not going straight, it is further determined whether the target vehicle is turning left or right.
[0085] In some embodiments, the start-end point vector may be a vector connecting the start trajectory point and the trajectory end point, and the direction of the start-end point vector is from the start trajectory point to the trajectory end point. The translation may be any method applicable to vector translation. The first reference coordinate system is a rectangular coordinate system, and the quadrants of the first reference coordinate system include the first quadrant, the second quadrant, the third quadrant, and the fourth quadrant. The concavity of the trajectory means that if any arc segment on the trajectory is located below the chord, the trajectory is concave; if any arc segment on the trajectory is located above the chord, the trajectory is convex.
[0086] In some embodiments, determining whether the target vehicle is turning left or right based on the quadrant where the start and end point vectors are located and the concavity and convexity of the driving trajectory is specifically:
[0087] When the start and end point vectors are located in the first quadrant of the first reference coordinate system, if the trajectory is convex, it is determined that the target vehicle is turning right; if the trajectory is concave, it is determined that the target vehicle is turning left.
[0088] When the start and end point vectors are located in the second quadrant of the first reference coordinate system, if the trajectory is convex, it is determined that the target vehicle is turning left; if the trajectory is concave, it is determined that the target vehicle is turning right.
[0089] When the start and end point vectors are located in the third quadrant of the first reference coordinate system, if the trajectory is convex, it is determined that the target vehicle is turning left; if the trajectory is concave, it is determined that the target vehicle is turning right.
[0090] When the start and end point vectors are located in the fourth quadrant of the first reference coordinate system, if the trajectory is convex, it is determined that the target vehicle is turning right; if the trajectory is concave, it is determined that the target vehicle is turning left.
[0091] For more specific judgment process and rules, please refer to the Figure 6 and Figure 7 Related content.
[0092] In some embodiments, after determining that the target vehicle is turning left or right, the number of vehicles turning left or right may be counted. The number may be the number within a certain period of time, for example, the number of vehicles turning left or right within 1 hour. The number may also be the number within a certain time period, for example, the number of vehicles turning left or right between 7 am and 9 am; or the number of vehicles turning left or right between 2 pm and 6 pm.
[0093] Figure 3 is an exemplary sub-flow chart of coordinate transformation in step 206 according to some embodiments of the present invention.
[0094] Figure 4 is a schematic diagram of obtaining sampling points and their coordinate data in step 206 according to some embodiments of the present invention.
[0095] Step 302: Acquire multiple sampling points and their coordinate data in at least one image among multiple images.
[0096] In some embodiments, the multiple images may include deformed images, and the deformation includes but is not limited to one of tilt deformation, rotation deformation, shear deformation, etc. The sampling points may be multiple points selected by a user in at least one of the multiple images and their coordinate data in the image (such as the initial coordinate system).
[0097] See also Figure 4 , the acquiring of multiple sampling points and their coordinate data in at least one image among the multiple images specifically includes: receiving multiple annotation points 2 drawn by a user in at least one image among the multiple images; the multiple annotation points 2 are not collinear; generating a standard rectangle 3 based on the multiple annotation points 2; and determining vertices 4 of the standard rectangle 3 and their coordinate data in at least one image among the multiple images as the multiple sampling points and their coordinate data.
[0098] In some embodiments, the plurality of annotation points are located on a road marking line in at least one image among the plurality of images, and the plurality of annotation points form a quadrilateral.
[0099] The road markings may be pre-marked road boundary lines or pre-marked road lane lines. The quadrilaterals include but are not limited to rectangles, squares, rhombuses, parallelograms, trapezoids and other irregular quadrilaterals.
[0100] In some embodiments, the plurality of annotation points 2 may be manually annotated by a user. For example, a user may draw a plurality of annotation points 2 in one of the plurality of images through a user terminal. In some embodiments, the user's annotable range may be constrained. For example, the user may be constrained to draw points on road markings. For another example, the number of annotation points constrained to the user is 4.
[0101] In some embodiments, the multiple sides of the standard rectangle 3 may include all user-annotated points 2. The standard rectangle 3 may be the minimum circumscribed rectangle of multiple user-annotated points 2. At least one vertex 4 of the standard rectangle 3 coincides with at least one of the multiple annotated points 2.
[0102] In some embodiments, the user may draw multiple annotation points for each road area of the intersection, obtain a standard rectangle for each road area, and then obtain multiple sampling points and coordinate data of each road.
[0103] Step 304: Acquire multiple reference points and their coordinate data in the target coordinate system.
[0104] In some embodiments, the reference point may be a point corresponding to the sampling point in the target coordinate system. The coordinate data of the reference point is the coordinate data of the reference point in the target coordinate system.
[0105] Step 306: Establish a coordinate transformation matrix based on the coordinate data of the plurality of sampling points and the coordinate data of the plurality of reference points.
[0106] In some embodiments, the coordinate transformation matrix may be an affine matrix, and the process of establishing the coordinate transformation matrix may be based on performing affine transformation calculations on the multiple sampling points and the multiple reference points to obtain an affine matrix of the affine transformation. The affine transformation may include translation, scaling, flipping, rotation, and shearing.
[0107] Specifically, the affine matrix may be in the following form:
[0108]
[0109] Among them, (t x , t y ) represents the translation amount, a i (i=1 to 4) indicates changes such as rotation and scaling.
[0110] The sampling point and its coordinate data and the reference point and its coordinate data satisfy the following formula:
[0111]
[0112] in, is the matrix representation of the sampling point coordinate position, is a matrix representation of the coordinate position of the reference point. When enough sampling points and reference points are taken, the coordinate transformation matrix can be calculated. In some embodiments, the translation amount (t x , t y ) can be (0, 0), so the coordinate transformation matrix can be solved by taking 4 sampling points and their coordinate data and the corresponding 4 reference points and their coordinate data.
[0113] In some embodiments, an affine transformation matrix may be generated and pre-stored for each road based on multiple sampling points and coordinate data of each road, and the pre-stored affine transformation matrix may be applied to the coordinate transformation of the corresponding road area. For example, if the intersection is a crossroad, 4 affine transformation matrices may be generated and pre-stored for 4 roads, and the corresponding affine transformation matrix may be selected for processing according to the road the vehicle enters, thereby improving the detection accuracy.
[0114] Step 308 : Calculate the coordinate position of the target vehicle in the multiple images and the coordinate transformation matrix to obtain the multiple track points.
[0115] In some embodiments, the operation of the coordinate position of the target vehicle in the plurality of images with the coordinate transformation matrix may be multiplying the coordinate position of the target vehicle with the coordinate transformation matrix. The specific calculation may be in the following form:
[0116]
[0117] in is the matrix representation of the target vehicle’s coordinate position in the initial coordinate system, It is the matrix representation of the coordinate position of the target vehicle in the target coordinate system.
[0118] Figure 5 is an exemplary sub-flowchart for determining straight travel in step 106 according to some embodiments of the present invention.
[0119] Step 502: Generate one or more trajectory point vectors based on the plurality of trajectory points.
[0120] In some embodiments, the trajectory point vector may be a plurality of vectors formed by connecting a designated trajectory point with all other trajectory points. The designated trajectory point may be a starting trajectory point.
[0121] Step 504: calculate the angle associated with one or more trajectory point vectors to obtain the angle of one or more trajectory point vectors.
[0122] In some embodiments, the reference vector may be a vector established in the initial coordinate system or the target coordinate system for determining the direction or angle of the trajectory point vector. The reference vector may also be a vector established in the first reference coordinate system for determining the direction or angle of the trajectory point vector. For example, the reference vector may be a vector in the initial coordinate system or the target coordinate system or the first reference coordinate system that is parallel to the extension direction of the road along which the vehicle enters. The reference vector may also be a vector along the x-axis of one of the aforementioned coordinate systems, or a vector along the y-axis of one of the aforementioned coordinate systems.
[0123] In some embodiments, the trajectory point vector angle may be the angle between the trajectory point vector and the reference vector, the angle between the trajectory point vector and the unit vector along the x-axis, or the angle between the trajectory point vector and the unit vector along the y-axis. In some embodiments, the trajectory point vector angle may also be the angle between two trajectory point vectors in the trajectory point vector group with the maximum angle and the minimum angle with the reference vector. The angle between the two trajectory point vectors in the trajectory point vector with the maximum angle and the minimum angle with the reference vector may be calculated by the difference in angles between the two trajectory point vectors and the reference vector. In some embodiments, in order to improve the detection accuracy, the N trajectory point vectors with the largest angles with the reference vector and the M trajectory point vectors with the smallest angles with the reference vector may be filtered out simultaneously in the step of acquiring trajectory points (such as step 204), where N≥1 and M≥1.
[0124] Step 506: Compare the angle between the one or more trajectory point vectors and a preset angle threshold.
[0125] In some embodiments, the preset angle threshold may be an angle value set manually. In some embodiments, the preset angle threshold is 5°, 7°, 9°, 10°, etc. In some embodiments, the angle threshold is applicable to the judgment of the trajectory point vector angle of the target vehicle in all road areas of the intersection. In some embodiments, it can be determined whether the angle threshold needs to be adjusted based on whether the judgment result is consistent with the actual situation. Specifically, if the judgment result is consistent with the actual situation, there is no need to adjust the angle threshold; if the judgment result is inconsistent with the actual situation, the angle threshold should be increased or decreased accordingly. For example, if a turning vehicle is judged to be going straight, the angle threshold should be reduced. For another example, if a straight-moving vehicle is judged to be turning, the angle threshold should be increased.
[0126] Step 508: In response to the one or more trajectory point angles being less than a preset angle threshold, it is determined that the target vehicle is moving straight.
[0127] In some embodiments, each of the multiple trajectory point vector angles may be compared with an angle threshold, and if each trajectory point vector angle is less than the angle threshold, it is determined that the target vehicle is traveling straight; if any trajectory point vector angle is greater than the angle threshold, it is determined that the target vehicle is not traveling straight. In some embodiments, the maximum angle among the multiple trajectory point vector angles may also be compared with the angle threshold, and if the maximum angle is less than the angle threshold, it is determined that the target vehicle is traveling straight; if the maximum angle is greater than the angle threshold, it is determined that the target vehicle is not traveling straight.
[0128] Figure 6 is an exemplary sub-flowchart of determining a turning direction according to some embodiments of the present invention.
[0129] Figure 7 It is a schematic diagram of judgment rules for determining a turning direction according to some embodiments of the present invention.
[0130] Step 602: Generate a start and end point vector based on the multiple trajectory points.
[0131] In some embodiments, the start-end vector may be a vector connecting the start trajectory point and the trajectory end point, and the direction of the start-end vector is from the start trajectory point to the trajectory end point. Specifically, the coordinates of the start trajectory point are (x0, y0), the coordinates of the trajectory end point are (x1, y1), and the coordinate representation of the start-end vector may be in the following form: (x1-x0, y1-y0).
[0132] Step 604: determine the quadrant of the start and end point vectors in the first reference coordinate system.
[0133] The first reference coordinate system is a rectangular coordinate system, and the quadrants are respectively the first quadrant, the second quadrant, the third quadrant, and the fourth quadrant. In some embodiments, the quadrant of the start-end point vector in the first reference coordinate system can be understood as the quadrant in which the start-end point vector is located when the start-end point vector is translated so that the starting point of the start-end point vector coincides with the origin of the first reference coordinate system.
[0134] In some embodiments, the quadrant of the start and end point vectors in the first reference coordinate system may be determined by calculating the product of the start and end point vectors with unit vectors along two coordinate axes, and determining the quadrant of the start and end point vectors based on the positive and negative values of the product. The product may be a vector dot product.
[0135] More specifically, in some embodiments, the product value of the start and end point vector and the x-axis unit vector is value_x, and the product value of the start and end point vector and the y-axis unit vector is value_y. The specific rule for determining the quadrant of the start and end point vector based on the positive and negative values of the product is:
[0136] If value_x>0 and value_y<0, it is judged to be in the first quadrant.
[0137] value_x<0 and value_y<0, it is judged to be the second quadrant.
[0138] If value_x<0 and value_y>0, it is judged to be in the third quadrant.
[0139] If value_x>0 and value_y>0, it is judged to be in the fourth quadrant.
[0140] In some embodiments, the method of determining the quadrant where the start and end point vectors are located in the first reference coordinate system may also be as follows: output the coordinate representation (x, y) of the start and end point vectors, where x = x1-x0, y = y1-y0, (x0, y0) are the coordinates of the starting point of the trajectory, and (x1, y1) are the coordinates of the end point of the trajectory. The quadrant where the start and end point vectors are located is determined by the positivity of x and y. The judgment rule is:
[0141] If x>0 and y<0, it is judged to be a quadrant.
[0142] x<0 and y<0, it is judged to be the second quadrant.
[0143] x<0 and y>0, it is judged to be in the third quadrant.
[0144] If x>0 and y>0, it is judged to be in the fourth quadrant.
[0145] In some embodiments, the quadrant in which the start and end point vectors are located in the first reference coordinate system can be determined by translating the start and end point vectors in the first reference coordinate system. After the translation, the origin of the first reference coordinate system is the starting point of the start and end point vectors.
[0146] Output the end point coordinates (x′1, y′1) of the start and end point vectors after translation; determine the quadrant of the start and end point vectors by judging the positivity of the end point coordinates x′1, y′1. The judgment rule is:
[0147] x′1>0 and y′1<0, it is judged to be a quadrant.
[0148] x′1<0 and y′1<0, it is judged to be the second quadrant.
[0149] x′1<0 and y′1>0, it is judged to be in the third quadrant.
[0150] x′1>0 and y′1>0, it is judged to be the fourth quadrant.
[0151] Step 606, determining the concavity and convexity of the target vehicle's driving trajectory relative to the straight line where the start and end point vectors are located according to the multiple trajectory points;
[0152] The concavity of the trajectory means that if any arc segment on the trajectory is located below the stretched chord, the trajectory is concave; if any arc segment on the trajectory is located above the stretched chord, the trajectory is convex.
[0153] In some embodiments, determining the concavity and convexity of the vehicle driving trajectory based on the vehicle driving trajectory may specifically be as follows: Generate a linear equation based on the starting trajectory point and the ending trajectory point, such as y = ax + b, further obtain the function f(x) = ax + b, select the abscissa value of any one of the vehicle's multiple trajectory points except the starting trajectory point and the ending trajectory point and substitute it into the function, calculate the function value as the reference ordinate value corresponding to the trajectory point, compare the reference ordinate value with the ordinate value of the trajectory point, and determine the concavity and convexity of the vehicle driving trajectory based on the comparison result.
[0154] More specifically, the magnitudes of y' and y may be compared, where y' is the reference ordinate value and y is the ordinate value of the trajectory point, and the concavity and convexity of the trajectory are determined based on the comparison result. The specific rules are as follows:
[0155] If y > y', the trajectory is convex;
[0156] If y < y', the trajectory is concave.
[0157] In some embodiments, determining the concavity and convexity of the vehicle driving trajectory based on the vehicle driving trajectory may specifically also be as follows: Fit the trajectory points to generate a continuous curve; perform operations on the coordinates of any two points on the curve and compare the operation results; determine the concavity and convexity of the trajectory based on the comparison result.
[0158] More specifically, in some embodiments, the fitting may obtain the trajectory equation y = f(x) of the target vehicle driving trajectory. The fitting includes linear fitting and non-linear fitting, and the algorithms for fitting include but are not limited to the least squares algorithm, image thinning algorithm, parametric curve model algorithm, etc. The trajectory equation y = f(x) is continuous. The abscissas of any two points on the curve may be x1 and x2 respectively. The operation may be to calculate the function values based on the abscissa values of the two points and the function values The specific rule for determining the concavity and convexity of the trajectory based on the comparison result is as follows:
[0159] If then the trajectory f(x) is concave;
[0160] If then the trajectory f(x) is convex.
[0161] In some embodiments, determining the concavity and convexity of the vehicle driving trajectory based on the vehicle driving trajectory may specifically also be as follows: Plot the trajectory points to generate a continuous curve and generate the curve equation y = f(x); take the second derivative f″(x) of the function f(x); determine the concavity and convexity of the trajectory based on the positivity or negativity of the second derivative f″(x).
[0162] More specifically, the rule for determining the convexity of the trajectory based on the positive or negative nature of the second-order derivative f″(x) is:
[0163] If f″(x)>0, the trajectory is concave;
[0164] If f″(x)<0, then the trajectory is convex.
[0165] Step 608: Determine whether the target vehicle is turning left or right based on the quadrant where the start and end point vectors are located and the concavity of the driving trajectory.
[0166] See also Figure 7 , the determination of whether the target vehicle is turning left or right based on the quadrant where the start and end point vectors are located and the concavity and convexity of the driving trajectory is specifically:
[0167] When the start and end point vectors are located in the first quadrant of the first reference coordinate system, if the trajectory is convex, it is determined that the target vehicle is turning right; if the trajectory is concave, it is determined that the target vehicle is turning left.
[0168] When the start and end point vectors are located in the second quadrant of the first reference coordinate system, if the trajectory is convex, it is determined that the target vehicle is turning left; if the trajectory is concave, it is determined that the target vehicle is turning right.
[0169] When the start and end point vectors are located in the third quadrant of the first reference coordinate system, if the trajectory is convex, it is determined that the target vehicle is turning left; if the trajectory is concave, it is determined that the target vehicle is turning right.
[0170] When the start and end vectors are located in the fourth quadrant of the first reference coordinate system, if the trajectory is convex, the target vehicle is determined to be turning right; if the trajectory is concave, the target vehicle is determined to be turning left. It should be noted that the above description of process 600 is only for example and explanation, and does not limit the scope of application of one or more embodiments of this specification. For those skilled in the art, various modifications and changes can be made to process 600 under the guidance of one or more embodiments of this specification. However, these modifications and changes are still within the scope of one or more embodiments of this specification. For example, in some embodiments, the order of step 604 and step 606 can be exchanged. Such variations are within the protection scope of one or more embodiments of this specification.
[0171] Based on the above method for identifying the driving direction of a vehicle at an intersection, the present invention also provides a device for identifying the driving direction of a vehicle at an intersection, the device comprising at least one processor and at least one memory;
[0172] The at least one memory is used to store computer instructions;
[0173] The at least one processor is used to execute at least part of the computer instructions to implement the method for identifying the driving direction of a vehicle at an intersection as described in any of the above embodiments.
[0174] The at least one memory is used to store computer instructions. The memory may include a mass storage, a removable memory, a volatile read-write memory, a read-only memory (ROM), etc. or any combination thereof. Exemplary mass storage may include a magnetic disk, an optical disk, a solid-state disk, etc. Exemplary removable memory may include a flash drive, a floppy disk, an optical disk, a memory card, a compressed disk, a magnetic tape, etc. Exemplary volatile read-write memory may include a random access memory (RAM). Exemplary RAM may include a dynamic random access memory (DRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), a static random access memory (SRAM), a thyristor random access memory (T-RAM), and a zero capacitance random access memory (Z-RAM), etc. Exemplary ROM may include a mask read-only memory (MROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (PEROM), an electrically erasable programmable read-only memory (EEPROM), a compact disk read-only memory (CD-ROM), and a digital versatile disk read-only memory, etc.
[0175] The at least one processor is used to execute at least part of the computer instructions to implement the method for identifying the direction of vehicle travel at an intersection described in any of the above embodiments. The processor may include at least one hardware processor, such as a microcontroller, a microprocessor, a reduced instruction set computer (RISC), an application-specific integrated circuit (ASIC), an application-specific instruction set processor (ASIP), a central processing unit (CPU), a graphics processing unit (GPU), a physical processing unit (PPU), a microcontroller unit, a digital signal processor (DSP), a field programmable gate array (FPGA), an advanced RISC machine (ARM), a programmable logic device (PLD), any circuit or processor capable of performing at least one function, etc. or any combination thereof.
[0176] Based on the above method for identifying the direction of vehicle travel at an intersection, the present invention also provides a computer-readable storage medium, which stores computer instructions. When the computer reads the computer instructions in the storage medium, the computer executes the method for identifying the direction of vehicle travel at an intersection in any of the above embodiments.
[0177] The beneficial effects that may be brought about by the embodiments of the present invention include but are not limited to: (1) The method of the present invention is easy to process and simple to calculate; (2) The method of the present invention can adapt to different camera angles and can handle vehicle driving trajectories that are not "straight", and has a wider range of applicability; (3) The method of the present invention can simultaneously perform statistics on all directions of the intersection, and has low complexity. It should be noted that the beneficial effects that may be produced by different embodiments may be different. In different embodiments, the beneficial effects that may be produced may be any one or a combination of the above, or any other possible beneficial effects.
[0178] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only for example and does not constitute a limitation of the present invention. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements and corrections to the present invention. Such modifications, improvements and corrections are suggested in the present invention, so such modifications, improvements and corrections still belong to the spirit and scope of the exemplary embodiments of the present invention.
[0179] At the same time, the present invention uses specific words to describe the embodiments of the present invention. For example, "one embodiment", "an embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of the present invention. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more in different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures or characteristics in one or more embodiments of the present invention can be appropriately combined.
[0180] In addition, those skilled in the art will appreciate that various aspects of the present invention may be illustrated and described by a number of patentable categories or situations, including any new and useful process, machine, product or combination of substances, or any new and useful improvements thereto. Accordingly, various aspects of the present invention may be performed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software may all be referred to as "data blocks", "modules", "engines", "units", "components" or "systems". In addition, various aspects of the present invention may be represented as a computer product located in one or more computer-readable media, which includes computer-readable program code.
[0181] A computer storage medium may include a propagated data signal containing computer program code, for example, in baseband or as part of a carrier wave. The propagated signal may be in a variety of forms, including electromagnetic, optical, etc., or a suitable combination. A computer storage medium may be any computer-readable medium other than a computer-readable storage medium, which can be connected to an instruction execution system, device or apparatus to communicate, propagate or transmit the program for use. The program code on the computer storage medium may be transmitted via any suitable medium, including radio, cable, fiber optic cable, RF, or similar media, or any combination of the above media.
[0182] The computer program coding required for the operation of each part of the present invention can be written in any one or more programming languages, including object-oriented programming languages such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, Python, etc., conventional procedural programming languages such as C language, VisualBasic, Fortran2003, Perl, COBOL2002, PHP, ABAP, dynamic programming languages such as Python, Ruby and Groovy, or other programming languages, etc. The program coding can be run completely on the user's computer, or run on the user's computer as an independent software package, or run partly on the user's computer and partly on the remote computer, or run completely on the remote computer or processing device. In the latter case, the remote computer can be connected to the user's computer through any network form, such as a local area network (LAN) or a wide area network (WAN), or connected to an external computer (e.g., via the Internet), or in a cloud computing environment, or used as a service such as software as a service (SaaS).
[0183] In addition, unless explicitly stated in the claims, the order of the processing elements and sequences described in the present invention, the use of alphanumeric characters, or the use of other names are not intended to limit the order of the processes and methods of the present invention. Although the above disclosure discusses some of the invention embodiments currently considered useful through various examples, it should be understood that such details are only for illustrative purposes, and the attached claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the essence and scope of the embodiments of the present invention. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing processing device or mobile device.
[0184] Similarly, it should be noted that in order to simplify the description of the present invention and thus facilitate the understanding of one or more embodiments of the invention, in the above description of the embodiments of the invention, multiple features are sometimes combined into one embodiment, figure or description thereof. However, this disclosure method does not mean that the subject matter of the invention requires more features than those mentioned in the claims. In fact, the features of the embodiments are less than all the features of the single embodiment disclosed above.
[0185] In some embodiments, numbers describing the quantity of components and attributes are used. It should be understood that such numbers used in the description of the embodiments are modified by the modifiers "about", "approximately" or "substantially" in some examples. Unless otherwise specified, "about", "approximately" or "substantially" indicate that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may change according to the required features of individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and adopt the general method of retaining digits. Although the numerical domains and parameters used to confirm the breadth of the range in some embodiments of the present invention are approximate values, in specific embodiments, the setting of such numerical values is as accurate as possible within the feasible range.
[0186] Each patent, patent invention, patent invention disclosure, and other materials, such as articles, books, instructions, publications, documents, etc., cited in this invention are hereby incorporated by reference in their entirety. Except for invention history documents that are inconsistent with or conflicting with the content of this invention, documents that limit the broadest scope of the claims of this invention (currently or later attached to this invention) are also excluded. It should be noted that if the description, definition, and / or use of terms in the accompanying materials of this invention are inconsistent or conflicting with the content of this invention, the description, definition, and / or use of terms in this invention shall prevail.
[0187] Finally, it should be understood that the embodiments described in the present invention are only used to illustrate the principles of the embodiments of the present invention. Other variations may also fall within the scope of the present invention. Therefore, as an example and not a limitation, the alternative configurations of the embodiments of the present invention may be considered consistent with the teachings of the present invention. Accordingly, the embodiments of the present invention are not limited to the embodiments explicitly introduced and described in the present invention.
Claims
1. A method for identifying the moving direction of an object at an intersection. It is characterized in that The method comprises: Acquire a plurality of images of the intersection, at least one of the plurality of images including a target object; Based on the coordinate positions of the target object in the multiple images, a plurality of trajectory points on the motion trajectory of the target object are obtained; Analyze the positional relationship of the plurality of trajectory points according to a preset algorithm to determine the movement direction of the target object; Analyzing the positional relationship of the plurality of trajectory points according to a preset algorithm to determine the movement direction of the target object comprises: Generate multiple trajectory vectors based on the multiple trajectory points; the multiple trajectory points include a starting trajectory point and an ending trajectory point, the trajectory vector includes a trajectory point vector and a starting and ending point vector, the trajectory point vector is a multiple vector formed by connecting the starting trajectory point with other trajectory points, and the starting and ending point vector is a vector pointing from the starting trajectory point to the ending trajectory point; Determine at least one reference frame, wherein the reference frame is determined based on a road, a center of an intersection, or a lane line at the intersection; Determining a moving direction of the target object based on a relative position relationship between a plurality of trajectory vectors and at least one reference system; The reference system includes a reference vector; Analyzing the positional relationship of the plurality of trajectory points according to a preset algorithm to determine the movement direction of the target object includes: generating a plurality of trajectory point vectors based on the plurality of trajectory points; Angles associated with a plurality of trajectory point vectors are calculated to obtain a plurality of trajectory point vector angles; the plurality of trajectory point vector angles include angles between a plurality of trajectory point vectors and the reference vector or include a difference between a maximum angle and a minimum angle among the angles between a plurality of trajectory point vectors and the reference vector.
2. The method according to claim 1, It is characterized in that The plurality of images are taken from video data of the intersection.
3. The method according to claim 1, It is characterized in that The movement direction includes at least one of going straight, turning left or turning right.
4. The method according to claim 1, It is characterized in that The analyzing the positional relationship of the plurality of trajectory points according to a preset algorithm to determine the movement direction of the target object further includes: Comparing the angles of the plurality of trajectory point vectors with a preset angle threshold; In response to the fact that the angles of the plurality of trajectory point vectors are all smaller than a preset angle threshold, it is determined that the target object is moving in a straight line.
5. The method according to claim 4, It is characterized in that The reference vector reflects an extension direction of at least one entry road of the target object in the multiple images; the target object enters the intersection via the entry road.
6. The method according to claim 4, It is characterized in that The reference system includes a first reference coordinate system; In response to the angle between the plurality of trajectory points being not less than a preset angle threshold, the following steps are performed: Generate a start and end point vector based on the multiple trajectory points; Determine the quadrant of the start-end point vector in the first reference coordinate system; the quadrant of the start-end point vector in the first reference coordinate system is the quadrant where the start-end point vector is located when the start-end point vector is translated so that the starting point of the start-end point vector coincides with the origin of the first reference coordinate system; as well as, Determine the concavity and convexity of the motion trajectory of the target object relative to the straight line where the start and end point vectors are located according to the multiple trajectory points; Based on the quadrant where the start and end point vectors are located and the concavity and convexity of the motion trajectory, it is determined whether the target object is turning left or right.
7. The method according to claim 1, It is characterized in that The reference system includes a first reference coordinate system; Analyzing the positional relationship of the plurality of trajectory points according to a preset algorithm to determine the movement direction of the target object comprises: Generate a start and end point vector based on the multiple trajectory points; Determine the quadrant of the start-end point vector in the first reference coordinate system; the quadrant of the start-end point vector in the first reference coordinate system is the quadrant where the start-end point vector is located when the start-end point vector is translated so that the starting point of the start-end point vector coincides with the origin of the first reference coordinate system; as well as, Determine the concavity and convexity of the motion trajectory of the target object relative to the straight line where the start and end point vectors are located according to the multiple trajectory points; Based on the quadrant where the start and end point vectors are located and the concavity and convexity of the motion trajectory, it is determined whether the target object is turning left or right.
8. The method according to claim 6 or 7, It is characterized in that At least one coordinate axis of the first reference coordinate system reflects an extension direction of at least one road in the plurality of images.
9. The method according to claim 8, It is characterized in that At least one coordinate axis of the first reference coordinate system reflects an extension direction of at least one entry road of the target object in the plurality of images; the target object enters the intersection via the entry road.
10. The method according to claim 1, It is characterized in that The step of obtaining a plurality of trajectory points on the motion trajectory of the target object based on the coordinate position of the target object in the plurality of images further comprises: The coordinate positions of the target object in the multiple images are mapped to a target coordinate system to obtain the multiple track points; the extension direction of at least one road in the multiple images is parallel to a certain coordinate axis in the target coordinate system.
11. The method according to claim 10, It is characterized in that The step of obtaining a plurality of trajectory points on the motion trajectory of the target object based on the coordinate position of the target object in the plurality of images further comprises: Acquire a plurality of sampling points and coordinate data thereof in at least one image of the plurality of images; Acquire multiple reference points and their coordinate data in the target coordinate system; Establishing a coordinate transformation matrix based on the coordinate data of the plurality of sampling points and the coordinate data of the plurality of reference points; The coordinate positions of the target object in the multiple images are calculated with the coordinate transformation matrix to obtain the multiple track points.
12. The method according to claim 11, It is characterized in that The acquiring of a plurality of sampling points and coordinate data thereof in at least one image among the plurality of images comprises: receiving a plurality of annotation points drawn by a user in at least one image of the plurality of images; the plurality of annotation points are not collinear; generating a standard rectangle based on the plurality of annotation points; The vertices of the canonical rectangle and their coordinate data in at least one image of the multiple images are determined as the multiple sampling points and their coordinate data.
13. The method according to claim 12, It is characterized in that The plurality of marking points are located on a road marking line in at least one image among the plurality of images, and the plurality of marking points form a quadrilateral.
14. A system for identifying the direction of vehicle movement at an intersection. It is characterized in that The system comprises: An acquisition module, configured to acquire a plurality of images of the intersection, at least one of the plurality of images including a target object; A trajectory point determination module, used to obtain multiple trajectory points on the target object's motion trajectory based on the coordinate positions of the target object in multiple images; A processing module, configured to analyze the positional relationship of the plurality of trajectory points according to a preset algorithm to determine the movement direction of the target object; The processing module is specifically used to generate a plurality of trajectory vectors based on the plurality of trajectory points; the plurality of trajectory points include a starting trajectory point and an ending trajectory point, the trajectory vector includes a trajectory point vector and a starting and ending point vector, the trajectory point vector is a plurality of vectors formed by connecting the starting trajectory point with other trajectory points, and the starting and ending point vector is a vector pointing from the starting trajectory point to the ending trajectory point; determine at least one reference system, the reference system is determined based on the road, intersection center or lane line of the intersection; determine the movement direction of the target object based on the relative position relationship between the plurality of trajectory vectors and at least one reference system; The reference system includes a reference vector, and the processing module is further specifically used to generate multiple trajectory point vectors based on the multiple trajectory points; calculate the angles related to the multiple trajectory point vectors to obtain multiple trajectory point vector angles; the multiple trajectory point vector angles include the angles between the multiple trajectory point vectors and the reference vector or include the difference between the maximum angle and the minimum angle among the angles between the multiple trajectory point vectors and the reference vector.
15. A device for identifying the direction of vehicle movement at an intersection. It is characterized in that The apparatus comprises at least one processor and at least one memory; The at least one memory is used to store computer instructions; The at least one processor is used to execute at least part of the computer instructions to implement the operation of the method for identifying the movement direction of an object at an intersection as described in any one of claims 1 to 13.
16. A computer storable medium, It is characterized in that The storage medium stores computer instructions. When at least part of the computer instructions are executed by the processor, the operation of the method for identifying the moving direction of an object at an intersection as claimed in any one of claims 1 to 13 is implemented.
Citation Information
Patent Citations
Method for judging behavior patterns of vehicles in video stream
CN102176285A
Method and device for disappeared direction determination and method and device for camera self-calibration
CN102789642A
Learning and anomaly detection method based on multi-feature motion modes of vehicle traces
CN103605362A
Method used for identifying object motion directions and equipment
CN105225248A
Urban intersection automatic identification method based on low precision space-time trajectory data
CN105788273A
Cited By
Determining a set of geographic position traces to be used to produce a digital map of a region of interest
US20250244139A1
