Method and device for acquiring pedestrian trajectory in port scene based on computer vision

By installing camera equipment on the lamp pole in the port area to obtain video stream images, perform perspective transformation and rectangular frame processing, the problem of large error in object trajectory calculation in traditional systems is solved, and the accuracy of trajectory calculation is improved.

CN118887260AActive Publication Date: 2024-11-01NINGBO PORT INFORMATION COMM CO LTD
View PDF 10 Cites 0 Cited by

Patent Information

Application Number
CN202411388305.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-08
Publication Date
2024-11-01
Estimated Expiration
2044-10-08

AI Technical Summary

Technical Problem

In traditional human-vehicle collision prevention systems, the tracking trajectory calculation error of objects is large, and the existing technology has failed to effectively solve this problem.

Method used

By installing camera equipment on the lamp pole in the port area to obtain video stream images, determine the pedestrian profile and the minimum external rectangle, perform perspective transformation, select the bottom edge of the rectangle and make a vertical line, obtain the third minimum external rectangle frame in the top view, thereby Determine the pedestrian trajectory and eliminate the view distance deviation caused by the perspective effect of the camera's camera's near, far and small.

Benefits of technology

It improves the accuracy of object tracking trajectory calculation, reduces errors, and achieves more accurate pedestrian trajectory acquisition.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118887260A_ABST
    Figure CN118887260A_ABST
Patent Text Reader

Abstract

The invention discloses a port scene pedestrian trajectory acquisition method and device based on computer vision, and relates to the field of image processing. The method comprises the following steps: acquiring a video frame image in a video stream image of camera equipment mounted on a lamp post of a harbor district; determining a pedestrian contour in each video frame image and a first minimum enclosing rectangle corresponding to the pedestrian contour; performing perspective transformation on the first minimum enclosing rectangle to obtain a second minimum enclosing rectangle under a top view corresponding to each video frame image; the bottom edge of the second minimum enclosing rectangle is selected, vertical lines are drawn towards the other rectangle edge, parallel to the bottom edge, in the second minimum enclosing rectangle through the two end points of the bottom edge, and the length of the vertical lines is equal to that of the bottom edge, so that a third minimum enclosing rectangle frame is obtained; determining a pedestrian trajectory according to the third minimum bounding rectangular frame of each frame in the video frame image; the problem of large object tracking trajectory calculation error is solved, and the object tracking trajectory calculation accuracy is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method, device, electronic device and storage medium for acquiring pedestrian trajectories in port scenes using computer vision. Background Art

[0002] With the increase in transportation volume year by year, the safety management problem of the port area has become increasingly prominent. The port area is often densely populated with people, vehicles and goods. Some people and vehicles may not comply with safety regulations, such as people and vehicles entering prohibited areas, which may cause people and vehicles to get too close, and even cause safety accidents of collisions between people and vehicles. Therefore, efficient safety management measures to prevent collisions between people and vehicles are very important. Using security personnel to conduct on-site inspections or watch video surveillance is a management method to detect safety hazards, but this manual detection method is not efficient. In recent years, video analysis technology based on computer vision has developed rapidly and is widely used in various fields including security. It has the advantages of high efficiency and good real-time performance. It can monitor and analyze multiple areas equipped with surveillance 24 hours a day to determine the possibility of collisions between people and vehicles in the picture.

[0003] Traditional systems for preventing collisions between people and vehicles require the installation of radar or binocular cameras on each vehicle, which is achieved through radar 3D modeling or binocular ranging technology. These methods require the installation of radar or binocular cameras and corresponding computer equipment on each vehicle, which is costly. In addition, as the vehicle moves, the images obtained will have distance deviations in the view, that is, images that are closer appear larger and images that are farther away appear smaller, resulting in low accuracy of the obtained results, which in turn leads to large errors in the calculation of the object's tracking trajectory.

[0004] Currently, no effective solution has been proposed for the problem of large errors in object tracking trajectory calculation in related technologies. Summary of the invention

[0005] The embodiments of the present application provide a method, device, electronic device and storage medium for acquiring pedestrian trajectories in a port scene using computer vision, so as to at least solve the problem of large calculation errors in tracking trajectories of objects in the related art.

[0006] In a first aspect, an embodiment of the present application provides a method for acquiring pedestrian trajectories in a port scene based on computer vision, comprising: Acquire a video frame image from a video stream image of a camera device installed on a lamp pole in a port area; Determine a pedestrian outline in each frame of the video frame image and a first minimum circumscribed rectangle corresponding to the pedestrian outline; Performing perspective transformation on the first minimum circumscribed rectangle to obtain a second minimum circumscribed rectangle in a top view corresponding to each frame of the video frame image; Select the bottom side of the second minimum circumscribed rectangle, and draw perpendicular lines through the two end points of the bottom side to another rectangular side of the second minimum circumscribed rectangle that is parallel to the bottom side, and the lengths of the perpendicular lines are equal to that of the bottom side, so as to obtain a third minimum circumscribed rectangle frame; The pedestrian trajectory is determined according to the third minimum circumscribed rectangular frame of each frame in the video frame image.

[0007] In a second aspect, an embodiment of the present application provides a device for acquiring pedestrian trajectories in a port scene using computer vision, comprising: An acquisition module, used for acquiring a video frame image in a video stream image of a camera device installed on a lamp pole in the port area; A first determination module is used to determine the pedestrian outline in each frame of the video frame image and the first minimum circumscribed rectangle corresponding to the pedestrian outline; A transformation module, used for performing perspective transformation on the first minimum circumscribed rectangle to obtain a second minimum circumscribed rectangle in a top view corresponding to each frame of the video frame image; A processing module, configured to select the bottom side of the second minimum circumscribed rectangle, and draw perpendicular lines through two end points of the bottom side to another rectangular side in the second minimum circumscribed rectangle that is parallel to the bottom side, and the lengths of the perpendicular lines are equal to that of the bottom side, so as to obtain a third minimum circumscribed rectangular frame; The second determination module is used to determine the pedestrian trajectory according to the third minimum circumscribed rectangular frame of each frame in the video frame image.

[0008] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for acquiring pedestrian trajectories in port scenes based on computer vision as described in the first aspect above is implemented.

[0009] In a fourth aspect, an embodiment of the present application provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for acquiring pedestrian trajectories in a port scene using computer vision as described in the first aspect above.

[0010] Compared with the related art, the computer vision method, device, electronic device and storage medium for obtaining pedestrian trajectories in port scenes provided in the embodiments of the present application obtain video frame images in the video stream images of the camera equipment installed on the lamp poles in the port area; determine the pedestrian contour in each frame of the video frame image and the first minimum bounding rectangle corresponding to the pedestrian contour; perform perspective transformation on the first minimum bounding rectangle to obtain the second minimum bounding rectangle under the top view corresponding to each frame of the video frame image; select the base of the second minimum bounding rectangle, and make perpendicular lines through the two endpoints of the base to another rectangular side parallel to the base in the second minimum bounding rectangle, and the length of the perpendicular line is equal to that of the base, so as to obtain a third minimum bounding rectangle frame; determine the pedestrian trajectory according to the third minimum bounding rectangle frame of each frame in the video frame image, which solves the problem of large error in the calculation of the tracking trajectory of objects in the related art and improves the accuracy of the calculation of the tracking trajectory of objects.

[0011] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 It is a hardware structure block diagram of a terminal of a method for acquiring pedestrian trajectories in a port scene using computer vision according to an embodiment of the present invention; Figure 2 is a flowchart of a method for acquiring pedestrian trajectories in a port scene using computer vision according to an embodiment of the present application; Figure 3 is a schematic diagram of the outline, horizontal bounding rectangle and minimum bounding rectangle of a pedestrian video frame in a port scene according to computer vision of an embodiment of the present application; Figure 4 is a schematic diagram of selecting the bottom edge of the third minimum circumscribed rectangular frame in a top view according to an embodiment of the present application; Figure 5 It is a structural block diagram of a device for acquiring pedestrian trajectories in a port scene using computer vision according to an embodiment of the present application. DETAILED DESCRIPTION

[0013] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application. In addition, it can also be understood that although the efforts made in this development process may be complex and lengthy, for ordinary technicians in the field related to the contents disclosed in the present application, some changes such as design, manufacturing or production based on the technical contents disclosed in the present application are only conventional technical means, and should not be understood as insufficient contents disclosed in the present application.

[0014] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0015] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantity limitation, and may indicate the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application means greater than or equal to two. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. The terms "first", "second", "third" and the like involved in the present application are merely used to distinguish similar objects and do not represent a specific ordering of the objects.

[0016] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. Taking running on a terminal as an example, Figure 1 1 is a hardware structure block diagram of a terminal of a method for acquiring pedestrian trajectories in a port scene using computer vision according to an embodiment of the present invention. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0017] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for obtaining pedestrian trajectories in a port scene of computer vision in an embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0018] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0019] This embodiment provides a method for obtaining pedestrian trajectories in a port scene using computer vision. Figure 2 is a flowchart of a method for acquiring pedestrian trajectories in a port scene using computer vision according to an embodiment of the present application. Figure 2 As shown, the process includes the following steps: Step S201: acquiring a video frame image in a video stream image of a camera device installed on a lamp pole in a port area.

[0020] In this step, the images used are taken by camera equipment of lamp poles and / or lighthouses in the port area, and the camera equipment may shoot at a bird's-eye view.

[0021] Step S202 : determining a pedestrian outline in each video frame image and a first minimum circumscribed rectangle corresponding to the pedestrian outline.

[0022] In this step, determining the pedestrian outline in the video frame image can be achieved through image recognition, deep learning models and other methods.

[0023] Step S203 : performing perspective transformation on the first minimum bounding rectangle to obtain a second minimum bounding rectangle in the top view corresponding to each video frame image.

[0024] Step S204, select the bottom side of the second minimum circumscribed rectangle, and draw perpendicular lines through the two end points of the bottom side to another rectangular side of the second minimum circumscribed rectangle parallel to the bottom side, and the lengths of the perpendicular lines are equal to the bottom side, so as to obtain a third minimum circumscribed rectangle.

[0025] In steps S203 to S204, the pedestrian tracking trajectory is determined in a bird's-eye view through perspective transformation, which eliminates the distance deviation in the view caused by the perspective effect of the camera device where objects are larger near and smaller far away, and can make the judgment result more accurate.

[0026] Step S205 , determining the pedestrian trajectory according to the third minimum circumscribed rectangular frame of each frame in the video frame image.

[0027] Through the above steps S201 to S205, the top view is converted through perspective transformation, the minimum circumscribed rectangular frame of the pedestrian is obtained on the top view, and the final tracking trajectory is determined based on the minimum circumscribed rectangular frame on the top view, thereby eliminating the distance deviation in the view caused by the perspective effect of the camera where things appear larger when near and smaller when far away, solving the problem of large error in the calculation of the tracking trajectory of objects in the related art, and improving the accuracy of the calculation of the tracking trajectory of objects.

[0028] In some of the embodiments, determining the outline of a pedestrian in each frame of video frame image and the first minimum bounding rectangle corresponding to the pedestrian outline includes: using a deep learning algorithm to segment the video frame image to obtain the outline of the pedestrian in the video frame image; determining the horizontal bounding rectangle corresponding to the pedestrian based on the pedestrian outline; tracking the horizontal bounding rectangle to obtain the first minimum bounding rectangle corresponding to the pedestrian.

[0029] In this embodiment, a deep learning algorithm is used to segment the video frame image to obtain the pedestrian contour in the video frame image, including: inputting the video frame image to the input end of the yolov8 segmentation network; obtaining the contour coordinate points and contour categories from the output end of the yolov8 segmentation network; and determining the pedestrian contour according to the contour coordinate points and contour categories.

[0030] The following is a detailed description and explanation of the above embodiment for determining the pedestrian contour in each video frame image and the first minimum circumscribed rectangle corresponding to the pedestrian contour.

[0031] Use yolov8 to segment each frame of the image, filter out pedestrians, and extract the outline of pedestrians. Figure 3 In the Q graph, each set of pedestrian contour points can be represented by an array [L1, L2, L3, ..., L n ] indicates that Ln indicates the nth point on the contour, (L n .x, L n .y) represents the coordinates of the nth point on the contour.

[0032] According to the target contour array [L1, L2, L3, ..., L n ] Calculate the horizontal bounding rectangle and minimum bounding rectangle of the target. The horizontal bounding rectangle is used for pedestrian tracking in video frames to determine which specific target the minimum bounding rectangle of multiple consecutive frames belongs to. The minimum bounding rectangle is used to transform the tracking box under the top view image after perspective transformation. The four coordinate points of the minimum bounding rectangle of the video frame are [R min1 , R min2 , R min3 , R min4 ] indicates that (R min.x , R min.y ) represents the x,y coordinates of the minimum enclosing rectangle.

[0033] The specific steps are as follows: Converting an outline to a horizontal bounding rectangle includes the following methods: 1.1 Initialize the variables min_x and max_x to the x coordinates of the first point L1 in the contour (L1.x, L1.y).

[0034] 1.2 Traverse each point Ln on the contour and obtain the coordinates (L n .x, L n .y).

[0035] 1.3 Update min_x to the L of the currently traversed point n .The smaller value of the x coordinate and min_x, update max_x to the L of the currently traversed point n The larger of the x coordinate and max_x.

[0036] 1.4 Update min_y to the smaller value of the Ln.y coordinate of the currently traversed point and min_y, and update max_y to the Ln.y coordinate of the currently traversed point n The larger of the y coordinate and max_y.

[0037] The coordinates of the four points of the final horizontal circumscribed rectangle are (min_x, min_y), (max_x, min_y), (max_x,max_y), (min_x, max_y), and the final horizontal circumscribed rectangle is as follows Figure 3 The T graph in .

[0038] Converting an outline to a minimum enclosing rectangle can be done in the following ways: 2.1 Calculate the convex hull of the contour and use the Graham scanning method to calculate the convex hull.

[0039] 2.1.1 Find the contour array [L1, L2, L3, ..., L n ]The point with the smallest y coordinate. If there are multiple points with the same y coordinate, the point with the smallest x coordinate is selected. This point is the first point H1 of the convex hull.

[0040] 2.1.2 Calculate the polar angles from all points in the contour array to the first point H1, and store the points and polar angles together in a new array [(L1,θ1), (L2,θ2), (L3,θ3), ..., (L n-1 ,θ n-1 ), (L n ,θ n )].

[0041] 2.1.3 For the array [(L1,θ1), (L2,θ2), (L3,θ3), … , (L n-1 ,θ n-1 ), (L n ,θ n )], according to the polar angle θ x Sort the points in ascending order. Assume that the array of sorted contour points is (M1, M2, M3, …, M n-1 ,M n ), M1 is the array [(L1,θ1), (L2,θ2), (L3,θ3), … , (L n-1 ,θ n-1 ), (L n ,θ n )] is the point with the smallest θx value, M n is theta in the array x The point with the largest value.

[0042] 2.1.4 Use a stack to store the points on the convex hull. n-1 ,M n ], push the first two points M1 and M2 into the stack, and the stack becomes [M1, M2], with the lightest and the heaviest M2 as the top of the stack. Then, traverse the sorted point set starting from M3.

[0043] 2.1.5 For the currently traversed point M i (i>=3), connect the stack vertex M k (k>=2) and M1, we get the straight line L: Then judge M i (Coordinates are (M i .x,M i On which side of line L is y)? , then M i On the right side of line L, , then M i On the left side of line L. If M i On the right side of line L, point M at the top of the stack is shown k is a depression, M k should be deleted. At this time, the top point M k Pop the stack and repeat this step until the top point of the stack is on the left side of line L. Then, place M i Push onto the stack.

[0044] 2.1.6 After all points have been checked, the remaining points in the stack are the points on the convex hull. Move the points in the stack into the array. The final result is recorded as [K1, K2, K3, ..., K n-1 , K n ].

[0045] 2.2 Solving the minimum enclosing rectangle using the convex hull method includes the following methods: 2.2.1 Traverse all edges K of the convex hull m K m+1 For each edge, try to find the point that is perpendicular to this edge and farthest from this edge, assuming it is point K x . Through the edge K m K m+1 and K x Construct four points that enclose the convex hull rectangle [R m1 , R m2 ,R m3 , R m4 ], and record the area Sm of the rectangle. After traversing all the edges of the convex hull, we have an array [ [R 11 ,R 12, R13, R 14 ], [ [R 11 , R 12, R 13, R 14 ], …… ,[R m1 , R m2 , R m3 , R m4 ] ] and the corresponding array of rectangular areas [S1, S2, S3, …, S m-1 , S m ].

[0046] 2.2.2 Find [S1, S2, S3, …, S m-1 , S m ] is the area with the smallest value S min and the corresponding rectangle [R min1 , R min2 , R min3 , R min4 ], which is the minimum enclosing rectangle. Figure 3 The W graphic in .

[0047] In the above embodiment, the specific process of perspective transformation may include: Set two sets of parallel lines on the video frame. These two sets of parallel lines must be perpendicular to each other in real physical space. These two sets of parallel lines will have 4 intersection points A, B, C, D in the image.

[0048] The horizontal circumscribed rectangle EFGH of quadrilateral ABCD, points A, B, and C correspond to points E, F, and G respectively. Calculate the perspective transformation matrix of quadrilateral ABCD to rectangle EFGH. Assume that the coordinates of A, B, and C are ,(X b ,Y b ), (X c ,Y c ), the coordinates of E, F, and G are ,(X f ,Y f ), (X g ,Y g ), then the matrix composed of the vectors of the three vertices A, B, and C is: ; E, F, G The matrix composed of the vectors of the three vertices is: ;

[0049] Then the perspective transformation matrix M for: .

[0050] like Figure 4 As shown, find the minimum bounding rectangle of the target to be tracked in each frame [ R min1 , R min2 , R min3 , R min4 ] Among the four points y The point with the largest value is assumed to be R min1 , with this point as the endpoint, there are two adjacent edges R mini R min(i-1) , R mini , R min(i+1) , we choose x The side with the smaller angle between the axes (the angle here is an acute angle, if the angle is an obtuse angle, its supplementary angle needs to be calculated), assuming it is R mini R min(i+1) , perform perspective transformation on the two points of the edge, , With this side as the base, draw a vertical line in the negative direction of the y-axis coordinate. In this embodiment, the target is a pedestrian, and the length of the vertical line is The steps to calculate the center point of the pedestrian's tracking box are as follows: Calculation Point and Point The center point T c , the calculation formula is as follows: ; ; Calculate the length of the vertical line, the length of the vertical line of the pedestrian dis human equal , the calculation formula is as follows: ; To calculate the arc of the perpendicular line, you can use the two-point formula to solve the arc. The calculation formula is: ; Calculate the end point coordinates of the vertical line: The end point coordinates of the pedestrian vertical line are V human The calculation formula is: ; In some of the embodiments, determining the pedestrian trajectory according to the third minimum circumscribed rectangular frame of each frame in the video frame image includes: determining the center point of the third minimum circumscribed rectangular frame of each frame; determining the pedestrian trajectory according to the center point of the third minimum circumscribed rectangular frame of each frame.

[0051] In this embodiment, based on the above embodiment, the center point of the third minimum circumscribed rectangular frame of each frame may be determined in the following manner: After calculating the perspective transformation, the coordinates of the center point of the pedestrian tracking rectangle are ( ), the calculation formula is: ; The center point of the pedestrian's tracking frame in each frame after perspective transformation is retained as the tracking trajectory. The tracking trajectory of the pedestrian after perspective transformation is uniformly represented as an array [E center1 , E center2 , E center3 , ……, E center(n-1) , E center(n) ], and record the time of each trajectory point [ T 1 , T 2 , T 3 , ……, T (n-1) , T n ].

[0052] In some of the embodiments, before performing preset straight line fitting on the pedestrian trajectory and determining the predicted trajectory corresponding to the pedestrian trajectory, the pedestrian trajectory may be smoothed.

[0053] In this embodiment, the specific method of smoothing may be: Use Gaussian filtering to track the trajectory array [ E center1 , E center2 , E center3 , ……, E center(n-1) , E center(n) ] to smooth and form a new coordinate array [ E g1 , E g2 , E g2 , ……, E gn-1, E gn ], where the Gaussian function is , using the Gaussian function, we get the normalized Gaussian filter kernel [0.319, 0.362, 0.319]. In the array, the filter formula for the coordinate point with subscript i is: .

[0054] In some of the embodiments, after determining the pedestrian trajectory based on the third minimum circumscribed rectangular frame of each frame in the video frame image, the method also includes: performing a preset straight line fitting on the pedestrian trajectory to determine a predicted trajectory corresponding to the pedestrian trajectory; judging whether the pedestrian trajectory is at risk of collision based on the predicted trajectory; and issuing an alarm if it is judged that the pedestrian trajectory is at risk of collision.

[0055] In this embodiment, the preset straight line fitting includes at least one of the following: a first straight line fitting and a second straight line fitting.

[0056] The specific preset straight line fitting in the above embodiment can be in the following manner: Take the most recent 100 trajectory points [ E g(n-99) , E g(n-98) , ……, E gn-1 , E gn ] and the corresponding time to record the time of each trajectory point [ T (n-99) , T (n-98) ,……, T (n-1) , T n ], and fit the array with a straight line function and a quadratic curve function. The straight line function fitting formula is: , the quadratic curve fitting function is Using the least squares method, we can calculate a , b , c , d , e The value of: ; ; ; ; ; Calculate the fitted linear array [ E l(n-99), E l(n-98) , ……, E l(n-1) , E ln ] and quadratic linear array [ E d(n-99) , E d(n-98) , ……, E d(n-1) , E dn ] and the original real array [ E g(n-99) , E g(n-98) , ……, E g(n-1) , E gn ]exist y Error in coordinates Difference 1 and Difference 2 : ; ; if Difference 1 > Difference 2 , then select the quadratic curve fitting trajectory, if Difference 1 < Difference 2 , then select the linear function fitting trajectory.

[0057] Based on the fitted trajectory, it is predicted whether the pedestrian’s target tracking box will intersect after perspective transformation at the next time T.

[0058] Assume that the fitting trajectory of a pedestrian in the image is y=f(x) , f(x) The form can be or The above tracking point E g(n-1) Go to current tracking point E gn The speed is used as the speed of the predicted trajectory at the next moment. The speed calculation formula is: ;

[0059] The moving distance at the next time T is .

[0060] (1) Assume that the trajectory of the target is , the current tracking point coordinates are (E gn .x, ), calculate the next moment E g(n+1) .x and E g(n+1) .y: ; .

[0061] (2) Assume that the trajectory of the target is , the current tracking point coordinates are ( E gn .x, ), calculate the next moment E g(n+1) .x and E g(n+1) .y: beg The derivative of : ; ; ; Point ( E g(n+1) .x, f(E g(n+1) .x) ) is the center of the circle, is the diameter, forming a circle. Determine each trajectory in ( E g(n+1) .x, f(E g(n+1) .x) ) Whether there is an intersection between the circular areas at the center points.

[0062] If so, it is considered that there will be human-machine crosstalk, otherwise it is considered that there will be no risk of human-machine crosstalk.

[0063] This embodiment also provides a computer vision device for acquiring pedestrian trajectories in a port scene, which is used to implement the above embodiments and preferred implementations, and will not be repeated here. As used below, the terms "module", "unit", "subunit", etc. can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0064] Figure 5 is a structural block diagram of a device for acquiring pedestrian trajectories in a port scene using computer vision according to an embodiment of the present application, such as Figure 5 As shown, the device comprises: An acquisition module 51 is used to acquire a video frame image in a video stream image of a camera device installed on a lamp pole in the port area; A first determination module 52, configured to determine a pedestrian outline in each video frame image and a first minimum circumscribed rectangle corresponding to the pedestrian outline; A transformation module 53, configured to perform perspective transformation on the first minimum bounding rectangle to obtain a second minimum bounding rectangle in a top view corresponding to each frame of the video frame image; A processing module 54 is used to select the bottom side of the second minimum circumscribed rectangle, and draw perpendicular lines through two end points of the bottom side to another rectangular side of the second minimum circumscribed rectangle that is parallel to the bottom side, and the lengths of the perpendicular lines are equal to that of the bottom side, so as to obtain a third minimum circumscribed rectangle; The second determination module 55 is used to determine the pedestrian trajectory according to the third minimum circumscribed rectangular frame of each frame in the video frame image.

[0065] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0066] This embodiment further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0067] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0068] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program: Step S201: acquiring a video frame image in a video stream image of a camera device installed on a lamp pole in a port area.

[0069] Step S202 : determining a pedestrian outline in each video frame image and a first minimum circumscribed rectangle corresponding to the pedestrian outline.

[0070] Step S203 : performing perspective transformation on the first minimum bounding rectangle to obtain a second minimum bounding rectangle in the top view corresponding to each video frame image.

[0071] Step S204, select the bottom side of the second minimum circumscribed rectangle, and draw perpendicular lines through the two end points of the bottom side to another rectangular side of the second minimum circumscribed rectangle parallel to the bottom side, and the lengths of the perpendicular lines are equal to the bottom side, so as to obtain a third minimum circumscribed rectangle.

[0072] Step S205 , determining the pedestrian trajectory according to the third minimum circumscribed rectangular frame of each frame in the video frame image.

[0073] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0074] In addition, in combination with the above-mentioned method for obtaining pedestrian trajectories in a port scene using computer vision, the present application embodiment may provide a storage medium for implementation. The storage medium stores a computer program; when the computer program is executed by a processor, any of the above-mentioned methods for obtaining pedestrian trajectories in a port scene using computer vision is implemented.

[0075] Those skilled in the art should understand that the technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0076] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for obtaining pedestrian trajectories in port scenes based on computer vision. It is characterized in that include: Acquire a video frame image from a video stream image of a camera device installed on a lamp pole in a port area; Determine a pedestrian outline in each frame of the video frame image and a first minimum circumscribed rectangle corresponding to the pedestrian outline; Performing perspective transformation on the first minimum circumscribed rectangle to obtain a second minimum circumscribed rectangle in a top view corresponding to each frame of the video frame image; Select the bottom side of the second minimum circumscribed rectangle, and draw perpendicular lines through the two end points of the bottom side to another rectangular side of the second minimum circumscribed rectangle that is parallel to the bottom side, and the lengths of the perpendicular lines are equal to that of the bottom side, so as to obtain a third minimum circumscribed rectangle frame; The pedestrian trajectory is determined according to the third minimum circumscribed rectangular frame of each frame in the video frame image.

2. According to the method for obtaining pedestrian trajectories in port scenes based on computer vision according to claim 1, It is characterized in that Determining the pedestrian outline in each frame of the video frame image and the first minimum bounding rectangle corresponding to the pedestrian outline includes: Segmenting the video frame image using a deep learning algorithm to obtain pedestrian contours in the video frame image; Determine a horizontal circumscribed rectangle corresponding to the pedestrian according to the pedestrian outline; The horizontal circumscribed rectangle is tracked to obtain a first minimum circumscribed rectangle corresponding to the pedestrian.

3. According to the method for obtaining pedestrian trajectories in port scenes based on computer vision in claim 2, It is characterized in that The video frame image is segmented by using a deep learning algorithm to obtain the pedestrian contour in the video frame image, including: Input the video frame image to the input end of the yolov8 segmentation network; Obtain contour coordinate points and contour categories from the output end of the yolov8 segmentation network; The pedestrian contour is determined according to the contour coordinate points and the contour category.

4. According to the method for obtaining pedestrian trajectories in port scenes based on computer vision in claim 1, It is characterized in that Determining the pedestrian trajectory according to the third minimum circumscribed rectangular frame of each frame in the video frame image includes: Determine the center point of the third minimum circumscribed rectangular frame of each frame; The pedestrian trajectory is determined according to the center point of the third minimum circumscribed rectangular frame of each frame.

5. According to the method for obtaining pedestrian trajectories in port scenes based on computer vision in claim 1, It is characterized in that After determining the pedestrian trajectory according to the third minimum circumscribed rectangular frame of each frame in the video frame image, the method further includes: Performing a preset straight line fitting on the pedestrian trajectory to determine a predicted trajectory corresponding to the pedestrian trajectory; According to the predicted trajectory, determining whether the pedestrian trajectory has a collision risk; When it is determined that there is a risk of collision with the pedestrian's trajectory, an alarm is issued.

6. According to the method for obtaining pedestrian trajectories in port scenes based on computer vision according to claim 5, It is characterized in that The preset straight line fitting includes at least one of the following: a first straight line fitting and a second straight line fitting.

7. According to the method for obtaining pedestrian trajectories in port scenes based on computer vision in claim 5, It is characterized in that Before performing a preset straight line fitting on the pedestrian trajectory to determine a predicted trajectory corresponding to the pedestrian trajectory, the method further includes: The pedestrian trajectory is smoothed.

8. A computer vision device for acquiring pedestrian trajectories in port scenes. It is characterized in that include: An acquisition module, used for acquiring a video frame image in a video stream image of a camera device installed on a lamp pole in the port area; A first determination module, used to determine the pedestrian outline in each frame of the video frame image and a first minimum circumscribed rectangle corresponding to the pedestrian outline; A transformation module, used for performing perspective transformation on the first minimum circumscribed rectangle to obtain a second minimum circumscribed rectangle in a top view corresponding to each frame of the video frame image; A processing module, configured to select the bottom side of the second minimum circumscribed rectangle, and draw perpendicular lines through two end points of the bottom side to another rectangular side in the second minimum circumscribed rectangle that is parallel to the bottom side, and the lengths of the perpendicular lines are equal to that of the bottom side, so as to obtain a third minimum circumscribed rectangular frame; The second determination module is used to determine the pedestrian trajectory according to the third minimum circumscribed rectangular frame of each frame in the video frame image.

9. An electronic device comprising a memory and a processor, It is characterized in that The memory stores a computer program, and the processor is configured to run the computer program to execute the method for acquiring pedestrian trajectories in a port scene using computer vision according to any one of claims 1 to 7.

10. A storage medium, It is characterized in that The storage medium stores a computer program, wherein the computer program is configured to execute the method for acquiring pedestrian trajectories in a port scene using computer vision according to any one of claims 1 to 7 when running.

Citation Information

Patent Citations

  • Intelligent analysis method of store people stream track on the basis of surveillance video

    CN104933710A

  • Urinary sediment crystal identification method and device, computer device and storage medium

    CN109190590A

  • Pedestrian positioning and trajectory tracking method based on video image

    CN113643363A

  • Track acquisition method, system and device for multi-target pedestrian, and medium

    CN113658192A

  • Graph affine transformation method and device

    CN116128715A