Port scene pedestrian trajectory acquisition method and device for computer vision

Abstract

The application discloses a kind of computer vision's port scene pedestrian trajectory acquisition method and device, it is related to image processing field;The method comprises: by obtaining the video frame image in the video stream image of camera equipment installed on the lamp pole in port area;Determine the pedestrian contour in each frame video frame image and the first minimum circumscribed rectangle corresponding to pedestrian contour;Perspective transformation is carried out to the first minimum circumscribed rectangle, and the second minimum circumscribed rectangle under the bird's eye view corresponding to each frame video frame image is obtained;The bottom side of second minimum circumscribed rectangle is selected, and the vertical line of the other rectangular side parallel to bottom side is made to the two end points of bottom side respectively in second minimum circumscribed rectangle, and the length of vertical line is all equal to bottom side, to obtain third minimum circumscribed rectangle frame;According to the third minimum circumscribed rectangle frame of each frame in video frame image, determine pedestrian trajectory;Solved the problem that the object tracking trajectory calculation error is big, improved the object's tracking trajectory calculation accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, in particular to a port scene pedestrian trajectory acquisition method and device based on computer vision, an electronic device and a storage medium. BACKGROUND

[0002] With the increase of transportation volume year by year, the safety management problem of the port area is increasingly prominent. The port area is often crowded with people, vehicles and goods, and some people and vehicles may not comply with safety standards, such as the intrusion of people and vehicles into prohibited areas, which may cause people and vehicles to be too close, and even cause safety accidents such as collision between people and vehicles. Therefore, efficient safety management measures to prevent people and vehicles from colliding are very important. On-site patrol or monitoring of security personnel is a management method for checking safety hazards, but this manual detection method is not efficient. In recent years, video analysis technology based on computer vision has developed rapidly and is widely used in various fields including security, with the advantages of high efficiency and good real-time performance, which can monitor and analyze multiple areas equipped with monitoring at any time without interruption, and determine the possibility of collision between people and vehicles in the picture.

[0003] Traditional systems for preventing people and vehicles from colliding require the installation of radars or binocular cameras on each vehicle, which are achieved through radar 3D modeling or binocular ranging technology. These methods require the installation of radars or binocular cameras and corresponding computer equipment on each vehicle, which is very costly. Moreover, as the vehicle moves, the distance deviation in the view of the acquired image will exist, i.e. the distance is large in the near view and small in the far view, which makes the accuracy of the acquired results not high, and further leads to large tracking trajectory calculation error of the object.

[0004] At present, there is no effective solution to the problem of large tracking trajectory calculation error of the object in the related art. SUMMARY

[0005] The embodiments of the present application provide a port scene pedestrian trajectory acquisition method and device based on computer vision, an electronic device and a storage medium to at least solve the problem of large tracking trajectory calculation error of the object in the related art.

[0006] In a first aspect, the embodiments of the present application provide a port scene pedestrian trajectory acquisition method based on computer vision, comprising:

[0007] acquiring a video frame image in a video stream image of a camera device installed on a lamp post in a port area;

[0008] determining a pedestrian contour in each frame of the video frame image and a first minimum bounding rectangle corresponding to the pedestrian contour;

[0009] Perform perspective transformation on the first minimum bounding rectangle to obtain the second minimum bounding rectangle in the top view corresponding to each video frame image;

[0010] Select the bottom edge of the second minimum bounding rectangle, and draw perpendicular lines from the two endpoints of the bottom edge to another rectangular side of the second minimum bounding rectangle that is parallel to the bottom edge, with the length of the perpendicular lines being equal to that of the bottom edge, to obtain the third minimum bounding rectangle.

[0011] The pedestrian trajectory is determined based on the third minimum bounding rectangle of each frame in the video frame image.

[0012] Secondly, embodiments of this application provide a computer vision-based device for acquiring pedestrian trajectories in a port scene, comprising:

[0013] The acquisition module is used to acquire video frame images from the video stream images of the camera equipment installed on the light poles in the port area;

[0014] The first determining module is used to determine the pedestrian outline in each frame of the video frame image and the first minimum bounding rectangle corresponding to the pedestrian outline;

[0015] The transformation module is used to perform perspective transformation on the first minimum bounding rectangle to obtain the second minimum bounding rectangle under the top view corresponding to each video frame image;

[0016] The processing module is used to select the bottom edge of the second minimum bounding rectangle, and draw perpendicular lines from the two endpoints of the bottom edge to another rectangular side of the second minimum bounding rectangle that is parallel to the bottom edge, and the length of the perpendicular lines is equal to that of the bottom edge, so as to obtain the third minimum bounding rectangle.

[0017] The second determining module is used to determine the pedestrian trajectory based on the third minimum bounding rectangle of each frame in the video frame image.

[0018] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the computer vision method for obtaining pedestrian trajectories in a port scene as described in the first aspect above.

[0019] Fourthly, embodiments of this application provide a storage medium storing a computer program thereon, which, when executed by a processor, implements the computer vision method for obtaining pedestrian trajectories in a port scene as described in the first aspect above.

[0020] Compared to related technologies, the computer vision method, apparatus, electronic device, and storage medium for acquiring pedestrian trajectories in port scenes provided in this application embodiment acquire video frame images from video stream images of camera equipment installed on light poles in the port area; determine the pedestrian contour and the corresponding first minimum bounding rectangle in each video frame image; perform perspective transformation on the first minimum bounding rectangle to obtain the second minimum bounding rectangle in the top view corresponding to each video frame image; select the bottom edge of the second minimum bounding rectangle, and draw perpendicular lines from the two endpoints of the bottom edge to another rectangular side parallel to the bottom edge in the second minimum bounding rectangle, with the length of the perpendicular lines being equal to the length of the bottom edge, to obtain a third minimum bounding rectangle frame; and determine the pedestrian trajectory based on the third minimum bounding rectangle frame of each frame in the video frame image. This method solves the problem of large calculation errors in object tracking trajectories in related technologies and improves the accuracy of object tracking trajectory calculation.

[0021] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0022] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0023] Figure 1 This is a hardware structure block diagram of the terminal for the computer vision-based method for obtaining pedestrian trajectories in a port scene according to an embodiment of the present invention.

[0024] Figure 2 This is a flowchart of a computer vision method for obtaining pedestrian trajectories in a port scene according to an embodiment of this application;

[0025] Figure 3 This is a schematic diagram of the outline, horizontal bounding rectangle, and minimum bounding rectangle of a pedestrian video frame in a port scene according to an embodiment of this application using computer vision.

[0026] Figure 4 This is a schematic diagram of the selection of the bottom edge of the third minimum bounding rectangle under a top view according to an embodiment of this application;

[0027] Figure 5 This is a structural block diagram of a computer vision-based device for acquiring pedestrian trajectories in a port scene, according to an embodiment of this application. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application. Furthermore, it is understood that although the efforts made in such a development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, modifications to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0029] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0030] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application means two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The terms “first,” “second,” “third,” etc., used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0031] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. Taking running on a terminal as an example,Figure 1 This is a hardware structure block diagram of the terminal for the computer vision-based method for acquiring pedestrian trajectories in a port scene, according to an embodiment of the present invention. Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. Optionally, the terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0032] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the computer vision method for acquiring pedestrian trajectories in a port scene according to this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0033] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0034] This embodiment provides a computer vision method for obtaining pedestrian trajectories in a port scene. Figure 2 This is a flowchart of a computer vision method for obtaining pedestrian trajectories in a port scene according to an embodiment of this application, as shown below. Figure 2 As shown, the process includes the following steps:

[0035] Step S201: Obtain video frame images from the video stream images of the camera equipment installed on the light poles in the port area.

[0036] In this step, the images used are taken by the port area's light poles and / or lighthouse cameras, which can be shot from a top-down angle.

[0037] Step S202: Determine the pedestrian outline in each video frame image and the first minimum bounding rectangle corresponding to the pedestrian outline.

[0038] In this step, determining the pedestrian outline in the video frame image can be achieved through methods such as image recognition and deep learning models.

[0039] Step S203: Perform perspective transformation on the first minimum bounding rectangle to obtain the second minimum bounding rectangle in the top view corresponding to each video frame image.

[0040] Step S204: Select the bottom edge of the second minimum bounding rectangle, and draw perpendicular lines from the two endpoints of the bottom edge to another rectangle edge in the second minimum bounding rectangle that is parallel to the bottom edge, with the length of the perpendicular lines being equal to that of the bottom edge, to obtain the third minimum bounding rectangle.

[0041] In steps S203 to S204, the pedestrian tracking trajectory is determined by perspective transformation to a top view, which eliminates the distance deviation in the view caused by the perspective effect of the camera device, which makes the judgment result more accurate.

[0042] Step S205: Determine the pedestrian trajectory based on the third minimum bounding rectangle of each frame in the video frame image.

[0043] Through the above steps S201 to S205, the perspective transformation is used to convert to a top view, and the minimum bounding rectangle of the pedestrian is obtained on the top view. The final tracking trajectory is determined based on the minimum bounding rectangle on the top view. This eliminates the distance deviation on the view caused by the perspective effect of the camera, which makes objects appear larger when closer and smaller when farther away. It solves the problem of large errors in the calculation of the object's tracking trajectory in related technologies and improves the accuracy of the object's tracking trajectory calculation.

[0044] In some embodiments, determining the pedestrian contour in each video frame and the first minimum bounding rectangle corresponding to the pedestrian contour includes: segmenting the video frame image using a deep learning algorithm to obtain the pedestrian contour in the video frame image; determining the horizontal bounding rectangle corresponding to the pedestrian based on the pedestrian contour; and tracing the horizontal bounding rectangle to obtain the first minimum bounding rectangle corresponding to the pedestrian.

[0045] In this embodiment, the process of segmenting video frame images using a deep learning algorithm to obtain pedestrian contours in the video frame images includes: inputting the video frame images into the input end of a YOLOv8 segmentation network; obtaining contour coordinates and contour categories from the output end of the YOLOv8 segmentation network; and determining pedestrian contours based on the contour coordinates and contour categories.

[0046] The following is a detailed description and explanation of how the above embodiments determine the pedestrian outline in each video frame and the first minimum bounding rectangle corresponding to the pedestrian outline.

[0047] Using YOLOv8, target segmentation is performed on each frame of the image to filter out pedestrians and extract their contours, as shown in the image below. Figure 3 In the Q-shape, the points of each group of pedestrian contours can be represented by an array [L1, L2, L3, ..., L...]. n ] indicates that Ln represents the nth point on the contour, (L n .x, L n .y) represents the coordinates of the nth point on the contour.

[0048] Based on the target contour array [L1, L2, L3, ..., L... n The horizontal bounding rectangle and minimum bounding rectangle of the target are calculated. The horizontal bounding rectangle is used for pedestrian tracking in video frames to determine which specific target the minimum bounding rectangle of multiple consecutive frames belongs to. The minimum bounding rectangle is used to transform the tracking box under the top view image after perspective transformation. The four coordinate points of the minimum bounding rectangle of the video frame are represented by [R]. min1 , R min2 , R min3 , R min4 ] indicates, (R min.x , R min.y ) represents the x, y coordinates of the smallest bounding rectangle.

[0049] The specific steps are as follows:

[0050] Converting a contour to a horizontal bounding rectangle includes the following methods:

[0051] 1.1 Initialize the variables min_x and max_x as the x-coordinates (L1.x, L1.y) of the first point L1 in the contour.

[0052] 1.2 Traverse each point Ln on the contour and obtain the coordinates (L... n .x, L n .y).

[0053] 1.3 Update min_x to the L value of the currently visited point. n The smaller of the x-coordinate and min_x is used to update max_x to the L value of the currently visited point.n The larger of the x-coordinate and max_x.

[0054] 1.4 Update min_y to the smaller of the Ln.y coordinates of the currently traversed point and min_y, and update max_y to the Ln.y coordinates of the currently traversed point. n The larger of the .y coordinate and max_y.

[0055] The coordinates of the four points of the final horizontal bounding rectangle are (min_x, min_y), (max_x, min_y), (max_x, max_y), and (min_x, max_y), and its final horizontal bounding rectangle is as follows. Figure 3 The T-shaped figure in the image.

[0056] The outline can be converted into the smallest bounding rectangle in the following way:

[0057] 2.1 Calculate the convex hull of the profile using the Graham scan method.

[0058] 2.1.1 Find the contour array [L1, L2, L3, ..., L... n The point with the smallest y-coordinate is selected. If multiple points have the same y-coordinate, the point with the smallest x-coordinate is chosen. This point is the first point H1 of the convex hull.

[0059] 2.1.2 Calculate the polar angles from all points in the contour array to the first point H1, and store the points and polar angles together in a new array [(L1,θ1), (L2,θ2), (L3,θ3), ..., (L... n-1 ,θ n-1 ), (L n ,θ n )).

[0060] 2.1.3 For the array [(L1,θ1), (L2,θ2), (L3,θ3), …… , (L n-1 ,θ n-1 ), (L n ,θ n According to the polar angle θ] x Sort the points in ascending order, assuming the sorted array of contour points is (M1, M2, M3, ..., M...). n-1 M n M1 is an array [(L1,θ1), (L2,θ2), (L3,θ3), ..., (L... n-1 ,θ n-1 ), (L n ,θ n The point M with the smallest θx value in the equation )]n θ in the array x The point with the largest value.

[0061] 2.1.4 Use a stack to store points on the convex hull. Create a convex hull array [M1, M2, M3, ..., M...]. n-1 M n The first two points M1 and M2 in the set are pushed onto the stack, making the stack [M1, M2], with M2 as the top of the stack. Then, starting from the M3 point, the sorted set of points is traversed.

[0062] 2.1.5 For the currently visited point M i (i>=3), connect stack vertex M k (k>=2) and M1, we obtain the straight line L: Then determine M. i (coordinates are (M) i .x,M i On which side of line L is .y)? Then M i To the right of line L, Then M i To the left of line L. If M i To the right of line L, point M represents the top of the stack. k It is a depression, M k It should be deleted. At this point, move the top point M of the stack. k Pop the stack and repeat this step until the top of the stack is to the left of line L. Then, move M... i Push it onto the stack.

[0063] 2.1.6 After all points have been checked, the remaining points in the stack are the points on the convex hull. Move the points from the stack into an array, and the final result is denoted as [K1, K2, K3, ..., K]. n-1 , K n ].

[0064] 2.2 Solving for the minimum bounding rectangle using the convex hull method includes the following approaches:

[0065] 2.2.1 Traverse all edges K of the convex hull m K m+1 For each edge, try to find the point that is perpendicular to the edge and furthest from it, let's say it's point K. x Through edge K m K m+1 and point K x Construct four points [R] that can enclose the rectangle of the convex hull. m1 , R m2 ,R m3 , R m4], and record the area Sm of the rectangle. After traversing all edges of the convex hull, we have an array [ [R] to store the rectangle. 11 ,R 12, R 13, R 14 ], [ [R 11 , R 12, R 13, R 14 ], …… ,[R m1 , R m2 , R m3 , R m4 [S1, S2, S3, ..., S] and the corresponding array of rectangle areas [S1, S2, S3, ..., S] m-1 , S m ].

[0066] 2.2.2 Find [S1, S2, S3, ..., S m-1 , S m The smallest area S in the [ ] min and the corresponding rectangle [R min1 , R min2 , R min3 , R min4 ], which is the minimum bounding rectangle, and its minimum bounding rectangle is as follows: Figure 3 The W-shaped graph in the image.

[0067] In the above embodiments, the specific process of perspective transformation may include:

[0068] Two sets of parallel lines are placed on a video frame. These two sets of parallel lines must be perpendicular to each other in real physical space. These two sets of parallel lines will intersect at four points A, B, C, and D in the image.

[0069] The horizontal circumscribed rectangle EFGH of quadrilateral ABCD is given, with points A, B, and C corresponding to points E, F, and G, respectively. Calculate the perspective transformation matrix from quadrilateral ABCD to rectangle EFGH. Assume the coordinates of A, B, and C are... , (X b ,Y b ), (X c ,Y c The coordinates of E, F, and G are respectively , (X f ,Y f ), (X g ,Y g If the vectors of vertices A, B, and C are given, then the matrix formed by them is:

[0070] ;

[0071] E, F, G The matrix formed by the vectors of the three vertices is:

[0072] ;

[0073] Then the perspective transformation matrix M for:

[0074] .

[0075] like Figure 4 As shown, find the minimum bounding rectangle of the target in each frame. R min1 , R min2 , R min3 , R min4 Of the four points y Let's assume the point with the largest value is... R min1 There are two adjacent edges with that point as the endpoint. R mini R min(i-1) , R mini , R min(i+1) We select from them x The side with the smaller included angle (here, the included angle is acute; if the included angle is obtuse, its supplementary angle needs to be calculated), let's assume it's... R mini R min(i+1) Perform perspective transformation on the two points on this side. , Using this side as the base, draw a perpendicular line in the negative direction of the y-axis. In this embodiment, the target is a pedestrian, and the length of the perpendicular line is... Consistent. The center point of the pedestrian's tracking bounding box is calculated using the following steps:

[0076] Calculation points and points center point T c The calculation formula is as follows:

[0077] ;

[0078] ;

[0079] Calculate the length of the perpendicular line, the length of the pedestrian's perpendicular line. dis human equal The calculation formula is as follows: ;

[0080] To calculate the radian of a perpendicular line, the two-point method can be used. The formula is as follows:

[0081] ;

[0082] Calculate the coordinates of the endpoint of the perpendicular line: The coordinates of the endpoint of the pedestrian's perpendicular line are... V human The calculation formula is:

[0083] ;

[0084] In some embodiments, determining the pedestrian trajectory based on the third minimum bounding rectangle of each frame in the video frame image includes: determining the center point of the third minimum bounding rectangle of each frame; and determining the pedestrian trajectory based on the center point of the third minimum bounding rectangle of each frame.

[0085] In this embodiment, based on the above embodiments, the center point of the third minimum bounding rectangle of each frame can be determined in the following way:

[0086] The coordinates of the center point of the pedestrian tracking rectangle after perspective transformation are ( The calculation formula is:

[0087] ;

[0088] The center point of the pedestrian's tracking bounding box in each frame after perspective transformation is retained as the tracking trajectory. The pedestrian's perspective-transformed tracking trajectory is then uniformly represented as an array. [E center1 , E center2 , E center3 , ……, E center(n-1) , E center(n) Simultaneously record the time of each trajectory point. T 1 , T 2 , T 3 , ……, T (n-1) , T n ].

[0089] In some of these embodiments, the pedestrian trajectory can be smoothed before performing a preset straight-line fitting on the pedestrian trajectory to determine the predicted trajectory corresponding to the pedestrian trajectory.

[0090] In this embodiment, the specific method of smoothing can be:

[0091] Apply Gaussian filtering to the tracking trajectory array [ E center1 , E center2 , E center3 , ……, E center(n-1) , E center(n) Smooth the coordinates to form a new coordinate array. E g1 , E g2 , E g2 , ……, E gn-1 , E gn ], where the Gaussian function is Using the Gaussian function, a normalized Gaussian filter kernel [0.319, 0.362, 0.319] is obtained. The filtering formula for the coordinate point with index i in the array is:

[0092] .

[0093] In some embodiments, after determining the pedestrian trajectory based on the third minimum bounding rectangle of each frame in the video frame image, the method further includes: performing a preset straight line fitting on the pedestrian trajectory to determine the predicted trajectory corresponding to the pedestrian trajectory; judging whether there is a collision hazard based on the predicted trajectory; and issuing an alarm if a collision hazard is judged to exist in the pedestrian trajectory.

[0094] In this embodiment, the preset straight line fitting includes at least one of the following: first-order straight line fitting and second-order straight line fitting.

[0095] The specific preset linear fitting in the above embodiments can be done in the following ways:

[0096] Take the 100 most recent trajectory points [ E g(n-99) , E g(n-98) , ……, E gn-1 , E gn ] and the corresponding time record for each trajectory point [ T (n-99) , T (n-98) ,……, T (n-1) , T nThe array is fitted with a linear function and a quadratic curve function. The formula for the linear function fitting is: The quadratic curve fitting function is Using the least squares method, calculate a , b , c , d , e The value:

[0097] ;

[0098] ;

[0099] ;

[0100] ;

[0101] ;

[0102] Calculate the fitted linear arrays respectively. E l(n-99) , E l(n-98) , ……, E l(n-1) , E ln ] and quadratic line array [ E d(n-99) , E d(n-98) , ……, E d(n-1) , E dn ] and the original real array[ E g(n-99) , E g(n-98) , ……, E g(n-1) , E gn ]exist y Error in coordinates Difference 1 and Difference 2 :

[0103] ;

[0104] ;

[0105] if Difference 1 > Difference 2 Then choose the quadratic curve to fit the trajectory, ifDifference 1 < Difference 2 If so, then a linear function is chosen to fit the trajectory.

[0106] Based on the fitted trajectory, predict whether the pedestrian's target tracking box will intersect with the target tracking box after perspective transformation at the next time T.

[0107] Suppose the fitted trajectory of a pedestrian in the image is y = f(x) , f(x) The form can be or The above tracking point E g(n-1) To the current tracking point E gn The velocity is used as the velocity for predicting the trajectory at the next moment. The formula for calculating the velocity is:

[0108] ;

[0109] The distance moved at the next time point T is .

[0110] (1) Assume the trajectory of the target is The current tracking point coordinates are ( E gn .x, ), calculate the next time step E g(n+1) .x and E g(n+1) .y:

[0111] ;

[0112] .

[0113] (2) Assume the trajectory of the target is The current tracking point coordinates are ( E gn .x, ), calculate the next time step E g(n+1) .x and E g(n+1) .y:

[0114] beg The derivative: ;

[0115] ;

[0116] ;

[0117] With point ( E g(n+1) .x, f(E g(n+1) .x) ( ) is the center of the circle. Let the diameter be a circle. Determine the location of each trajectory within (...). E g(n+1) .x, f(E g(n+1) .x) Does the circular region at the center point have any overlap?

[0118] If such a situation exists, it is considered that human-machine interaction will occur; otherwise, it is considered that there is no risk of human-machine interaction.

[0119] This embodiment also provides a computer vision-based pedestrian trajectory acquisition device for port scenes. This device is used to implement the above embodiments and preferred embodiments, and details already described will not be repeated. As used below, the terms "module," "unit," "subunit," etc., can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0120] Figure 5 This is a structural block diagram of a computer vision-based device for acquiring pedestrian trajectories in a port scene, according to an embodiment of this application. Figure 5 As shown, the device includes:

[0121] The acquisition module 51 is used to acquire video frame images from the video stream images of the camera equipment installed on the light poles in the port area;

[0122] The first determining module 52 is used to determine the pedestrian outline in each frame of the video frame image and the first minimum bounding rectangle corresponding to the pedestrian outline;

[0123] Transformation module 53 is used to perform perspective transformation on the first minimum bounding rectangle to obtain the second minimum bounding rectangle under the top view corresponding to each video frame image;

[0124] Processing module 54 is used to select the bottom edge of the second minimum bounding rectangle, and draw perpendicular lines from the two endpoints of the bottom edge to another rectangular side of the second minimum bounding rectangle that is parallel to the bottom edge, and the length of the perpendicular lines is equal to that of the bottom edge, so as to obtain the third minimum bounding rectangle.

[0125] The second determining module 55 is used to determine the pedestrian trajectory based on the third minimum bounding rectangle of each frame in the video frame image.

[0126] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0127] This embodiment also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0128] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0129] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0130] Step S201: Obtain video frame images from the video stream images of the camera equipment installed on the light poles in the port area.

[0131] Step S202: Determine the pedestrian outline in each video frame image and the first minimum bounding rectangle corresponding to the pedestrian outline.

[0132] Step S203: Perform perspective transformation on the first minimum bounding rectangle to obtain the second minimum bounding rectangle in the top view corresponding to each video frame image.

[0133] Step S204: Select the bottom edge of the second minimum bounding rectangle, and draw perpendicular lines from the two endpoints of the bottom edge to another rectangle edge in the second minimum bounding rectangle that is parallel to the bottom edge, with the length of the perpendicular lines being equal to that of the bottom edge, to obtain the third minimum bounding rectangle.

[0134] Step S205: Determine the pedestrian trajectory based on the third minimum bounding rectangle of each frame in the video frame image.

[0135] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0136] Furthermore, in conjunction with the computer vision-based pedestrian trajectory acquisition method for port scenes described in the above embodiments, this application embodiment can provide a storage medium for implementation. This storage medium stores a computer program; when executed by a processor, the computer program implements any of the computer vision-based pedestrian trajectory acquisition methods for port scenes described in the above embodiments.

[0137] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0138] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A computer vision based method for obtaining pedestrian trajectories in a port scene, the method comprising: The method comprises the following steps: obtaining a video frame image in a video stream image of a camera device installed on a lamp pole in a port area; determining a pedestrian contour in each frame of the video frame image and a first minimum bounding rectangle corresponding to the pedestrian contour, wherein the first minimum bounding rectangle is a minimum bounding rectangle that can enclose the pedestrian contour and has the smallest area; perspective transformation is performed on the first minimum bounding rectangle to obtain a second minimum bounding rectangle in the top view corresponding to each frame of the video frame image; selecting a bottom side of the second minimum bounding rectangle, and drawing a perpendicular line from each end point of the bottom side to another rectangular side in the second minimum bounding rectangle which is parallel to the bottom side, and the length of the perpendicular line is equal to that of the bottom side, to obtain a third minimum bounding rectangle; determining the pedestrian trajectory according to the third minimum bounding rectangle of each frame of the video frame image; wherein determining the pedestrian contour in each frame of the video frame image and the first minimum bounding rectangle corresponding to the pedestrian contour comprises: segmenting the video frame image by using a deep learning algorithm to obtain the pedestrian contour in the video frame image; determining a horizontal bounding rectangle corresponding to the pedestrian according to the pedestrian contour; and tracking the horizontal bounding rectangle to obtain the first minimum bounding rectangle corresponding to the pedestrian; wherein segmenting the video frame image by using a deep learning algorithm to obtain the pedestrian contour in the video frame image comprises: inputting the video frame image into the input end of a yolov8 segmentation network; obtaining contour coordinate points and contour categories from the output end of the yolov8 segmentation network; and determining the pedestrian contour according to the contour coordinate points and the contour categories; wherein determining the pedestrian trajectory according to the third minimum bounding rectangle of each frame of the video frame image comprises: determining the center point of the third minimum bounding rectangle of each frame; and determining the pedestrian trajectory according to the center point of the third minimum bounding rectangle of each frame. 2.The computer vision based port scene pedestrian trajectory acquisition method according to claim 1, characterized in that, After determining the pedestrian trajectory according to the third minimum bounding rectangle of each frame of the video frame image, the method further comprises: performing a preset straight line fitting on the pedestrian trajectory to determine a predicted trajectory corresponding to the pedestrian trajectory; judging whether the pedestrian trajectory is in collision danger according to the predicted trajectory; performing an alarm in the case that the pedestrian trajectory is in collision danger. 3.The computer vision based port scene pedestrian trajectory acquisition method according to claim 2, characterized in that, The preset straight line fitting comprises at least one of the following: a first order straight line fitting and a second order straight line fitting. 4.The computer vision based port scene pedestrian trajectory acquisition method according to claim 2, characterized in that, Before performing the preset straight line fitting on the pedestrian trajectory to determine the predicted trajectory corresponding to the pedestrian trajectory, the method further comprises: smoothing the pedestrian trajectory.

5. A device for acquiring a port scene pedestrian trajectory of computer vision, characterized in that, The method comprises the following steps: an acquisition module, configured to acquire a video frame image in a video stream image of a camera device installed on a lamp pole in a port area; a first determination module, configured to determine a pedestrian contour in each frame of the video frame image and a first minimum bounding rectangle corresponding to the pedestrian contour, wherein the first minimum bounding rectangle is a minimum bounding rectangle that can enclose the pedestrian contour and has the smallest area; The transformation module is configured to perform perspective transformation on the first minimum bounding rectangle to obtain a second minimum bounding rectangle corresponding to a top view of each frame of the video frame image; The processing module is configured to select a bottom side of the second minimum bounding rectangle, and draw a perpendicular line from each end point of the bottom side to another rectangular side of the second minimum bounding rectangle, which is parallel to the bottom side, and the length of the perpendicular line is equal to that of the bottom side, to obtain a third minimum bounding rectangle; The second determination module is configured to determine the pedestrian trajectory according to the third minimum bounding rectangle of each frame of the video frame image; The method comprises the following steps: segmenting the video frame image by using a deep learning algorithm to obtain a pedestrian contour in the video frame image; determining a horizontal bounding rectangle corresponding to the pedestrian according to the pedestrian contour; and tracking the horizontal bounding rectangle to obtain a first minimum bounding rectangle corresponding to the pedestrian. The method comprises the following steps: inputting the video frame image into an input end of a yolov8 segmentation network; obtaining contour coordinate points and a contour category from an output end of the yolov8 segmentation network; and determining the pedestrian contour according to the contour coordinate points and the contour category. The method comprises the following steps: determining a center point of the third minimum bounding rectangle of each frame; and determining the pedestrian trajectory according to the center point of the third minimum bounding rectangle of each frame. 6.An electronic device comprising a memory and a processor, the electronic device comprising: The memory stores a computer program, and the processor is configured to run the computer program to execute the computer vision port scene pedestrian trajectory acquisition method in any one of claims 1 to 4.

7. A storage medium, characterized by The storage medium stores a computer program, and the computer program is configured to execute the computer vision port scene pedestrian trajectory acquisition method in any one of claims 1 to 4 when running.

Images