Passing method and device of detection area, server, medium and product

Abstract

The invention provides a passing method and device of a detection area, a server, a medium and a product, and belongs to the technical field of automatic driving. The method comprises the following steps: acquiring video data of a detection area; according to the video data, pre-estimated data of a current time point in the detection area are obtained, and the pre-estimated data represent motion pre-estimated tracks of all moving targets in the detection area at a next time point; and determining a strategy for the vehicle to pass through the detection area at the current time point according to the estimated data at the current time point. On the premise that safety is guaranteed, the vehicle passing rate of the pedestrian zebra crossing is increased to the maximum extent, and meanwhile driving safety and efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving technology, and in particular to a method, device, server, medium and product for detecting a passage in an area. Background Art

[0002] When a vehicle passes through a detection area, such as a zebra crossing, the commonly used radar + camera method of real-time identification and decision-making on whether the conditions for passing are met has the logical complexity of multi-device interaction. At the same time, the existing passing solutions either excessively pursue safety and wait until the target area is completely safe before passing, or enter the target area too early and cause a collision. This has the disadvantages of slow real-time response and low passing efficiency. Summary of the invention

[0003] In view of this, one of the purposes of the embodiments of the present application is to provide a method for passing through a detection area, which can improve the problems of slow real-time response and low passing efficiency when passing through the detection area in the prior art.

[0004] In order to achieve the above technical objectives, the technical solutions adopted in this application are as follows:

[0005] In a first aspect, an embodiment of the present application provides a method for passing through a detection area, the method comprising:

[0006] Obtain video data of the detection area;

[0007] Obtaining estimated data at a current time point in the detection area, wherein the estimated data represents an estimated motion trajectory of all moving targets in the detection area at a next time point;

[0008] A strategy for the vehicle to pass through the detection area at the current time point is determined based on the estimated data at the current time point.

[0009] According to the above technical means, the traffic strategy of the automobile detection area at the current time point is determined by the estimated data at the current time point, so that the vehicle can start to pass through the detection area at a time point before there is no moving target in the detection area, thereby improving the response rate when passing through the detection area and improving efficiency.

[0010] Further, obtaining the estimated data of the optimal current time point includes:

[0011] The estimated data at the current time point is obtained by combining the detection data at the current time point and the estimated data at the previous time point, wherein the detection data includes the motion trajectories of all moving targets in the detection area, and the detection data is obtained based on the video data.

[0012] According to the above technical means, the estimated data at the current time point is determined by combining the detection data at the current time point and the estimated data at the previous time point. The estimated data at the previous time point can be corrected by the detection data at the current time point, further improving the accuracy of the moving target trajectory estimation.

[0013] Further, the combining the detection data at the current time point and the estimated data at the previous time point to obtain the estimated data at the current time point includes:

[0014] Processing the estimated data at the previous time point through a Kalman filter to obtain initialization data;

[0015] Removing the moving targets and the corresponding estimated motion trajectories that are not located in the detection area from the initialization data to obtain intermediate data;

[0016] The moving target in the detection data at the current time point is matched with the moving target in the intermediate data, and based on the matching result, the estimated data at the current time point is obtained.

[0017] Furthermore, the matching of the moving target in the detection data at the current time point with the moving target in the intermediate data, and obtaining the estimated data at the current time point based on the matching result, includes:

[0018] Process the intermediate data through a Kalman filter;

[0019] Determine whether there is a new sports target;

[0020] If it does not exist, the intermediate data processed by the Kalman filter is the estimated data at the current time point;

[0021] If it exists, the newly added moving target and its motion trajectory in the detection data at the current time point are obtained to obtain the newly added data; the intermediate data and the newly added data processed by the Kalman filter are combined to obtain the estimated data at the current time point.

[0022] Further, matching the moving target in the detection data at the current time point with the moving target in the intermediate data includes:

[0023] The moving targets in the detection data at the current time point are matched with the moving targets in the intermediate data based on the Hungarian algorithm.

[0024] Further, the strategy for determining that the vehicle passes through the detection area at the current time point according to the estimated data at the current time point includes:

[0025] According to the estimated data at the current time point, if there is no moving target in the detection area at the next time point, the vehicle starts to start at the current time point and passes through the detection area, and the passing speed of the detection area is set based on the motion trajectory of the moving target in the detection data at the current time point.

[0026] Further, after obtaining the video data of the detection area, the method further includes:

[0027] Extracting a plurality of frame image data from the video data;

[0028] Binarizing the frame image data;

[0029] A moving target in the frame image data is obtained.

[0030] Further, binarizing the current frame image to obtain a moving target in the current frame image includes:

[0031] Processing the frame image data by a background difference strategy to obtain a first binary image;

[0032] Processing the frame image data by a three-frame difference method to obtain a second binary image;

[0033] The first binarized image and the second binarized image are combined, and a moving object is obtained according to the combined image.

[0034] Furthermore, after binarizing the frame image data, the method further includes: performing dilation processing and / or erosion processing on the binarized frame image data.

[0035] In a second aspect, this embodiment provides a passage device for detecting an area, including:

[0036] An acquisition module configured to acquire video data of a detection area;

[0037] A calculation module, configured to obtain estimated data at a current time point in the detection area, wherein the estimated data represents the estimated motion trajectory of all moving targets in the detection area at a next time point;

[0038] The driving module is configured to determine a strategy for the vehicle to pass through the detection area at the current time point according to the estimated data at the current time point.

[0039] In a third aspect, this embodiment proposes a server, which includes a processor and a memory coupled to each other, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the server executes the above method.

[0040] In a fourth aspect, this embodiment proposes a computer-readable storage medium, in which a computer program is stored. When the computer program runs on a computer, the computer executes the above method.

[0041] In a fifth aspect, this embodiment proposes a computer program product, including a computer program, which implements the above method when executed by a processor.

[0042] The invention adopting the above technical solution has the following advantages:

[0043] In the technical solution provided in the present application, the traffic strategy for the detection area at the current time point is determined according to the estimated data at the current time point, so that the vehicle can plan the driving path in advance according to the estimated data to avoid collision with moving targets, thereby making advance judgments and decisions on the passability of the intelligent driving motor vehicle to the target area at the current moment. While ensuring safety, the vehicle passing rate of the pedestrian zebra crossing is maximized, while also improving driving safety and efficiency.

[0044] The technical solution of the present application can more comprehensively understand the motion state of the moving target by combining the detection data at the current time point and the estimated data at the previous time point, thereby more accurately predicting its motion trajectory at the next time point. At the same time, the solution processes the estimated data and the detection data through the Kalman filter, which can effectively remove noise and interference and improve the accuracy of trajectory prediction.

[0045] The technical solution of the present application reduces unnecessary calculations and optimizes resource utilization by removing moving targets that are not located in the detection area and the corresponding motion estimation trajectories. At the same time, by combining the Kalman filter and the Hungarian algorithm, the matching accuracy of the moving target in the previous and next frames of images is improved, which can reduce the calculation complexity and improve the real-time performance of the algorithm while ensuring the prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The present application may be further described by the non-limiting embodiments given in the accompanying drawings. It should be understood that the following drawings only illustrate certain embodiments of the present application and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings may be obtained based on these drawings without creative effort.

[0047] Figure 1 An overall flow chart provided for an embodiment of the present application.

[0048] Figure 2 This is a flow chart of the S121-S123 method provided in an embodiment of the present application.

[0049] Figure 3This is a flow chart of the S1231-S1235 methods provided in an embodiment of the present application.

[0050] Figure 4 A flowchart for predicting pedestrian motion trajectories provided in an embodiment of the present application.

[0051] Figure 5 This is a structural diagram of the passage device in the detection area, where the direction of the arrow indicates the direction of signal flow.

[0052] Icon: 1-acquisition module; 2-calculation module; 3-driving module; 4-visual processing module. DETAILED DESCRIPTION

[0053] The present application will be described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that in the drawings or descriptions, similar or identical parts use the same figure numbers, and the implementation methods not shown or described in the drawings are forms known to ordinary technicians in the relevant technical field. In the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0054] Example 1

[0055] The present embodiment proposes a method for passing through a detection area, which solves the problem that the prior art excessively pursues safety and waits until the target area is completely safe before passing through, or enters the target area too early, causing a collision, thereby resulting in low passing efficiency.

[0056] In this embodiment, the detection area is a general term for the area that the vehicle needs to pass through during automatic driving. Pedestrians often pass through the detection area. The vehicle needs to determine whether there are pedestrians in the detection area during automatic driving so that the vehicle can safely enter the detection area. The detection area in this embodiment is a sidewalk with a zebra crossing, but the detection area may also include other passages that pedestrians often pass through, such as an aisle at one end in front of the entrance to a community garage. Vehicles need to pass through the aisle to enter the community garage, and since the aisle is located near the community, pedestrians often pass by.

[0057] The technical solution of this embodiment adopts a pure visual solution. Based on the real-time video data collected by the on-board video acquisition equipment, the foreground target of the target area is extracted, and then the motion trajectory of the foreground target over a period of time is estimated in combination with relevant algorithms. Finally, the motion trajectory of the target at the next moment is estimated at the current moment, so that the passability of the intelligent driving motor vehicle to the target area can be judged in advance at the current moment, and the vehicle passing rate of the pedestrian zebra crossing can be maximized while ensuring safety.

[0058] Please refer to Figure 1The present application also provides a method for detecting a passage through an area, which may include the following steps:

[0059] Step 110, obtaining video data of the detection area;

[0060] Step 120, obtaining estimated data at the current time point in the detection area, the estimated data representing the estimated motion trajectory of all moving targets in the detection area at the next time point;

[0061] Step 130, determining a traffic strategy for the vehicle in the detection area at the current time point based on the estimated data at the current time point.

[0062] The following will describe in detail the steps of the method for passing through the detection area, as follows:

[0063] In step 110, the video data may be in MP4, AVI, and RMVB formats.

[0064] In step 120, the estimated data represents the estimated motion trajectory of all moving targets in the detection area at the next time point. For example, if the estimated data is obtained at time point Tn, then the estimated data represents the estimated motion trajectory of all moving targets at time point Tn+1 predicted at time point Tn. Therefore, the estimated data at the current time point represents the estimated motion trajectory of all moving targets predicted at the current time point at the next time point, and the estimated data at the previous time point represents the estimated motion trajectory of all moving targets predicted at the previous time point at the current time point. The moving targets in this embodiment represent pedestrians.

[0065] In step 120, the estimated data at the current time point is obtained by combining the detection data at the current time point and the estimated data at the previous time point, wherein the detection data includes the motion trajectories of all moving targets in the detection area, and the detection data is obtained based on the video data.

[0066] For the convenience of description, the parameters in this embodiment are set as follows:

[0067] 1. Assume that the detection area is a rectangular box of x*y, and (0, 0) is the origin of the coordinate system;

[0068] 2. Let the set S t (m,n) represents the set of moving targets detected at time t (i.e., detection data), m represents the number of moving targets, n is a fixed value of 4, and represents the 4-dimensional data of each moving target, recorded as (px,py,vx,vy), where (px,py) represents the position component, and (vx,vy) represents the velocity component. In this embodiment, it is assumed that the moving target moves at a uniform speed over a period of time;

[0069] 3. Let the area where the position component px>x of the moving target (x is the right boundary of the target detection area x*y) be Area1;

[0070] Let the area with position component px<0 (x=0 is the left boundary of target detection area x*y) be Area2;

[0071] 4. Let X′ t (m,n) is the estimated data at time point t. The definitions of m and n are the same as those in variable convention 2.

[0072] In this embodiment, the detection data represents the motion trajectory of all moving targets detected in the detection area at the current time point, and the estimated data at the previous time point represents the estimated motion trajectory of all moving targets at the current time point predicted at the previous time point. When the two are combined, the estimated data at the previous time point for the current time point can be corrected according to the detection data, and then the estimated data at the previous time point with the correction can be combined to obtain the estimated data at the current time point.

[0073] Among them, Figure 2 As shown in the figure, the specific process of obtaining the estimated data at the current time point includes:

[0074] S121: Processing the estimated data at the previous time point through a Kalman filter to obtain initialization data;

[0075] S122: removing the moving targets and the corresponding estimated motion trajectories that are not located in the detection area in the initialization data to obtain intermediate data;

[0076] S123: Match the moving target in the detection data at the current time point with the moving target in the intermediate data, and obtain the estimated data at the current time point based on the matching result.

[0077] In this embodiment, the Kalman filter is based on the Kalman filter algorithm, and the core of the Kalman filter algorithm is prediction and update.

[0078] In step S121, the estimated data at the previous time point is processed by a Kalman filter to form initialization data, and in step S122, the moving targets and corresponding motion estimation trajectories that are not located in the detection area in the initialization data are removed to obtain intermediate data. In step S123, the initialization data is corrected by the detection data at the current time point.

[0079] In this embodiment, Figure 3 As shown, S123 specifically includes:

[0080] S1231: Processing intermediate data through a Kalman filter;

[0081] S1232: Determine whether there is a newly added moving target;

[0082] S1233: If it does not exist, the intermediate data processed by the Kalman filter is the estimated data at the current time point;

[0083] S1234: If yes, obtain the newly added moving target and its moving trajectory in the detection data at the current time point to obtain the newly added data;

[0084] S1235: Combine the intermediate data and the newly added data processed by the Kalman filter to obtain the estimated data at the current time point.

[0085] In this embodiment, Figure 4 As shown, steps S121-S123 are further explained.

[0086] Figure 4 The prediction analysis process from time t1 to time tN is analyzed, where the green arrows represent the prediction and update operations of the Kalman filter, which are detailed as follows:

[0087] 1. When t = 1,

[0088] 1. Obtain the detection data S at time t1 1 (8,4), the formula means that 8 targets are detected, and the position and velocity of each target are 4-dimensional data (px, py, vx, vy);

[0089] 2. Since this is the first set of data, the optimal estimated data cannot be estimated from the previous moment, so the current moment's detection data S is used 1 (8,4) for X′ t (m,n) is initialized to obtain the optimal estimated data X′ at time t1 1 (8,4);

[0090] The detection targets at time t=1 are numbered sequentially.

[0091] 2. When t = 2,

[0092] 1. Get the current detection data S 2 (6,4), the formula means that 6 targets are detected.

[0093] 2. According to the principle of Kalman filter algorithm, the optimal estimated data X′(8,4) at time t1 is used to obtain the estimated data X″(8,4) at the current time.

[0094] 3. Traverse the estimated data X″(8,4)

[0095] If the position component px>x of the target, it is determined that the target moves out of the detection area toward Area1;

[0096] Similarly, if px<0, it is determined that the target moves out of the detection area toward Area2.

[0097] At the same time, it is necessary to remove the targets that leave the detection area from the estimated data. Figure 4 As shown, targets 1, 3, 7, and 8 are removed, and the effective estimated data X″ is obtained. _ (4,4).

[0098] 4. According to the Hungarian algorithm, the detection data S 2 (6,4) and effective estimated data X″ _ (4,4) Perform target pair matching;

[0099] For the matching data, the Kalman filter algorithm is used to update the optimal estimated data X′(4,4) at the current moment;

[0100] There are 2 unmatched data (i.e., 2 new targets). For the new targets, initialize them with the detection data at the current moment and number them to obtain the current optimal estimated data X′(2,4) of the new targets;

[0101] 5. Combine the two parts of data to obtain the optimal estimated data X' at the current moment 2 (6,4).

[0102] The optimal estimated data at the current moment is used as input and the iteration is continued.

[0103] In this embodiment, the Kalman filter algorithm mainly includes two steps: prediction (Predict) and update (Update).

[0104] 1. Prediction steps:

[0105] ① Based on the current state and control input, predict the state at the next moment.

[0106] ② Simultaneously predict the estimated uncertainty (covariance) of the current state.

[0107] 2. Update steps:

[0108] ① Use new measurement data to update the prediction.

[0109] ② Calculate a gain (Kalman gain), which determines which part of the prediction and measurement is more reliable.

[0110] ③Update state estimation and covariance estimation.

[0111] In this embodiment, the detection data and the effective estimated data of each frame need to be matched. This can be regarded as a bipartite graph matching problem, in which the detection data and the estimated data are respectively regarded as two independent subsets of the bipartite graph, and the matching represents the corresponding relationship between the two.

[0112] The core of the Hungarian algorithm is to find augmenting paths. In the bipartite graph matching problem, the algorithm continuously searches for and expands augmenting paths to find the maximum matching of the bipartite graph.

[0113] 1. Bipartite graph and matching:

[0114] A bipartite graph is a special graph structure whose nodes can be divided into two disjoint subsets, and each edge in the graph connects two nodes belonging to different subsets.

[0115] A matching is a subset of edges in a bipartite graph where no two edges have common vertices.

[0116] 2. Augmenting Path:

[0117] An augmenting path is an important concept in bipartite graph matching. It is a path that starts from a non-matching point, passes through a series of matching edges and non-matching edges that appear alternately, and finally reaches another non-matching point. The first and last nodes of an augmenting path are non-matching points, and the remaining nodes are alternately matching points and non-matching points. In an augmenting path, the number of non-matching edges is always one more than the number of matching edges.

[0118] 3. Algorithm steps:

[0119] ①Initialize an empty match.

[0120] ②Continuously search for augmenting paths, and replace the matching edges and non-matching edges in the augmenting paths through XOR operations (or other methods) to obtain a larger match.

[0121] ③ Repeat the above steps until no new augmenting path is found. At this point, the matching obtained is the maximum matching of the bipartite graph.

[0122] 3. When t = N,

[0123] 1. Get the current detection data S N (2,4), the formula means that two targets are detected.

[0124] 2. According to the principle of Kalman filter algorithm, the estimated data X″(5,4) at the current moment is obtained from the optimal estimated data X′(5,4) at the moment tN-1;

[0125] 3. According to the rules in variable convention 3, traverse the estimated data X″(5,4):

[0126] If the position component px>x of the target, it is determined that the target moves out of the detection area toward Area1;

[0127] Similarly, if px<0, it is determined that the target moves out of the detection area toward Area2.

[0128] At the same time, it is necessary to remove the targets that leave the detection area from the estimated data. Figure 4 As shown, targets 12, 13, and 14 are removed, and the effective estimated data X″ is obtained. _ (2,4)

[0129] 4. According to the Hungarian algorithm, the detection data S N (2,4) and effective estimated data X″ _ (2,4) to match the target pair. For the matching data, the Kalman filter algorithm is used to update the optimal estimated data X′(2,4) at the current moment. The number of unmatched data is 0 (i.e., no new targets are added), so the optimal estimated data at the current moment is X′ N (2,4).

[0130] This embodiment is based on the core idea of ​​the Kalman filter algorithm: prediction and update. The present invention obtains the position of the moving target at time tm through the above five steps, and can accurately deduce and estimate the position of the moving target at the next time tm+1. Recursively, the motion trajectory of the moving target within the period from tm to tn (n>m) can be obtained.

[0131] In summary, at any time point, the estimated trajectory of the moving target at the next time point can be obtained. By traversing the estimated trajectory of the moving target at the next time point at the current time point, it is determined whether there are still pedestrians in the detection area. If not, the vehicle can enter the detection area in advance and pass through the detection area safely. Specifically, for example, at time tn, there are pedestrians in the detection area. According to the estimated time tn+1, there are no pedestrians in the detection area. Then the motor vehicle can start at time tn and pass through the detection area (control the vehicle speed in combination with the pedestrian's movement speed) without having to wait until time tn+1 to pass.

[0132] Therefore, in S130, the vehicle's traffic strategy in the detection area at the current time point is determined based on the estimated data at the current time point. The strategy is that when the estimated data obtained at the current time point indicates that there will be no pedestrians in the detection area at the next moment, the vehicle starts at the current time point, and the vehicle speed is set based on the motion trajectory of the moving target in the detection data at the current time point.

[0133] The method flow of acquiring the moving target in S120 of this embodiment is as follows:

[0134] ① Extract several frames of image data based on video data;

[0135] ② Binarize the frame image data;

[0136] ③Get the moving target in the frame image data.

[0137] Wherein in step ②, the current frame image is binarized to obtain the moving target in the current frame image, including: processing the frame image data through the background difference strategy to obtain a first binary image; processing the frame image data through the three-frame difference method to obtain a second binary image; combining the first binary image and the second binary image, and obtaining the moving target based on the combined image.

[0138] The traditional inter-frame difference method has the advantages of simple algorithm implementation, low sensitivity to scene changes such as light, and good stability. However, it cannot extract the complete area of ​​the object (it can only extract the boundary) and is prone to causing holes and breakpoints. The advantages of the background difference method are high real-time performance and the ability to extract the complete foreground image, but it also has the disadvantages of being sensitive to changes in lighting and environment.

[0139] Combining the advantages and disadvantages of the inter-frame difference method and the background difference method, this embodiment combines the two. On the one hand, based on the traditional three-frame difference method, the two difference images are accumulated, and the accumulated result is thresholded, which can effectively suppress the noise generated by the difference; on the other hand, the foreground binary image obtained by the background difference method and the binary image obtained by the improved three-frame difference method are ORed, and then the moving target result obtained by binarization can increase the weight of the target information, make the target contour more accurate and clear, and also effectively suppress the noise generated by the static background.

[0140] In this embodiment, for step ①, the video is read in, the video is converted into images with one frame as an interval, and the images are gray-scale processed in sequence to obtain a pre-processed video sequence.

[0141] In step ②, the moving target is extracted by background difference method, specifically:

[0142] Considering that the background in the application scenario does not change much, we can pre-extract one or several frames of images without foreground moving objects from the video sequence in step ① as the background frame, which has the following formula:

[0143]

[0144] Among them, B n (x,y) represents the background frame, image(x,y) represents each frame of the image, and N is the number of selected image frames.

[0145] Let the current frame image be F n (x,y), combined with the current background frame B n(x,y), get the foreground binary image DB n (x,y) is as follows,

[0146]

[0147] In the formula, the threshold T is set, and the pixels are binarized one by one to obtain a binary image, in which the points with a gray value of 1 represent the foreground (moving target) points, and the points with a gray value of 0 represent the background points.

[0148] ②In this paper, the improved three-frame difference method is used to extract the moving target

[0149] Assume that the three adjacent frames are F n-1 (x,y), F n (x,y), F n+1 (x,y), calculate the difference image between two adjacent frames:

[0150] DF n (x,y)=F n (x,y)-F n-1 (x,y)

[0151] DF n+1 (x,y)=F n+1 (x,y)-F n (x,y)

[0152] The simulation results show that the noise generated by the difference can be effectively suppressed by first accumulating the two differential images and then thresholding the accumulated results.

[0153]

[0154] The accumulated DM n (x,y) is binarized to obtain the difference binary image as follows:

[0155]

[0156] Wherein, the threshold T (<=255) is obtained by an adaptive threshold segmentation method.

[0157] (4) Improved moving target extraction method

[0158] The foreground binary image DB obtained by the background difference method n (x, y) and the binary image D obtained by the improved three-frame difference method n (x,y) OR operation

[0159]

[0160] Where M n(x, y) is the desired moving target image, where the point with a gray value of 1 represents the foreground (moving target) point, and the point with a gray value of 0 represents the background point. Simulations show that this improved moving target extraction method can quickly and effectively extract moving personnel in complex environments with an accuracy rate of more than 95%.

[0161] (5) Mathematical morphology filtering

[0162] After completing the binarization of the image, the outline of the foreground image (pedestrian) can be roughly displayed. However, due to noise, light reflection, and the gray value of the pedestrian is too close to the gray value of the environment, the foreground image inevitably has horizontal and vertical faults, the edge of the target is not very smooth, and there will be more or less holes inside the target. Therefore, it is necessary to remove interference, noise and differences from the binary image to make the pedestrian outline more obvious. There are many commonly used filtering algorithms, such as mean filtering, median filtering, Gaussian filtering, mathematical morphology filtering, etc. Different filtering algorithms have their own filtering effects on different noises. Since the noise of the differential image is randomly distributed, combined with the characteristics of morphological filtering that has a good removal effect on random noise, this system chooses mathematical morphology filtering to remove noise.

[0163] 5.1 Open Operation

[0164] For the target binary image M obtained in step (4) above n (x, y) is opened (i.e., corrosion is performed first, followed by dilation) to remove isolated dots, burrs, and bridges (dots connecting two areas) in the image. The specific steps are as follows:

[0165] Let A be the target binary image set, B be the structural element, Θ be the corrosion operator, is the dilation operator, (xy) represents the pixel of image A, and Φ is the empty set.

[0166] 5.1.1 Corrosion Operation

[0167] AΘB is recorded as A being corroded by B, and has the following formula:

[0168]

[0169] The formula means that an origin is defined in B. When the origin of B is translated to the pixel (x, y) of image A, if B at (x, y) is completely contained in the overlapping area of ​​image A, that is, the corresponding values ​​of image A at the element positions of B that are 1 are all 1, then the pixel (x, y) corresponding to the output image is assigned 1, otherwise it is assigned 0.

[0170] 5.1.2 Dilation Operation

[0171] It is recorded as A is expanded by B, and the formula is as follows:

[0172]

[0173] The formula means translating the origin of the structural element B to the image pixel (x, y). If the intersection of B and A at the image pixel (x, y) is not empty, that is, at least one of the image values ​​of A corresponding to the element position of 1 in B is 1, then the corresponding pixel (x, y) of the output image is assigned 1, otherwise it is assigned 0.

[0174] After completing the mathematical morphology filtering process, a clearer moving target contour can be obtained. We take the center point coordinates (px, py) of the moving target contour to represent its position component.

[0175] In the scenario where an intelligent driving motor vehicle is waiting for and passing through a pedestrian zebra crossing, this embodiment adopts a pure visual solution to extract the foreground target in the target area based on the real-time video data collected by the on-board video acquisition device, and then estimates the motion trajectory of the foreground target over a period of time in combination with relevant algorithms, and finally achieves the estimation of the motion target trajectory at the next moment at the current moment, so as to make a judgment in advance on the passability of the intelligent driving motor vehicle to the target area at the current moment, and maximize the vehicle passing rate of the pedestrian zebra crossing while ensuring safety.

[0176] Example 2

[0177] Please refer to Figure 5 The present application also provides a device for detecting passage in an area, based on the method for detecting passage in an area of ​​embodiment 1.

[0178] The access device of the detection area includes at least one software function module which can be stored in the storage module in the form of software or firmware or fixed in the operating system (OS). The processing module is used to execute the executable module stored in the storage module, such as the software function module and computer program included in the voice message device.

[0179] The passage device of the detection area includes an acquisition module 1, a calculation module 2 and a driving module 3, and the functions of each module may be as follows:

[0180] The acquisition module 1 is configured to acquire video data of the detection area;

[0181] The calculation module 2 is configured to obtain the estimated data of the current time point in the detection area, and the estimated data represents the estimated motion trajectory of all moving targets in the detection area at the next time point;

[0182] The driving module 3 is configured to determine the traffic strategy of the vehicle in the detection area at the current time point according to the estimated data at the current time point.

[0183] This embodiment also includes a visual processing module 4 configured to obtain a moving target of frame image data of video data.

[0184] The calculation module 2 of this embodiment combines the detection data at the current time point and the estimated data at the previous time point to obtain the estimated data at the current time point, wherein the detection data includes the motion trajectories of all moving targets in the detection area, and the detection data is obtained based on the video data.

[0185] The calculation module 2 is specifically obtained through the following logic:

[0186] Combine the detection data at the current time point with the estimated data at the previous time point to obtain the estimated data at the current time point, including:

[0187] The estimated data at the previous time point is processed by the Kalman filter to obtain the initialization data;

[0188] Remove the moving targets and the corresponding estimated motion trajectories that are not located in the detection area in the initialization data to obtain intermediate data;

[0189] The moving target in the detection data at the current time point is matched with the moving target in the intermediate data, and based on the matching result, the estimated data at the current time point is obtained.

[0190] Among them, the moving target in the detection data at the current time point is matched with the moving target in the intermediate data based on the Hungarian algorithm, and the estimated data at the current time point is obtained based on the matching result, including:

[0191] Process the intermediate data through a Kalman filter;

[0192] Determine whether there is a new sports target;

[0193] If it does not exist, the intermediate data processed by the Kalman filter is the estimated data at the current time point;

[0194] If it exists, the newly added moving target and its motion trajectory in the detection data at the current time point are obtained to obtain the newly added data; the intermediate data and the newly added data processed by the Kalman filter are combined to obtain the estimated data at the current time point.

[0195] The visual processing module 4 obtains the moving target of the frame image data of the video data based on the following method.

[0196] Extracting a number of frame image data according to the video data;

[0197] Binarize the frame image data;

[0198] Get the moving target in the frame image data.

[0199] The current frame image is binarized to obtain the moving targets in the current frame image including:

[0200] Processing the frame image data by a background difference strategy to obtain a first binary image;

[0201] Processing the frame image data by a three-frame difference method to obtain a second binary image;

[0202] The first binarized image and the second binarized image are combined, and a moving object is obtained according to the combined image.

[0203] After the frame image data is binarized, the binarized frame image data is dilated and / or eroded.

[0204] Example 3

[0205] This embodiment proposes a server, which includes a processor and a memory coupled to each other, wherein a computer program is stored in the memory. When the computer program is executed by the processor, the server executes the method for passing through the detection area as described in Embodiment 1.

[0206] In this embodiment, the processor may be an integrated circuit chip having the ability to process signals. For example, the processor may be a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and may implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application.

[0207] The memory is not limited to random access memory, read-only memory, programmable read-only memory, erasable programmable read-only memory, electrically erasable programmable read-only memory, etc. Of course, the memory can also be used to store a program, and the processor executes the program after receiving an execution instruction.

[0208] Example 4

[0209] The embodiment of the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is run on a computer, the computer executes the method described in Embodiment 1.

[0210] Example 5

[0211] An embodiment of the present application further provides a computer program product, including a computer program, which implements the method described in Embodiment 1 when executed by a processor.

[0212] Through the description of the above implementation methods, technical personnel in this field can clearly understand that the present application can be implemented through hardware, and can also be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various implementation scenarios of the present application.

[0213] In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method can also be implemented in other ways. The apparatus and method embodiments described above are merely schematic, for example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a part of a module, a program segment or a code, and a part of a module, a program segment or a code includes one or more executable instructions for implementing a specified logical function. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented by a dedicated hardware-based system that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. In addition, each functional module in each embodiment of the present application can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part.

[0214] The above are only embodiments of the present application and are not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for passing through a detection area, characterized in that: The method comprises: Obtain video data of the detection area; Obtaining estimated data of a current time point in the detection area according to the video data, wherein the estimated data represents an estimated motion trajectory of all moving objects in the detection area at a next time point; A strategy for the vehicle to pass through the detection area at the current time point is determined based on the estimated data at the current time point.

2. The method according to claim 1, characterized in that The estimated data obtained at the current time point includes: The estimated data at the current time point is obtained by combining the detection data at the current time point with the estimated data at the previous time point, wherein the detection data includes the motion trajectories of all moving targets in the detection area.

3. The method according to claim 2, characterized in that The combining the detection data at the current time point and the estimated data at the previous time point to obtain the estimated data at the current time point includes: Processing the estimated data at the previous time point through a Kalman filter to obtain initialization data; Removing the moving targets and the corresponding estimated motion trajectories that are not located in the detection area from the initialization data to obtain intermediate data; The moving target in the detection data at the current time point is matched with the moving target in the intermediate data, and based on the matching result, the estimated data at the current time point is obtained.

4. The method according to claim 3, characterized in that The matching of the moving target in the detection data at the current time point with the moving target in the intermediate data, and obtaining the estimated data at the current time point based on the matching result, includes: Process the intermediate data through a Kalman filter; Determine whether there is a new sports target; If it does not exist, the intermediate data processed by the Kalman filter is the estimated data at the current time point; If so, the newly added moving target and its motion trajectory in the detection data at the current time point are obtained to obtain the newly added data; the intermediate data processed by the Kalman filter and the newly added data are combined to obtain the estimated data at the current time point.

5. The method according to claim 3, characterized in that: The matching of the moving target in the detection data at the current time point with the moving target in the intermediate data includes: The moving targets in the detection data at the current time point are matched with the moving targets in the intermediate data based on the Hungarian algorithm.

6. The method according to claim 2, characterized in that The strategy for determining that the vehicle passes through the detection area at the current time point according to the estimated data at the current time point includes: According to the estimated data at the current time point, if there is no moving target in the detection area at the next time point, the vehicle starts to start at the current time point and passes through the detection area, and the passing speed of the detection area is set based on the motion trajectory of the moving target in the detection data at the current time point.

7. The method according to claim 1, characterized in that After obtaining the video data of the detection area, the method further includes: Extracting a plurality of frame image data from the video data; The frame image data is binarized to obtain a moving target in the frame image data.

8. The method according to claim 7, characterized in that The binarization process of the current frame image to obtain the moving target in the current frame image includes: Processing the frame image data by a background difference strategy to obtain a first binary image; Processing the frame image data by a three-frame difference method to obtain a second binary image; The first binarized image and the second binarized image are combined, and a moving object is obtained according to the combined image.

9. The method according to claim 7, characterized in that: After binarizing the frame image data, the method further includes: performing dilation processing and / or erosion processing on the binarized frame image data.

10. A passage device for detecting an area, characterized in that: include: An acquisition module configured to acquire video data of a detection area; A calculation module, configured to obtain estimated data at a current time point in the detection area, wherein the estimated data represents the estimated motion trajectory of all moving targets in the detection area at a next time point; The driving module is configured to determine a strategy for the vehicle to pass through the detection area at the current time point according to the estimated data at the current time point.

11. A server, characterized in that: The server comprises a processor and a memory coupled to each other, wherein a computer program is stored in the memory. When the computer program is executed by the processor, the server executes the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 9.

13. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images