Moving object detection method, detection device and electronic equipment

Abstract

The embodiments of the present application provide a method, a detection device and an electronic device for detecting a moving object, wherein the device comprises: a first processing unit, which pairs a target object in each frame image with a target object corresponding to the same object in other predetermined frame images for a first number of frames of images to form a target object pair; a second processing unit, which generates a directed acyclic graph based on the target object pairs, wherein the directed acyclic graph comprises nodes and edges connecting different nodes, the nodes represent each target object in the target object pair, the nodes are arranged in the order of each frame image, two nodes connected by an edge represent a target object pair, and the direction of the edge is from the target object in the previous frame image to the target object in the next frame image; a third processing unit, which extracts chains from the directed acyclic graph, in which at most one node is provided per frame image in each chain; and a fourth processing unit, which detects moving objects with a movement speed within a certain range based on the extracted chains.

Description

Technical Field

[0001] The present application relates to the field of electronic information technology. Background Art

[0002] Expressways and urban roads are the main routes of road traffic, which are characterized by high speed and heavy traffic. Once a certain traffic incident occurs, it is very likely to cause a traffic accident and seriously affect the load capacity and traffic efficiency of the road. In daily traffic management, relying solely on manual detection of traffic incidents is time-consuming and inefficient, and it is impossible to quickly handle abnormal behavior when it occurs.

[0003] The traditional road vehicle detection method analyzes traffic events by collecting traffic data generated by detectors installed on the road. This method has the advantages of low cost and easy operation, but also has the disadvantages of low detection rate, high false detection and lack of visualization.

[0004] With the development of information technology, traffic accident detection methods based on video images have made great progress. This method uses images captured by cameras installed on roads or tunnels as input, and through the processing and analysis of video images, it realizes automatic detection and tracking of vehicles, and then detects abnormal traffic behaviors such as low-speed driving.

[0005] It should be noted that the above introduction to the technical background is only for the convenience of providing a clear and complete description of the technical solutions of the present application and for the convenience of understanding by those skilled in the art. It cannot be considered that the above technical solutions are well known to those skilled in the art simply because they are described in the background technology section of the present application. Summary of the invention

[0006] Many traffic events, such as traffic jams and traffic accidents, can lead to low-speed driving events. Vehicles driving at low speeds usually indicate the occurrence of abnormal traffic events. In order to handle abnormal traffic behaviors in a timely manner, it is very necessary to quickly detect low-speed driving events and give alarms.

[0007] The inventors of the present application have discovered that there is still room for further improvement in the existing moving vehicle detection method based on video images. For example, the missed detection rate of moving vehicle detection can be further reduced.

[0008] The embodiments of the present application provide a moving object detection method, a detection device, and an electronic device. In the moving object detection method, the same target object in different images is detected and represented in a directed acyclic graph. The moving object is detected based on the directed acyclic graph, thereby reducing the missed detection rate of the moving object. The moving object detection method can be applied to, for example, detecting a moving vehicle, etc.

[0009] According to a first aspect of an embodiment of the present application, there is provided a moving object detection device, comprising:

[0010] A first processing unit, for a first number of frames of images, pairs a target object in each frame of image with a target object corresponding to the same object in other predetermined frames of image to form a target object pair;

[0011] a second processing unit, which generates a directed acyclic graph according to the target object pair, wherein the directed acyclic graph includes nodes and edges connecting different nodes, the nodes represent each target object in a target object pair, the nodes are arranged in the order of each frame image, two nodes connected by one edge represent one target object pair, and the direction of the edge is from the target object in the previous frame image to the target object in the next frame image;

[0012] a third processing unit, which extracts distinct chains from the directed acyclic graph, wherein each chain includes the nodes and the edges, and in each chain, at most one node is provided per frame image; and

[0013] A fourth processing unit detects a moving object with a moving speed within a certain range based on the extracted chain.

[0014] According to a second aspect of an embodiment of the present application, a method for detecting a moving object is provided, comprising:

[0015] For the first number of frames of images, pairing the target in each frame of image with the target corresponding to the same object in other predetermined frames of image to form a target pair;

[0016] Generate a directed acyclic graph according to the target object pair, wherein the directed acyclic graph includes nodes and edges connecting different nodes, the nodes represent each target object in the target object pair, the nodes are arranged in the order of each frame image, two nodes connected by one edge represent one target object pair, and the direction of the edge is from the target object in the previous frame image to the target object in the next frame image;

[0017] Extracting distinct chains from the directed acyclic graph, wherein each chain includes the nodes and the edges, and in each chain, each frame image provides at most one node; and

[0018] Based on the extracted chain, a moving object with a moving speed within a certain range is detected.

[0019] According to a third aspect of an embodiment of the present application, an electronic device is provided, which has the moving object detection device described in the first aspect of the embodiment.

[0020] The beneficial effect of the present application is that it can reduce the missed detection rate of moving object detection and improve the accuracy of detection.

[0021] With reference to the following description and accompanying drawings, the specific implementation of the application is disclosed in detail, indicating the way in which the principle of the application can be adopted. It should be understood that the implementation of the application is not limited in scope. Within the scope of the spirit and clauses of the attached notes, the implementation of the application includes many changes, modifications and equivalents.

[0022] Features described and / or illustrated with respect to one embodiment may be used in the same or similar manner in one or more other embodiments, combined with features in other embodiments, or substituted for features in other embodiments.

[0023] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, integers, steps or components, but does not exclude the presence or addition of one or more other features, integers, steps or components. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The included drawings are used to provide a further understanding of the embodiments of the present application, which constitute a part of the specification, are used to illustrate the implementation methods of the present application, and together with the text description, explain the principles of the present invention. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0025] Figure 1 is a schematic diagram of a moving object detection method according to the first aspect of an embodiment of the present application;

[0026] Figure 2 is a schematic diagram of multiple frame images in a time series of the first aspect of the embodiment of the present application;

[0027] Figure 3 is a schematic diagram of a method for comparing a pair of image frames to obtain a target object pair in operation 101;

[0028] Figure 4 It is a schematic diagram of the correction of the border;

[0029] Figure 5 is a schematic diagram of the directed acyclic graph of the present application;

[0030] Figure 6 It is a schematic diagram of the operation of the extraction chain;

[0031] Figure 7 is a schematic diagram of a chain obtained based on a directed acyclic graph in the first aspect of the embodiment of the present application;

[0032] Figure 8 It is a schematic diagram showing that the angle between two adjacent edges in the chain exceeds a predetermined angle threshold;

[0033] Fig. 9 is a schematic diagram of a moving object detection device 900 according to the second aspect of an embodiment of the present application;

[0034] Fig.10 It is a schematic diagram of the structure of an electronic device according to the third aspect of an embodiment of the present application. DETAILED DESCRIPTION

[0035] With reference to the accompanying drawings, the above and other features of the present application will become apparent through the following description. In the description and the accompanying drawings, specific embodiments of the present application are specifically disclosed, which show some embodiments in which the principles of the present application can be adopted. It should be understood that the present application is not limited to the described embodiments. On the contrary, the present application includes all modifications, variations and equivalents falling within the scope of the attached notes.

[0036] In the embodiments of the present application, the terms "first", "second", etc. are used to distinguish different elements in terms of title, but do not indicate the spatial arrangement or temporal order of these elements, etc., and these elements should not be limited by these terms. The term "and / or" includes any one and all combinations of one or more of the associated listed terms. The terms "comprising", "including", "having", etc. refer to the presence of the stated features, elements, components or components, but do not exclude the presence or addition of one or more other features, elements, components or components.

[0037] In the embodiments of the present application, the singular forms "a", "the", etc. include plural forms and should be broadly understood as "a kind" or "a type" rather than being limited to the meaning of "one"; in addition, the term "the" should be understood to include both singular and plural forms, unless the context clearly indicates otherwise. In addition, the term "according to" should be understood as "at least in part according to...", and the term "based on" should be understood as "at least in part based on...", unless the context clearly indicates otherwise.

[0038] First aspect of the embodiment

[0039] A first aspect of an embodiment of the present application provides a method for detecting a moving object. Figure 1 is a schematic diagram of a moving object detection method according to the first aspect of an embodiment of the present application. Figure 1 As shown, the method includes the following operations:

[0040] Operation 101, for a first number (m) of frames of images that are continuous in time, pairing a target in each frame of image with a target corresponding to the same object in other predetermined frames of image to form a target pair;

[0041] Operation 102: generating a directed acyclic graph according to the target object pair, wherein the directed acyclic graph includes nodes and edges connecting different nodes, the nodes represent each target object in a target object pair, the nodes are arranged in the order of each frame image, two nodes connected by an edge represent one target object pair, and the direction of the edge is from the target object in the previous frame image to the target object in the next frame image;

[0042] Operation 103: extracting chains from the directed acyclic graph, wherein each chain includes the nodes and the edges, and in each chain, each frame image provides at most one node; and

[0043] Operation 104 : Detect moving objects with a moving speed within a certain range based on the extracted chain.

[0044] According to the first aspect of the embodiment of the present application, in the moving object detection method, the same target object in different images is detected and represented in a directed acyclic graph. Since the directed acyclic graph can clearly represent the relationship between pairs of targets in multiple frames of images and the movement direction of the same target object, therefore, detecting moving objects based on the directed acyclic graph can reduce missed detections of moving objects. In addition, the speed and efficiency of detecting moving objects can be improved.

[0045] In operation 101, a first number of frame images in a time series may be processed to form a target object pair. The first number of frame images in a time series may be continuous images in a time series, or may be multiple frame images extracted from continuous frame images in a time series.

[0046] On the multiple frames of images, each frame of the image can be marked with the type of the object in the frame of the image and the bounding box of the object. The type and bounding box of the object can be obtained by detecting the image using a classifier based on deep learning. For the specific method of detecting the image by the classifier, reference can be made to the relevant technology. In addition, the multiple frames of images and the information of the type and bounding box of the object in each frame of the image can be pre-stored.

[0047] In at least one embodiment, the target object may be, for example, a vehicle, and the type of the target object may be, for example, a sedan, a hatchback, a truck, a bus, etc. In the following of this application, the parking detection method will be described by taking the target object being a vehicle as an example. In addition, when the target object is of other types, the detection method of this application is also applicable to detecting the movement of the other types of targets. For example, when the target object is a human body, the detection method of the moving object of this application can be used to detect a person in motion, so as to facilitate the determination of crowd congestion and the like.

[0048] Figure 2 It is a schematic diagram of multiple frame images in a time series according to the first aspect of an embodiment of the present application. Figure 2 Six frames of images are shown, and the indexes of the frame images are I0, I1, I2, I3, I4, and I5, respectively. From I0 to I5, the corresponding time is delayed successively.

[0049] like Figure 2 As shown, in each frame image, the detected target objects are marked as 0, 1, ..., etc. For example, for image I2, there are 2 targets, which are marked as 0 and 1 respectively.

[0050] In operation 101, for each frame image, the frame image and other predetermined frame images may be compared respectively, and in each comparison, a target object pair in the two frames for comparison is detected. The other predetermined frame images may be images that differ from the frame image by a predetermined number of frames in time sequence, and the predetermined number may be represented by m, which is a natural number greater than or equal to 2.

[0051] For example, if m is equal to 2, Figure 2 Taking the images I0, I1, I2, and I3 shown in the figure as an example, in operation 101, for I0, since I0 and I1 differ by 1 frame, and I0 and I2 differ by 2 frames, that is, I1, I2 and I0 differ by less than 2 frames in time series, I1 and I2 are other predetermined frame images relative to I0, therefore, I0 and I1 are compared to obtain a target object pair, and I0 and I2 are compared to obtain a target object pair; similarly, for I1, I1 and I2 are compared to obtain a target object pair, and I1 and I3 are compared to obtain a target object pair; for I2, I2 and I3 are compared to obtain a target object pair. In addition, for Figure 2 The same method can also be used to determine each pair of image frames for comparison.

[0052] In addition, Figure 2 In the figure, the arrow on the target object indicates the moving direction of the target object in the image.

[0053] Figure 3is a schematic diagram of a method for comparing a pair of image frames in operation 101 to obtain a pair of target objects, such as Figure 3 As shown, operation 101 may include the following operations:

[0054] Operation 301, calculate the distance between a target object (A) in the current frame image and each target object (B) corresponding to the target object (A) in another frame image. If the distance is in a first interval ([threshold_inner, threshold_outer]), pair the target object in the other frame image with the target object in the current frame image as a target pair.

[0055] In at least one embodiment, the target object A corresponds to the target object B, and the type of the target object A and the type of the target object B may be the same, such as the same vehicle type, or other parameters of the target object A and the target object B may be the same or similar.

[0056] For example, the current frame may be image I0, and the other frame image to be compared with image I0 may be image I1. In image I0, there are M objects. For the i-th object, its type may be marked as Ti, and its bounding box A i can be expressed as [x i ,y i ,w i ,h i ], where (x i ,y i ) represents the bounding box A i The coordinates of the center point in the width and height directions of the image, (w i ,h i ) represents the bounding box A i The width and height of , i is a natural number, 1≤i≤M;

[0057] In the image I1, there are, for example, Ni objects of type Ti. Among the Ni objects, the bounding box B of the jth object j can be expressed as [x j ,y j ,w j ,h j ], where (x j ,y j ) represents the bounding box B j The coordinates of the center point in the width and height directions of the image, (w j ,h j ) represents the bounding box B j The width and height of , j is a natural number, 1≤j≤Ni;

[0058] The Euclidean distance between the i-th object in image I0 and each of the Ni objects in image I1 is represented by the vector d i =[d i1 , d i2 , …, d ij , …, d iNi ], where d ij It represents the Euclidean distance between the i-th object in image I0 and the j-th object among Ni objects in image I1.

[0059] In operation 301, for the i-th object in image I0, find the vector d i If the elements in the first interval [threshold_inner, threshold_outer] are in the vector d i If only one element in is located in the first interval, the target object in image I1 corresponding to the element and the i-th target object in image I0 are taken as a target object pair.

[0060] If more than two elements in the vector di are located in the first interval, more than two objects in the image I1 corresponding to the more than two elements and the i-th object in the image I0 form candidate object pairs respectively.

[0061] like Figure 3 As shown, operation 101 may also include the following operations:

[0062] Operation 302 : Calculate the distance between the appearance features of two targets in each candidate target pair in the feature space, and use the candidate target pair with the smallest distance in the feature space and a distance not greater than a feature distance threshold as the target pair.

[0063] For example, if three elements of the vector di are located in the first interval, the three targets in image I1 corresponding to the three elements and the i-th target in image I0 form candidate target pairs p1, p2, and p3 respectively, and the distances of each candidate target pair in the feature space are dp1, dp2, and dp3 respectively. If dp2 is the smallest and dp2 is less than or equal to (i.e., not greater than) the feature distance threshold Tz, then the candidate target pair p2 is taken as the i-th target in image I0 to form a target pair with image I1; in addition, if dp2 is the smallest, but dp2 is greater than the feature distance threshold Tz, then the candidate target pairs p2, p1, and p3 are not taken as target pairs.

[0064] In operation 302, the distance between the features of the two targets in each candidate target pair in the feature space may be calculated according to a cosine metric learning algorithm. For example, a classifier may be trained according to a cosine metric learning algorithm, and then the trained classifier may be used to extract the feature r of each candidate target in the candidate target pair as its appearance feature, and the distance between the two features r of the candidate target pair in the feature space (i.e., dp1, dp2, dp3, etc.) may be calculated.

[0065] In the present application, the classifier trained by the cosine metric learning algorithm can obtain a lower intra-class distance (low intra-class distance) and a higher inter-class distance (high inter-class distance) compared with the classifier obtained by the traditional training method. Therefore, when calculating the distance between features based on the features extracted by the classifier, the accuracy of the calculation can be improved.

[0066] In the present application, the specific method of calculating the distance between the features of two targets in each candidate target pair in the feature space according to the cosine metric learning algorithm can refer to the relevant technology.

[0067] In addition, the present application may not be limited thereto, and in operation 302 , the distance between the features of the two targets in each candidate target pair in the feature space may also be calculated based on other methods.

[0068] In at least one embodiment, Figure 3 As shown, operation 101 may also include the following operations:

[0069] Operation 303 : Before operation 301 , calibrate the bounding box of each target object in the first number of frames of images to a bounding box in an image with a predetermined resolution.

[0070] After operation 303 , in operation 301 , distance calculation is performed using the size and position of the frame corrected by operation 303 .

[0071] In the present application, the resolutions of the images of the first number of frames may not be the same. Therefore, by correcting the borders in each frame image to the borders in the image of the same predetermined resolution through operation 301, the influence of different resolutions of the images on the distance calculation in operation 301 can be eliminated.

[0072] For example, in operation 303, according to the ratio between the resolution of each frame image and the predetermined resolution, the position of the bottom center point of the frame of the object in the image of the predetermined resolution is adjusted, and the length and width of the frame are adjusted.

[0073] Figure 4 This is a schematic diagram of the border correction. Figure 4 As shown, the resolution of image 401 is: 1080 pixels high and 1920 pixels wide. The resolution of image 402 with a predetermined resolution is: 480 pixels high and 720 pixels wide. After operation 303, the frame 4011 of the target object in image 401 is corrected to the frame 4012 of the target object in image 402; in step 301, the distance between the targets is calculated according to the position and size of the frame 4012 of the target object in the corrected image 402, so as to determine the target object pair or candidate target object pair.

[0074] In the present application, according to operation 101, the target object pairs are detected on the image frames of the first number of frames, and the image frames of the first number of frames are, for example, Figure 2 The six image frames I0 to I5 are shown.

[0075] In operation 102 , a directed acyclic graph may be generated according to the detection result of the target object pair in operation 101 .

[0076] Figure 5 is a schematic diagram of the directed acyclic graph of this application. Figure 5 As shown, the directed acyclic graph 500 includes nodes 501 and edges 502 connecting the nodes, the nodes 501 represent each target object in a target object pair, and the arrangement order of the nodes 501 in the first direction D1 corresponds to the arrangement order of each frame image I0-I5 on the time axis. Figure 5 In the example, two nodes 501 connected by an edge 502 represent an object pair. For example, for the object pair detected in operation 101, the node 501 corresponding to the i-th object in image I0 and the node 501 corresponding to the j-th object in image I1 can be connected by edge 502.

[0077] like Figure 5 As shown, in the directed acyclic graph, edge 502 will not be connected end to end to form a loop, and edge 502 has a direction. The direction of edge 502 is shown by the arrow of edge 502, pointing from the target in the previous frame image to the target in the next frame image, for example, from the target in image I1 to the target in image I2.

[0078] In addition, if Figure 5As shown, in the directed acyclic graph 500, the multiple nodes 501 corresponding to the multiple targets in each image frame can be arranged in the second direction D2 corresponding to the image frame. Figure 5 , the nodes 501 corresponding to the targets 0 and 1 in the image frame I2 can be arranged below the index I1 of the image frame.

[0079] In operation 103 , distinct chains may be extracted from the directed acyclic graph 500 .

[0080] Figure 6 is a schematic diagram of the operation of the extraction chain, such as Figure 6 As shown, the operations of extracting the chain include:

[0081] Operation 601: Select a starting node of a chain, and obtain multiple chains starting from the starting node; and

[0082] Operation 602: If the nodes and edges of the chain are all included in other chains, the chain is deleted. If the start node and end node of more than two chains are the same, the chain containing the largest number of nodes is retained.

[0083] In operation 601 , the selected starting node may be: a node in the image frame that is most forward in time among the nodes that do not belong to any existing chain (ie, among the remaining nodes).

[0084] For example, in operation 601, for Figure 5 In the directed acyclic graph shown in FIG. 1 , the node corresponding to the target object 0 of I0 is first used as the starting node of the chain to obtain multiple chains, each of which may include at least two nodes 501 and at least one edge 502, and the 502 on each chain is not interrupted. For example, the node corresponding to the target object 0I of I0 to the node corresponding to the target object 0 of I1 can form a chain, the node corresponding to the target object 0 of I0 to the node corresponding to the target object 0 of I1 and then to the node corresponding to the target object 0 of I2 can form a chain, the node corresponding to the target object 0 of I0 to the node corresponding to the target object 0 of I1 and then to the node corresponding to the target object 0 of I2 can form a chain, 2 to the node corresponding to the target object 0 of I4 and then to the node corresponding to the target object 1 of I4 can form a chain, and other chains are not listed; after multiple chains are obtained with the node corresponding to the target object 0 of I0 as the starting node of the chain, the node corresponding to the target object 1 of I1, the node corresponding to the target object 1 of I2 and the node corresponding to the target object 1 of I3 are not located on any chain, therefore, the node corresponding to the target object 1 in the image I1 that is the earliest in time is used as a new starting point to obtain at least one chain again; the above operation is repeatedly performed until all chains in the directed acyclic graph are obtained.

[0085] In operation 602, the multiple chains obtained in operation 601 are deleted or retained. For example, if the nodes and edges of a chain are all included in other chains, the chain belongs to the sub-chain of other chains, and thus the chain can be deleted, thereby reducing the number of chains and reducing the complexity of calculation; and / or, if the starting node and the ending node of more than two chains are the same, the chain containing the largest number of nodes is retained, and the other chains in the more than two chains are deleted, so that the retained chain can reflect the motion state of the target object corresponding to more nodes, and thus the detection result is more accurate.

[0086] Figure 7 is a schematic diagram of a chain obtained based on a directed acyclic graph in the first aspect of the embodiment of the present application. Figure 7 As shown, from Figure 5 Four different chains, namely, 701, 702, 703 and 704, are extracted from the directed acyclic graph 500. Each chain is composed of a node 501 and an edge 502. In each chain, each frame image provides at most one node, that is, in the same frame image, no more than two nodes are located in the same chain.

[0087] It should be noted that in Figure 7 In the example, the starting points of chain 702 and chain 703 are both nodes corresponding to object 0 of I0, and the end points are both nodes corresponding to object 1 of I5. Since chain 702 and chain 703 both contain 5 nodes, both chain 702 and chain 703 are retained, or only one of chain 702 and chain 703 can be retained.

[0088] In operation 104 , a moving object having a moving speed within a certain range may be detected according to the chain extracted in operation 103 .

[0089] In at least one embodiment, in operation 104, the distance between the first position in the image corresponding to the starting node at both ends of the chain and the second position in the image corresponding to the ending node can be calculated. When the distance is in the second interval [threshole_inner_total, threshold_outer_total], it is determined that the movement speed of the same moving object corresponding to each node on the chain is within a certain range.

[0090] For example, for Figure 7The starting point of chain 704 is the node corresponding to object 0 of I0, and the end point is the node corresponding to object 0 of I5. Therefore, the position of object 0 in image I0 (for example, the pixel position of the center of the frame of object 0 in image I0) is taken as the first position, and the position of object 0 in image I5 (for example, the pixel position of the center of the frame of object 0 in image I5) is taken as the second position, and the pixel distance between the first position and the second position is calculated. If the pixel distance is within the second interval, it is determined that the movement speed of the same moving object corresponding to each node on chain 704 is within a certain range. If the pixel distance is outside the second interval, it is determined that the movement speed of the same moving object corresponding to each node on chain 704 is outside the certain range, for example, the movement speed is greater than the upper limit value of the certain range or less than the lower limit value of the certain range.

[0091] In this embodiment, if Figure 1 As shown, the method may also include:

[0092] Operation 105: When the number of image frames spanned by the chain in the directed acyclic graph exceeds a quantity threshold, retain the chain; and / or

[0093] Operation 106: When the angle between two adjacent edges in the chain exceeds a predetermined angle threshold, delete the chain.

[0094] like Figure 7 As shown, the number of image frames spanned by chains 702, 703, and 704 is 6, and the number of image frames spanned by chain 701 is 3. If the number threshold is 4, chains 702, 703, and 704 can be retained in operation 105, and chain 701 can be deleted. According to operation 105, chains spanning a large number of image frames can be retained, and chains spanning a small number of image frames can be deleted, thereby reducing the amount of calculation.

[0095] Figure 8 It is a schematic diagram that the angle between two adjacent edges in the chain exceeds the predetermined angle threshold. Figure 8 As shown, in the chain 800 , the angle a1 between the edge 801 and the edge 802 is greater than a predetermined angle threshold, and therefore, the chain 800 is deleted in operation 106 .

[0096] In operation 106 , the predetermined angle threshold is, for example, 90 degrees.

[0097] Since vehicles traveling on the road usually do not have large bending movements, the chains corresponding to the traveling vehicles usually do not have large bending angles. According to operation 106, chains with large bending angles can be deleted, thereby avoiding false detection.

[0098] In at least one embodiment, the moving object detection method may include at least one of operation 105 and operation 106, and, in the case of including at least one of operation 105 and operation 106, operation 104 may be located after operation 105 and / or operation 106, thereby reducing the number of chains to be detected in operation 104, thereby reducing the amount of calculation of operation 104. In addition, the moving object detection method may not include operation 105 and operation 106.

[0099] According to the first aspect of the embodiment of the present application, determining whether the moving speed of a moving object is within a certain range based on a directed acyclic graph can reduce missed detections of moving objects. In addition, the speed and efficiency of detecting moving objects can be improved.

[0100] Second aspect of the embodiment

[0101] A second aspect of the embodiments of the present application provides a moving object detection device, which corresponds to the moving object detection method of the first aspect of the embodiments.

[0102] Fig. 9 is a schematic diagram of a moving object detection device 900 according to the second aspect of an embodiment of the present application. Fig. 9 As shown, the device 900 includes: a first processing unit 901 , a second processing unit 902 , a third processing unit 903 and a fourth processing unit 904 .

[0103] The first processing unit 901 pairs the target in each frame image with the target corresponding to the same object in other predetermined frame images to form a target pair for the first number of frames of images; the second processing unit 902 generates a directed acyclic graph based on the target pair, wherein the directed acyclic graph includes nodes and edges connecting different nodes, the nodes represent each target in the target pair, the nodes are arranged in the order of each frame image, two nodes connected by an edge represent a target pair, and the direction of the edge is from the target in the previous frame image to the target in the next frame image; the third processing unit 903 extracts distinct chains from the directed acyclic graph, wherein each chain includes the nodes and the edges, and in each chain, at most one node is provided for each frame image; the fourth processing unit 904 detects moving objects with a moving speed within a certain range based on the extracted chains.

[0104] In addition, as shown in the flowchart, the moving object detection device 900 may further include: a fifth processing unit 905 and / or a sixth processing unit 906.

[0105] The fifth processing unit 905 retains the chain when the number of image frames spanned by the chain in the directed acyclic graph exceeds a quantity threshold; the sixth processing unit 906 deletes the chain when the angle between two adjacent edges in the chain exceeds a predetermined angle threshold.

[0106] For the description of each unit of the moving object detection device 900 , reference may be made to the description of each operation of the moving object detection method in the first aspect of the embodiment.

[0107] According to the second aspect of the embodiment of the present application, determining whether the moving speed of a moving object is within a certain range based on a directed acyclic graph can reduce missed detections of moving objects. In addition, the speed and efficiency of detecting moving objects can be improved.

[0108] The third aspect of the embodiment

[0109] A third aspect of the embodiments of the present application provides an electronic device, comprising: a moving object detection device as described in the second aspect of the embodiments.

[0110] Fig.10 Schematic diagram of the electronic device according to the third aspect of the embodiment of the present application. Fig.10 As shown, the electronic device 1000 may include: a central processing unit (CPU) 1001 and a memory 1002; the memory 1002 is coupled to the central processing unit 1001. The memory 1002 may store various data; in addition, it may store a program for controlling, and execute the program under the control of the central processing unit 1001.

[0111] In one embodiment, the functions of the moving object detection device 900 may be integrated into the central processor 1001 .

[0112] The central processing unit 1001 may be configured to execute the moving object detection method described in the first aspect of the embodiment.

[0113] In another embodiment, the moving object detection device 900 can be configured separately from the processor 1001. For example, the moving object detection device 900 can be configured as a chip connected to the processor 1001, and the functions of the moving object detection device 900 are realized through the control of the processor 1001.

[0114] In addition, if Fig.10 As shown, the electronic device 1000 may also include: an input / output unit 1003 and a display unit 1004, etc.; wherein the functions of the above components are similar to those of the prior art and are not described in detail here. It is worth noting that the electronic device 1000 does not necessarily have to include Fig.10 In addition, the electronic device 1000 may also include Fig.10 For components not shown, reference may be made to the prior art.

[0115] An embodiment of the present application also provides a computer-readable program, wherein when the program is executed in a moving object detection device or an electronic device, the program enables the moving object detection device or the electronic device to perform the moving object detection method described in the first aspect of the embodiment.

[0116] An embodiment of the present application also provides a storage medium storing a computer-readable program, wherein the storage medium stores the above-mentioned computer-readable program, and the computer-readable program enables a moving object detection device or electronic device to execute the moving object detection method described in the first aspect of the embodiment.

[0117] The measuring device described in conjunction with the embodiments of the present invention may be directly embodied as hardware, a software module executed by a processor, or a combination of the two. Fig. 9 One or more of the functional block diagrams shown in and / or one or more combinations of the functional block diagrams may correspond to various software modules of the computer program flow or to various hardware modules. These software modules may correspond to various operations shown in the first aspect of the embodiment, respectively. These hardware modules may be implemented by solidifying these software modules, for example, using a field programmable gate array (FPGA).

[0118] The software module may be located in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. A storage medium may be coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium; or the storage medium may be an integral part of the processor. The processor and the storage medium may be located in an ASIC. The software module may be stored in a memory of a mobile terminal or in a memory card that can be inserted into the mobile terminal. For example, if the electronic device uses a large-capacity MEGA-SIM card or a large-capacity flash memory device, the software module may be stored in the MEGA-SIM card or the large-capacity flash memory device.

[0119] against Fig. 9 One or more of the functional block diagrams and / or one or more combinations of the functional block diagrams described herein may be implemented as a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, or any appropriate combination thereof for performing the functions described in the present application. Fig. 9One or more of the described functional block diagrams and / or one or more combinations of the functional block diagrams may also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in communication with a DSP, or any other such configuration.

[0120] The present application is described above in conjunction with specific embodiments, but it should be clear to those skilled in the art that these descriptions are exemplary and are not intended to limit the scope of protection of the present application. Those skilled in the art can make various modifications and variations to the present application based on the principles of the present application, and these modifications and variations are also within the scope of the present application.

[0121] Regarding the implementation methods including the above embodiments, the following additional notes are also disclosed:

[0122] 1. A moving object detection device, comprising:

[0123] A first processing unit, for a first number of frames of images, pairs a target object in each frame of image with a target object corresponding to the same object in other predetermined frames of image to form a target object pair;

[0124] a second processing unit, which generates a directed acyclic graph according to the target object pair, wherein the directed acyclic graph includes nodes and edges connecting different nodes, the nodes represent each target object in a target object pair, the nodes are arranged in the order of each frame image, two nodes connected by one edge represent one target object pair, and the direction of the edge is from the target object in the previous frame image to the target object in the next frame image;

[0125] a third processing unit, which extracts distinct chains from the directed acyclic graph, wherein each chain includes the nodes and the edges, and in each chain, at most one node is provided per frame image; and

[0126] A fourth processing unit detects a moving object with a moving speed within a certain range based on the extracted chain.

[0127] 2. The detection device as described in Supplement 1, wherein:

[0128] The first processing unit calculates the distance between a target object (A) in a current frame image and each target object (B) corresponding to the target object (of the same vehicle type) in another frame image. If the distance is in a first interval, the target object in the other frame image is paired with the target object in the current frame image as a target pair.

[0129] 3. The detection device as described in Supplement 2, wherein:

[0130] If the distances between a target object (A) in the current frame image and two or more target objects (B) in another frame image are all within the first interval,

[0131] The first processing unit also combines a target object (A) in the current frame image with the two or more targets (B) in the other frame image to form candidate target pairs, calculates the distance between the appearance features of the two targets in each candidate target pair in the feature space, and takes the candidate target pair with the smallest distance in the feature space and a distance not greater than the feature distance threshold as the target pair.

[0132] 4. The detection device as described in Supplement 3, wherein:

[0133] The first processing unit calculates the distance between the features of two targets in each candidate target pair in the feature space based on a cosine metric learning algorithm.

[0134] 5. The detection device as described in Supplement 2, wherein:

[0135] Before calculating the distance, the first processing unit further corrects the bounding box of each target object in the first number of frames of images to a bounding box in an image with a predetermined resolution.

[0136] When calculating the distance, the first processing unit uses the corrected border for calculation.

[0137] 6. The detection device according to Note 1, wherein the third processing unit:

[0138] Selecting a starting node of the chain, and starting from the starting node, obtaining multiple chains; and

[0139] If the nodes and edges of the chain are all included in other chains, the chain will be deleted. If the starting nodes and ending nodes of more than two chains are the same, the chain with the largest number of nodes will be retained.

[0140] 7. The detection device as described in Supplement 6, wherein:

[0141] The starting node is: a node in the image frame that is the most forward in time among the nodes that do not belong to any existing chain.

[0142] 8. The detection device according to Note 1, wherein the fourth processing unit:

[0143] The distance between the first position in the image corresponding to the starting node at both ends of the chain and the second position in the image corresponding to the ending node is calculated. When the distance is in the second interval, it is determined that the movement speed of the same moving object corresponding to each node on the chain is within a certain range.

[0144] 9. The detection device as described in Note 1, wherein the device further comprises:

[0145] A fifth processing unit, which retains the chain when the number of image frames spanned by the chain in the directed acyclic graph exceeds a number threshold; and / or

[0146] The sixth processing unit is configured to delete the chain when the angle between two adjacent edges in the chain exceeds a predetermined angle threshold.

[0147] 10. An electronic device comprising a moving object detection device as described in any one of Notes 1 to 9.

[0148] 11. A method for detecting a moving object, comprising:

[0149] For the first number of frames of images, pairing the target in each frame of image with the target corresponding to the same object in other predetermined frames of image to form a target pair;

[0150] Generate a directed acyclic graph according to the target object pair, wherein the directed acyclic graph includes nodes and edges connecting different nodes, the nodes represent each target object in the target object pair, the nodes are arranged in the order of each frame image, two nodes connected by one edge represent one target object pair, and the direction of the edge is from the target object in the previous frame image to the target object in the next frame image;

[0151] Extracting distinct chains from the directed acyclic graph, wherein each chain includes the nodes and the edges, and in each chain, each frame image provides at most one node; and

[0152] Based on the extracted chain, a moving object with a moving speed within a certain range is detected.

[0153] 12. The detection method according to supplementary note 11, wherein the method of forming a target pair comprises:

[0154] Calculate the distance between a target object (A) in the current frame image and each target object (B) corresponding to the target object (of the same vehicle type) in another frame image. If the distance is in the first interval, pair the target object in the other frame image with the target object in the current frame image as a target pair.

[0155] 13. The detection method as described in Supplement 12, wherein the method of forming a target pair further comprises:

[0156] If the distances between a target object (A) in the current frame image and two or more target objects (B) in another frame image are both within the first interval, then a target object (A) in the current frame image and the two or more target objects (B) in the other frame image are respectively combined to form a candidate target pair, and the distance between the appearance features of the two targets in each candidate target pair in the feature space is calculated, and the candidate target pair with the smallest distance in the feature space and a distance not greater than the feature distance threshold is taken as the target pair.

[0157] 14. The detection method as described in Supplement 13, wherein:

[0158] In the step of calculating the distance in the feature space, the distance between the features of the two targets in each candidate target pair in the feature space is calculated based on a cosine metric learning algorithm.

[0159] 15. The detection method as described in Supplement 12, wherein the method of forming a target pair further comprises:

[0160] Before calculating the distance, the bounding box of each target object in the first number of frames of images is corrected to a bounding box in an image with a predetermined resolution,

[0161] Wherein, in the step of calculating the distance, the corrected border is used for calculation.

[0162] 16. The detection method as described in Supplement 11, wherein the step of extracting a distinct chain from the directed acyclic graph comprises:

[0163] Selecting a starting node of the chain, and starting from the starting node, obtaining multiple chains; and

[0164] If the nodes and edges of the chain are all included in other chains, the chain (sub-chain) will be deleted. If the starting node and ending node of more than two chains are the same, the chain with the largest number of nodes will be retained.

[0165] 17. The detection method as described in Supplement 16, wherein:

[0166] The starting node is: a node in the image frame that is the most forward in time among the nodes that do not belong to any existing chain.

[0167] 18. The detection method as described in Supplement 11, wherein the step of detecting a moving object with a moving speed within a certain range based on the extracted chain comprises:

[0168] The distance between the first position in the image corresponding to the starting node at both ends of the chain and the second position in the image corresponding to the ending node is calculated. When the distance is in the second interval, it is determined that the movement speed of the same moving object corresponding to each node on the chain is within a certain range.

[0169] 19. The detection method according to Note 11, wherein the method further comprises:

[0170] When the number of image frames spanned by the chain in the directed acyclic graph exceeds a threshold number, retaining the chain; and / or

[0171] When the included angle between two adjacent edges in the chain exceeds a predetermined angle threshold, the chain is deleted.

Claims

1. A moving object detection device, comprising: A first processing unit, for a first number of frames of images, pairs a target object in each frame of image with a target object corresponding to the same object in other predetermined frames of image to form a target object pair; A second processing unit generates a directed acyclic graph according to the target object pair, wherein the directed acyclic graph includes nodes and edges connecting different nodes, the nodes represent each target object in the target object pair, the nodes are arranged in the order of each frame image, two nodes connected by one edge represent one target object pair, and the direction of the edge is from the target object in the previous frame image to the target object in the next frame image; a third processing unit, which extracts chains from the directed acyclic graph, wherein each chain includes the nodes and the edges, and in each chain, each frame image provides at most one node; and A fourth processing unit detects a moving object with a moving speed within a certain range based on the extracted chain.

2. The detection device according to claim 1, wherein: The first processing unit calculates the distance between a target in the current frame image and each target corresponding to the target in another frame image, and if the distance is in a first interval, pairs the target in the other frame image with the target in the current frame image as a target pair.

3. The detection device according to claim 2, wherein: If the distances between a target object in the current frame image and two or more target objects in another frame image are all within the first interval, The first processing unit also combines a target object in the current frame image with the two or more targets in the other frame image to form candidate target pairs, calculates the distance between the appearance features of the two targets in each candidate target pair in the feature space, and takes the candidate target pair with the smallest distance in the feature space and a distance not greater than the feature distance threshold as the target pair.

4. The detection device according to claim 3, wherein: The first processing unit calculates the distance between the features of two targets in each candidate target object pair in the feature space based on the cosine metric learning algorithm.

5. The detection device according to claim 2, wherein: Before calculating the distance, the first processing unit further corrects the borders of each target object in the first number of frames of images to the borders in the images with a predetermined resolution. When calculating the distance, the first processing unit uses the corrected border for calculation.

6. The detection device according to claim 1, wherein: The third processing unit: Selecting a starting node of the chain, and starting from the starting node, obtaining multiple chains; and If the nodes and edges of the chain are all included in other chains, the chain will be deleted. If the starting nodes and ending nodes of more than two chains are the same, the chain with the largest number of nodes will be retained.

7. The detection device according to claim 6, wherein: The starting node is: a node in the image frame that is the most forward in time among the nodes that do not belong to any existing chain.

8. The detection device according to claim 1, wherein: The fourth processing unit: The distance between the first position in the image corresponding to the starting node at both ends of the chain and the second position in the image corresponding to the ending node is calculated. When the distance is in the second interval, it is determined that the movement speed of the same moving object corresponding to each node on the chain is within a certain range.

9. An electronic device comprising the moving object detection device according to any one of claims 1 to 8.

10. A method for detecting a moving object, comprising: For a first number of frames of images, pairing a target object in each frame of image with a target object corresponding to the same object in other predetermined frames of image to form a target object pair; Generate a directed acyclic graph according to the target object pair, wherein the directed acyclic graph includes nodes and edges connecting different nodes, the nodes represent each target object in the target object pair, the nodes are arranged in the order of each frame image, two nodes connected by one edge represent one target object pair, and the direction of the edge is from the target object in the previous frame image to the target object in the next frame image; Extracting chains from the directed acyclic graph, wherein each chain includes the nodes and the edges, and in each chain, each frame image provides at most one node; and Based on the extracted chain, a moving object with a moving speed within a certain range is detected.

Images