Speed estimation method and apparatus for target, device, and medium
Patent Information
- Application Number
- MYPI2025006491
- Authority / Receiving Office
- MY · MY
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-04-24
- Filing Date
- 2023-09-05
- Publication Date
- 2024-10-24
AI Technical Summary
It is difficult for existing monocular cameras to accurately estimate the moving speed of targets in intelligent driving systems, especially in the initial period when the speed information is lost and the speed estimation is not smooth enough, and when the target moving speed changes, the filter recognition is delayed.
By setting preset cycle time nodes in the target velocity measurement task and the target optical flow estimation task respectively, the pixel coordinates of the target object in the current frame image and the previous frame image are obtained, the homography transformation matrix is calculated, and combined with the actual longitudinal distance to determine The moving speed of the target is filtered using a preset filter to obtain a smooth speed estimate.
It achieves accurate estimation of target moving speed, maintains the smoothness of speed estimation, avoids the loss of initial frame speed information, and improves real-time responsiveness to changes in target speed.
Smart Images

Figure PI2025006491_DRAWING_001 
Figure PI2025006491_DRAWING_002 
Figure PI2025006491_DRAWING_003
Abstract
Description
Method, device, equipment and medium for estimating target speed
[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on April 24, 2023, with application number 202310443696.4 and invention name “A method, device, equipment and medium for estimating the speed of a target”, the entire contents of which are incorporated by reference into this application. Technical Field
[0002] The present invention relates to the field of computer vision technology, and in particular to a method, device, equipment and medium for estimating the speed of a target. Background Art
[0003] Currently, more and more smart cameras are installed in cars with intelligent driving functions. Smart cameras are divided into monocular, binocular, and trinocular cameras. Monocular cameras are widely used due to their low price and small size. The challenge of intelligent driving systems based on monocular cameras lies in estimating the physical distance and speed of the target.
[0004] Currently, monocular cameras in the industry generally use the pinhole imaging principle to estimate the distance to a target. This method has sufficient stability within a certain distance range. However, the target's velocity is mostly estimated by calculating the position difference between the target's two frames, using kinematic principles to calculate the velocity, or by filtering with a traditional distance filter. Both methods have significant problems: Calculating velocity using the position difference between the two frames loses velocity information during the target's initial cycle, and the velocity calculated directly from the position difference is not smooth and exhibits large fluctuations. The velocity obtained using a distance filter also suffers from the loss of velocity information in the target's initial frame and exhibits hysteresis. When the target's velocity changes, the filter requires more computation cycles to recognize this change.
[0005] In summary, how to accurately estimate the target's moving speed, maintain the smoothness of the speed estimation, and not lose the initial frame speed information is a problem that needs to be solved.
[0006] Summary of the Invention
[0007] In view of this, the present invention aims to provide a method, apparatus, device, and medium for estimating the speed of a target, which can accurately estimate the target's moving speed while maintaining smoothness of the speed estimation and without losing the initial frame speed information. The specific scheme is as follows:
[0008] In a first aspect, the present application discloses a method for estimating target speed, comprising:
[0009] When the target speed measurement task is triggered at a first preset periodic time node, the first pixel coordinates and the second pixel coordinates corresponding to the pixel point of the target object in the current frame image and the previous frame image are obtained respectively; wherein the target speed measurement task is controlled by the task scheduler to be triggered after the target optical flow estimation task is completed; and the target optical flow estimation task is used to calculate the first pixel coordinates and the second pixel coordinates after being triggered at a second preset periodic time node;
[0010] Determining a homography transformation matrix between the previous frame image and the current frame image based on the first pixel coordinates and the second pixel coordinates;
[0011] Determining an actual longitudinal distance between the target object and the image sensor using the current frame image, and determining a moving speed of the target object using the homography transformation matrix and the actual longitudinal distance;
[0012] The moving speed is filtered using a preset filter to obtain a filtered moving speed.
[0013] Optionally, determining the actual longitudinal distance between the target object and the image sensor by using the current frame image includes:
[0014] Acquire an enclosing frame corresponding to the target object obtained by performing target detection on the current frame image by an image sensor, and determine the center point position coordinates, height information, and width information of the enclosing frame;
[0015] Based on the center point position coordinates, the height information and the width information, and using pinhole imaging projection technology, the actual longitudinal distance between the target object and the image sensor is determined.
[0016] Optionally, before obtaining the first pixel coordinates and the second pixel coordinates corresponding to the pixel point of the target object in the current frame image and the previous frame image, the method further includes:
[0017] After the target optical flow estimation task is triggered, obtaining the bounding boxes corresponding to the target object obtained by the image sensor performing target detection on the current frame image and the previous frame image respectively;
[0018] Determine the first pixel coordinates and the second pixel coordinates corresponding to each pixel point in the bounding box in the current frame image and the previous frame image respectively.
[0019] Optionally, determining first pixel coordinates and second pixel coordinates corresponding to each pixel point in the bounding box in the current frame image and the previous frame image, respectively, includes:
[0020] Determine the optical flow coordinates of each pixel point in the bounding box in the current frame image using a preset optical flow calculation algorithm, and determine the first pixel coordinates of each pixel point in the current frame image;
[0021] The second pixel coordinates of each pixel point in the bounding box in the previous frame image are determined based on the optical flow coordinates and the first pixel coordinates.
[0022] Optionally, determining a homography transformation matrix between the previous frame image and the current frame image based on the first pixel coordinates and the second pixel coordinates includes:
[0023] determining a scale change rate, a horizontal movement amount, and a vertical movement amount of the bounding box between the current frame image and the previous frame image based on the first pixel coordinates and the second pixel coordinates;
[0024] A homography transformation matrix between the previous frame image and the current frame image is obtained based on the scale change rate, the horizontal movement amount, and the vertical movement amount.
[0025] Optionally, the target speed estimation method further includes:
[0026] Determining a first width value of the bounding box in the current frame image, and determining a second width value of the bounding box in the previous frame image;
[0027] A ratio of the first width value to the second width value is determined as the scale change rate.
[0028] Optionally, determining the moving speed of the target object by using the homography transformation matrix and the actual longitudinal distance includes:
[0029] Determining a time difference between the current frame image and the previous frame image, and determining horizontal pixel coordinates of an anchor point of a bounding box in the current frame image;
[0030] Determining the longitudinal moving speed of the target object using the scale change rate, the actual longitudinal distance, and the time difference;
[0031] The lateral movement speed of the target object is determined using the horizontal movement amount, the actual longitudinal distance, the time difference, the horizontal pixel coordinates and the longitudinal movement speed.
[0032] In a second aspect, the present application discloses a target speed estimation device, comprising:
[0033] a pixel coordinate acquisition module, configured to acquire first pixel coordinates and second pixel coordinates corresponding to a pixel point of a target object in a current frame image and a previous frame image, respectively, after a target speed measurement task is triggered at a first preset periodic time node; wherein the target speed measurement task is controlled by a task scheduler to be triggered after a target optical flow estimation task is completed; and after the target optical flow estimation task is triggered at a second preset periodic time node, is configured to calculate the first pixel coordinates and the second pixel coordinates;
[0034] a matrix determination module, configured to determine a homography transformation matrix between the previous frame image and the current frame image based on the first pixel coordinates and the second pixel coordinates;
[0035] a speed determination module, configured to determine an actual longitudinal distance between the target object and the image sensor using the current frame image, and determine a moving speed of the target object using the homography transformation matrix and the actual longitudinal distance;
[0036] The filtering module is used to filter the moving speed using a preset filter to obtain a filtered moving speed.
[0037] In a third aspect, the present application discloses an electronic device, comprising:
[0038] Memory, used to store computer programs;
[0039] A processor is used to execute the computer program to implement the steps of the speed estimation method disclosed above.
[0040] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the steps of the speed estimation method of the aforementioned disclosed objectives are implemented.
[0041] It can be seen that, in the present application, when the target speed measurement task is triggered at the first preset cycle time node, the first pixel coordinates and the second pixel coordinates corresponding to the pixel points of the target object in the current frame image and the previous frame image are obtained respectively; wherein, the target speed measurement task is controlled by the task scheduler to be triggered after the target optical flow estimation task is completed; after the target optical flow estimation task is triggered at the second preset cycle time node, it is used to calculate the first pixel coordinates and the second pixel coordinates; based on the first pixel coordinates and the second pixel coordinates, the homography transformation matrix between the previous frame image and the current frame image is determined; the actual longitudinal distance between the target object and the image sensor is determined using the current frame image, and the moving speed of the target object is determined using the homography transformation matrix and the actual longitudinal distance; the moving speed is filtered using a preset filter to obtain the filtered moving speed. As can be seen from this, in this application, when the target speed measurement task is triggered, the corresponding target optical flow estimation task has been completed. That is, the first pixel coordinates and second pixel coordinates corresponding to the pixel points of the target object in the current frame image and the previous frame image, respectively, have been obtained through the target optical flow estimation task. In the above process, the target speed measurement task and the target optical flow estimation task are controlled by the task scheduler to be triggered at their respective corresponding periodic time nodes. The target speed measurement task and the target optical flow estimation task are two independent periodic tasks. Since the tasks are triggered at the preset periodic time nodes, the corresponding speed can be estimated even in the initial period, that is, the initial frame speed information is not lost. Furthermore, based on the obtained first pixel coordinates and second pixel coordinates, the homography transformation matrix between the previous frame image and the current frame image is determined, and the actual longitudinal distance between the target object and the image sensor is determined using the current frame image. In this way, the target object's moving speed can be determined using the homography transformation matrix and the actual longitudinal distance, thereby achieving an accurate estimation of the target moving speed. Finally, the moving speed is filtered using a preset filter to obtain a smoothed moving speed, maintaining the smoothness of the speed estimation. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.
[0043] FIG1 is a flow chart of a method for estimating target speed disclosed in the present application;
[0044] FIG2 is a flow chart of a specific target speed estimation method disclosed in this application;
[0045] FIG3 is a timing diagram of a specific task scheduling disclosed in this application;
[0046] FIG4 is a schematic structural diagram of a target speed estimation device disclosed in this application;
[0047] FIG5 is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION
[0048] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0049] Currently, the estimation of target motion speed mostly uses the method of calculating the position difference of the target between the previous and next frames, calculating the speed using the principle of kinematics, or obtaining the speed by filtering with a traditional distance filter. Both of these methods have obvious problems: the method of calculating the speed by using the position difference between the two frames will lose the speed information in the initial period of the target, and the speed value directly calculated by using the position difference is not smooth enough and has large jumps; the speed obtained by filtering with a distance filter also has the problem of losing the target initial frame speed information, and has hysteresis. When the target movement speed changes, the filter requires more calculation cycles to identify this change. To this end, the embodiments of the present application disclose a method, device, equipment and medium for estimating the speed of a target, which can accurately estimate the moving speed of the target, maintain the smoothness of the speed estimation, and do not lose the initial frame speed information.
[0050] 1 , an embodiment of the present application discloses a method for estimating target speed, the method comprising:
[0051] Step S11: When the target speed measurement task is triggered at the first preset cycle time node, the first pixel coordinates and the second pixel coordinates corresponding to the pixel points of the target object in the current frame image and the previous frame image are obtained respectively; wherein, the target speed measurement task is controlled by the task scheduler to be triggered after the target optical flow estimation task is completed; after the target optical flow estimation task is triggered at the second preset cycle time node, it is used to calculate the first pixel coordinates and the second pixel coordinates.
[0052] In this embodiment, when the target speed measurement task is triggered at the first preset periodic time node and is used to estimate the speed of the target in the current frame image, the corresponding target optical flow estimation task has already been completed. That is, the first pixel coordinates and second pixel coordinates corresponding to the pixel point of the target object in the current frame image and the previous frame image, respectively, have been obtained through the target optical flow estimation task. Furthermore, by controlling the triggering of tasks through the task scheduler, good real-time responsiveness can be ensured.
[0053] In the above process, the target velocity measurement task and the target optical flow estimation task are controlled by the task scheduler to be triggered at the first preset cycle time node and the second preset cycle time node, respectively. The target velocity measurement task and the target optical flow estimation task are two independent periodic tasks, and their time periods are inconsistent. Because the tasks are triggered at the preset cycle time nodes, the corresponding speed can be estimated even during the initial cycle. In other words, the target motion speed estimation can be completed from the initial cycle without losing the initial frame speed information. Figure 2 shows a specific task scheduling timing diagram. As shown in Figure 2, Task A is the target velocity measurement task, and Task C is the target optical flow estimation task. As can be seen from the figure, when Task A is triggered, that is, each time a new frame is received to perform distance and speed estimation on the target, Task C has already processed the current frame image and the previous frame image and calculated the optical flow. In other words, the scheduling period of the target optical flow estimation task is twice or more than that of the target velocity measurement task. The task scheduler ensures that every time a frame image is acquired and the target velocity is estimated, the optical flow change between two consecutive frames can be obtained from the optical flow task. For example, when performing distance and velocity processing on frame 2 in the figure, the optical flow information from frame 1 to frame 2 is already available. When performing distance and velocity processing on frame 6, the optical flow information from frame 5 to frame 6 is already available. Therefore, a key dependency of this application is to obtain the optical flow change of the target. Assuming that the target in frame t is being processed, the rate of change of the optical flow of these targets can be obtained from the difference between frame t-1 and frame t. As shown in Figure 2, the task scheduler ensures that the optical flow change from frame t-1 to frame t is always present.
[0054] Step S12: determining a homography transformation matrix between the previous frame image and the current frame image based on the first pixel coordinates and the second pixel coordinates.
[0055] In this embodiment, the homography transformation matrix between the previous frame image and the current frame image is determined based on the first pixel coordinates and the second pixel coordinates corresponding to the pixel points of the target object in the current frame image and the previous frame image, respectively.
[0056] Step S13: using the current frame image to determine the actual longitudinal distance between the target object and the image sensor, and using the homography transformation matrix and the actual longitudinal distance to determine the moving speed of the target object.
[0057] In this embodiment, image acquisition is based on an image sensor. Therefore, the actual longitudinal distance between the target object and the image sensor in the current frame is determined. The target object's movement speed is then determined using the homography matrix and the actual longitudinal distance, achieving an accurate estimation of the target's movement speed. Specifically, the image sensor can be a monocular camera, and the actual longitudinal distance is the distance between the monocular camera and the target object in the real world.
[0058] Furthermore, the above-mentioned determination of the actual longitudinal distance between the target object and the image sensor in the current frame image includes: obtaining the bounding box corresponding to the target object obtained by the image sensor performing target detection on the current frame image, and determining the center point position coordinates, height information and width information of the bounding box; based on the center point position coordinates, the height information and the width information, and using the pinhole imaging projection technology to determine the actual longitudinal distance between the target object and the image sensor. It can be understood that the detection result of the target by the image sensor is given in the form of a bounding box (i.e., a bounding box), and the detection process can be specifically performed in the Task B image processing task in Figure 2. Based on the center point position coordinates and height and width information of the bounding box, the actual longitudinal distance dx and the actual lateral distance dy of the target in the physical space can be calculated using the pinhole imaging projection principle.
[0059] Step S14: Filtering the moving speed using a preset filter to obtain a filtered moving speed.
[0060] In this embodiment, the velocity is filtered using a preset filter to obtain a smoothed velocity, thereby maintaining the smoothness of the velocity estimate. Specifically, the preset filter may be a Kalman filter. In addition to filtering the velocity, the actual longitudinal and lateral distances obtained may also be filtered to obtain smoothed distance values.
[0061] It can be seen that, in the present application, when the target speed measurement task is triggered at the first preset cycle time node, the first pixel coordinates and the second pixel coordinates corresponding to the pixel points of the target object in the current frame image and the previous frame image are obtained respectively; wherein, the target speed measurement task is controlled by the task scheduler to be triggered after the target optical flow estimation task is completed; after the target optical flow estimation task is triggered at the second preset cycle time node, it is used to calculate the first pixel coordinates and the second pixel coordinates; based on the first pixel coordinates and the second pixel coordinates, the homography transformation matrix between the previous frame image and the current frame image is determined; the actual longitudinal distance between the target object and the image sensor is determined using the current frame image, and the moving speed of the target object is determined using the homography transformation matrix and the actual longitudinal distance; the moving speed is filtered using a preset filter to obtain the filtered moving speed. It can be seen that in this application, when the visual speed measurement task is triggered, the corresponding target optical flow estimation task has been completed, that is, the first pixel coordinates and second pixel coordinates corresponding to the pixel points of the target object in the current frame image and the previous frame image have been obtained through the target optical flow estimation task. In the above process, the target speed measurement task and the target optical flow estimation task are controlled by the task scheduler to be triggered at their respective corresponding cycle time nodes. The target speed measurement task and the target optical flow estimation task are two independent cycle tasks. Since the tasks are triggered at the preset cycle time nodes, the corresponding speed can be estimated even in the initial cycle, that is, the initial frame speed information will not be lost. Furthermore, based on the obtained first pixel coordinates and second pixel coordinates, the homography transformation matrix between the previous frame image and the current frame image is determined, and the actual longitudinal distance between the target object and the image sensor is determined using the current frame image, so that the moving speed of the target object can be determined using the homography transformation matrix and the actual longitudinal distance, thereby achieving accurate estimation of the target moving speed. Finally, the moving speed is filtered using a preset filter to obtain the smoothed moving speed, maintaining the smoothness of the speed estimation.
[0062] As shown in FIG3 , the embodiment of the present application discloses a specific method for estimating the speed of a target. Compared with the previous embodiment, this embodiment further illustrates and optimizes the technical solution. Specifically, it includes:
[0063] Step S21: When the target speed measurement task is triggered at the first preset cycle time node, the first pixel coordinates and the second pixel coordinates corresponding to the pixel points of the target object in the current frame image and the previous frame image are obtained respectively; wherein, the target speed measurement task is controlled by the task scheduler to be triggered after the target optical flow estimation task is completed; after the target optical flow estimation task is triggered at the second preset cycle time node, it is used to calculate the first pixel coordinates and the second pixel coordinates.
[0064] In this embodiment, before obtaining the first pixel coordinates and the second pixel coordinates corresponding to the pixel points of the target object in the current frame image and the previous frame image, the process further includes: after the target optical flow estimation task is triggered, obtaining a bounding box corresponding to the target object obtained by the image sensor performing target detection on the current frame image and the previous frame image; and determining the first pixel coordinates and the second pixel coordinates corresponding to each pixel point in the bounding box in the current frame image and the previous frame image. It should be noted that the target speed measurement task requires the optical flow of the target as necessary input information. Optical flow estimation, as a classic traditional algorithm for motion image analysis in the computer vision industry, is currently supported by the vast majority of intelligent driving chips. Developers do not need to have a deep understanding of the nature of optical flow algorithms and the technical implementation capabilities. They only need to call the relevant technical support algorithms provided by the chip manufacturer to obtain the optical flow calculation results of an image. In simple terms, when the target moves in the image, each pixel point has a corresponding movement. Therefore, after the target optical flow estimation task is triggered, the bounding box corresponding to the target object obtained by the image sensor performing target detection on the current frame image and the previous frame image in the image processing task is obtained, and optical flow estimation is performed respectively to determine the first pixel coordinates and second pixel coordinates corresponding to each pixel point in the bounding box in the current frame image and the previous frame image respectively.
[0065] Furthermore, when determining the first pixel coordinates and the second pixel coordinates corresponding to each pixel point in the bounding box in the current frame image and the previous frame image, the method specifically includes: using a preset optical flow calculation algorithm to determine the optical flow coordinates of each pixel point in the bounding box in the current frame image, and determining the first pixel coordinates of each pixel point in the current frame image; based on the optical flow coordinates and the first pixel coordinates, determining the second pixel coordinates of each pixel point in the bounding box in the previous frame image. That is, first, using a preset optical flow calculation algorithm to determine the optical flow coordinates (du, dv) of each pixel point in the bounding box in the current frame image, and determining the first pixel coordinates (U t , V t ). It should be pointed out that the original representation of optical flow is also similar to a picture of the same size. Each point in the picture is a vector (du, dv), which indicates the moving speed of the pixel in the horizontal U direction and the vertical V direction respectively. Within the given bounding box, each pixel has (du, dv) information. In addition, U t is the horizontal coordinate of a pixel point in the bounding box in the current frame image, V t is the vertical coordinate of the pixel point in the current frame image. Then, the second pixel coordinate (U) of each pixel point in the bounding box in the previous frame image is determined based on the optical flow coordinate and the first pixel coordinate. t-1 , Vt-1 ), the relationship is as follows: U t =U t-1 +du; V t =V t-1 +dv;
[0066] Among them, U t-1 is the horizontal coordinate of a pixel point in the bounding box in the previous frame image, V t-1 is the vertical coordinate of the pixel in the previous frame image.
[0067] Step S22: Based on the first pixel coordinates and the second pixel coordinates, determine the scale change rate, horizontal movement amount and vertical movement amount of the bounding box between the current frame image and the previous frame image, and obtain the homography transformation matrix between the previous frame image and the current frame image based on the scale change rate, the horizontal movement amount and the vertical movement amount.
[0068] In this embodiment, after determining the first pixel coordinates and the second pixel coordinates by using the optical flow information of the points within the bounding box, the scale change rate, horizontal movement amount and vertical movement amount between the current frame image and the previous frame image can be determined, and the homography transformation matrix between the previous frame image and the current frame image can be obtained based on the scale change rate, horizontal movement amount and vertical movement amount.
[0069] The calculation equation is as follows: U t =s*U t-1 +T x V t =s*V t-1 +T y ;
[0070] Among them, s, T x , T y That is, the elements of the three-degree-of-freedom homography transformation matrix H represent the scale change rate, horizontal movement, and vertical movement of the bounding box between the current frame image and the previous frame image. The matrix H is:
[0071] It should be pointed out that when solving the elements of the homography transformation matrix, the first pixel coordinates and the second pixel coordinates corresponding to multiple pixel points can be given, that is, the problem is transformed into a given batch of (du, dv) and (U t , V t ) data, and solve the above equation to obtain the homography transformation matrix s, T x , T y This is an overdetermined problem, and we need to use the least squares method or other algorithms to solve the equations to get s, T x , T y value.
[0072] Calculating the scale change rate may specifically include: determining a first width value of the bounding box in the current frame image, determining a second width value of the bounding box in the previous frame image; and determining the scale change rate as the ratio of the first width value to the second width value. It will be appreciated that for two bounding boxes in two given frames, the scale change rate is equal to the ratio of the widths of the bounding boxes. That is, the first width value and the second width value of the bounding box in the current frame image and the previous frame image, respectively, are determined, and the ratio of the two widths is the scale change rate.
[0073] Step S23: using the current frame image to determine the actual longitudinal distance between the target object and the image sensor, and determining the time difference between the current frame image and the previous frame image, and determining the horizontal pixel coordinates of the bounding box anchor point in the current frame image.
[0074] In this embodiment, the actual longitudinal distance dx between the target object and the image sensor in the current frame image, the time difference Δt between the current frame image and the previous frame image, and the horizontal pixel coordinates of the anchor point of the bounding box in the current frame image are determined.
[0075] Step S24: Determine the longitudinal movement speed of the target object using the scale change rate, the actual longitudinal distance and the time difference, and determine the lateral movement speed of the target object using the horizontal movement amount, the actual longitudinal distance, the time difference, the horizontal pixel coordinates and the longitudinal movement speed.
[0076] In this embodiment, after obtaining the homography transformation matrix s, T x , T y After the element is generated, the target's moving speed is determined based on the actual longitudinal distance between the target object and the image sensor in the current frame image, the time difference between the current frame image and the previous frame image, and the horizontal pixel coordinates of the bounding box anchor point in the current frame image. The moving speed includes the longitudinal moving speed and the lateral moving speed.
[0077] Specifically, the longitudinal moving speed v of the target object is determined using the scale change rate s, the actual longitudinal distance dx and the time difference Δt. x , the calculation formula is as follows:
[0078] v x =(1-s)*dx / Δt;
[0079] Using the horizontal movement T x , actual longitudinal distance dx, time difference Δt, horizontal pixel coordinates and longitudinal speed v xDetermine the lateral moving speed v of the target object y , the calculation formula is as follows:
[0080] Step S25: Filtering the lateral movement speed and the longitudinal movement speed using a preset filter to obtain filtered lateral movement speed and longitudinal movement speed.
[0081] In this embodiment, the horizontal and vertical distances and the horizontal and vertical moving speeds of the target are calculated through the above process, and a vector (dx, dy, v x , v y ), and then input this vector into traditional preset filter and other algorithms to obtain smoothed distance and speed.
[0082] For a more specific processing procedure of the above step S21, reference may be made to the corresponding contents disclosed in the above embodiments, which will not be described again here.
[0083] As can be seen, the target speed measurement task of this application requires the target's optical flow as necessary input information. Therefore, after the target optical flow estimation task is triggered, the bounding box corresponding to the target object, obtained by the image sensor performing target detection on the current frame image and the previous frame image in the image processing task, is obtained. Optical flow estimation is then performed separately to determine the first pixel coordinates and second pixel coordinates corresponding to each pixel in the bounding box in the current frame image and the previous frame image, respectively. Based on the first pixel coordinates and the second pixel coordinates, the scale change rate, horizontal movement amount, and vertical movement amount between the current frame image and the previous frame image are determined, and the homography transformation matrix is further obtained. Finally, the homography transformation matrix, actual longitudinal distance, and other parameters are used to determine the horizontal and vertical movement speeds of the target object, and smoothing filtering is performed. In this way, the present application directly measures the target's movement speed by using the relevant principles of computer vision based on the image sensor. Compared with traditional speed measurement methods, it has the characteristics of smoother calculation results, better real-time responsiveness, and no loss of initial frame speed information.
[0084] As shown in FIG4 , an embodiment of the present application discloses a device for estimating target speed, the device comprising:
[0085] A pixel coordinate acquisition module 11 is configured to acquire first pixel coordinates and second pixel coordinates corresponding to a pixel point of a target object in a current frame image and a previous frame image, respectively, after a target speed measurement task is triggered at a first preset periodic time node; wherein the target speed measurement task is controlled by a task scheduler to be triggered after a target optical flow estimation task is completed; and after the target optical flow estimation task is triggered at a second preset periodic time node, calculate the first pixel coordinates and the second pixel coordinates;
[0086] a matrix determination module 12, configured to determine a homography transformation matrix between the previous frame image and the current frame image based on the first pixel coordinates and the second pixel coordinates;
[0087] a speed determination module 13, configured to determine an actual longitudinal distance between the target object and the image sensor using the current frame image, and determine a moving speed of the target object using the homography transformation matrix and the actual longitudinal distance;
[0088] The filtering module 14 is configured to filter the moving speed using a preset filter to obtain a filtered moving speed.
[0089] It can be seen that, in the present application, when the target speed measurement task is triggered at the first preset cycle time node, the first pixel coordinates and the second pixel coordinates corresponding to the pixel points of the target object in the current frame image and the previous frame image are obtained respectively; wherein, the target speed measurement task is controlled by the task scheduler to be triggered after the target optical flow estimation task is completed; after the target optical flow estimation task is triggered at the second preset cycle time node, it is used to calculate the first pixel coordinates and the second pixel coordinates; based on the first pixel coordinates and the second pixel coordinates, the homography transformation matrix between the previous frame image and the current frame image is determined; the actual longitudinal distance between the target object and the image sensor is determined using the current frame image, and the moving speed of the target object is determined using the homography transformation matrix and the actual longitudinal distance; the moving speed is filtered using a preset filter to obtain the filtered moving speed. It can be seen that in this application, when the visual speed measurement task is triggered, the corresponding target optical flow estimation task has been completed, that is, the first pixel coordinates and second pixel coordinates corresponding to the pixel points of the target object in the current frame image and the previous frame image have been obtained through the target optical flow estimation task. In the above process, the target speed measurement task and the target optical flow estimation task are controlled by the task scheduler to be triggered at their respective corresponding cycle time nodes. The target speed measurement task and the target optical flow estimation task are two independent cycle tasks. Since the tasks are triggered at the preset cycle time nodes, the corresponding speed can be estimated even in the initial cycle, that is, the initial frame speed information will not be lost. Furthermore, based on the obtained first pixel coordinates and second pixel coordinates, the homography transformation matrix between the previous frame image and the current frame image is determined, and the actual longitudinal distance between the target object and the image sensor is determined using the current frame image, so that the moving speed of the target object can be determined using the homography transformation matrix and the actual longitudinal distance, thereby achieving accurate estimation of the target moving speed. Finally, the moving speed is filtered using a preset filter to obtain the smoothed moving speed, maintaining the smoothness of the speed estimation.
[0090] Figure 5 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Specifically, the device may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps of the target speed estimation method performed by the electronic device disclosed in any of the aforementioned embodiments.
[0091] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device. The communication protocol it follows is any communication protocol that can be applied to the technical solution of this application and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world. Its specific interface type can be selected according to specific application needs and is not specifically limited here.
[0092] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.
[0093] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or CD, etc. The resources stored thereon include an operating system 221, a computer program 222 and data 223, etc. The storage method can be temporary storage or permanent storage.
[0094] The operating system 221 is used to manage and control the hardware devices and computer programs 222 on the electronic device 20, enabling the processor 21 to calculate and process the massive amount of data 223 in the memory 22. The operating system 221 can be Windows, Unix, Linux, etc. In addition to including computer programs capable of performing the target speed estimation method performed by the electronic device 20 as disclosed in any of the aforementioned embodiments, the computer programs 222 may further include computer programs capable of performing other specific tasks. The data 223 may include data received by the electronic device from external devices, as well as data collected by its own input / output interface 25.
[0095] Furthermore, an embodiment of the present application also discloses a computer-readable storage medium, in which a computer program is stored. When the computer program is loaded and executed by a processor, the method steps performed in the target speed estimation process disclosed in any of the aforementioned embodiments are implemented.
[0096] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. Reference can be made to the descriptions of the identical or similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and the relevant parts can be referred to the descriptions of the methods.
[0097] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0098] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
[0099] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.
[0100] The above describes in detail the target speed estimation method, device, equipment, and storage medium provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only intended to help understand the method and core concept of the present invention. At the same time, for those skilled in the art, according to the concept of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present invention.
Claims
1. A method for estimating the speed of a target, characterized in that: include: When the target speed measurement task is triggered at the first preset periodic time node, the first pixel coordinates and the second pixel coordinates corresponding to the pixel points of the target object in the current frame image and the previous frame image are obtained respectively; wherein the target speed measurement task is controlled by the task scheduler to be triggered after the target optical flow estimation task is completed; and the target optical flow estimation task is used to calculate the first pixel coordinates and the second pixel coordinates after being triggered at the second preset periodic time node; Determine a homography transformation matrix between the previous frame image and the current frame image based on the first pixel coordinates and the second pixel coordinates; Determine the actual longitudinal distance between the target object and the image sensor using the current frame image, and determine the moving speed of the target object using the homography transformation matrix and the actual longitudinal distance; The moving speed is filtered using a preset filter to obtain a filtered moving speed.
2. The target speed estimation method according to claim 1, characterized in that: The method of determining the actual longitudinal distance between the target object and the image sensor by using the current frame image includes: Acquire a bounding box corresponding to the target object obtained by performing target detection on the current frame image by an image sensor, and determine the center point position coordinates, height information, and width information of the bounding box; Based on the center point position coordinates, the height information and the width information, the actual longitudinal distance between the target object and the image sensor is determined by using pinhole imaging projection technology.
3. The target speed estimation method according to claim 1, characterized in that: Before obtaining the first pixel coordinates and the second pixel coordinates corresponding to the pixel point of the target object in the current frame image and the previous frame image respectively, the method further includes: After the target optical flow estimation task is triggered, obtaining a bounding box corresponding to the target object obtained by performing target detection on the current frame image and the previous frame image by the image sensor; Determine the first pixel coordinates and the second pixel coordinates corresponding to each pixel point in the bounding box in the current frame image and the previous frame image respectively.
4. The method for estimating the speed of a target according to claim 3, characterized in that: The determining of the first pixel coordinates and the second pixel coordinates corresponding to each pixel point in the bounding box in the current frame image and the previous frame image respectively comprises: Determine the optical flow coordinates of each pixel point in the bounding box in the current frame image by using a preset optical flow calculation algorithm, and determine the first pixel coordinates of each pixel point in the current frame image; The second pixel coordinates of each pixel point in the bounding box in the previous frame image are determined based on the optical flow coordinates and the first pixel coordinates.
5. The method for estimating the speed of a target according to claim 3, characterized in that: The determining, based on the first pixel coordinates and the second pixel coordinates, a homography transformation matrix between the previous frame image and the current frame image includes: Determine a scale change rate, a horizontal movement amount, and a vertical movement amount of the bounding box between the current frame image and the previous frame image based on the first pixel coordinates and the second pixel coordinates; A homography transformation matrix between the previous frame image and the current frame image is obtained based on the scale change rate, the horizontal movement amount, and the vertical movement amount.
6. The method for estimating the speed of a target according to claim 5, characterized in that: Also includes: Determine a first width value of the enclosing frame in the current frame image, and determine a second width value of the enclosing frame in the previous frame image; A ratio of the first width value to the second width value is determined as the scale change rate.
7. The method for estimating the speed of a target according to claim 5, characterized in that: The step of determining the moving speed of the target object by using the homography transformation matrix and the actual longitudinal distance includes: Determine the time difference between the current frame image and the previous frame image, and determine the horizontal pixel coordinates of the bounding box anchor point in the current frame image; Determine the longitudinal moving speed of the target object by using the scale change rate, the actual longitudinal distance and the time difference; The lateral movement speed of the target object is determined by using the horizontal movement amount, the actual longitudinal distance, the time difference, the horizontal pixel coordinates and the longitudinal movement speed.
8. A target speed estimation device, characterized in that: include: A pixel coordinate acquisition module, which is used to obtain the first pixel coordinates and the second pixel coordinates of the pixel points of the target object in the current frame image and the previous frame image respectively when the target speed measurement task is triggered at the first preset period time node; wherein the target speed measurement task is controlled by the task scheduler to be triggered after the target optical flow estimation task is completed; and the target optical flow estimation task is used to calculate the first pixel coordinates and the second pixel coordinates after being triggered at the second preset period time node; A matrix determination module, used to determine a homography transformation matrix between the previous frame image and the current frame image based on the first pixel coordinates and the second pixel coordinates; a speed determination module, configured to determine an actual longitudinal distance between the target object and the image sensor using the current frame image, and determine a moving speed of the target object using the homography transformation matrix and the actual longitudinal distance; The filtering module is used to filter the moving speed using a preset filter to obtain a filtered moving speed.
9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor is used to execute the computer program to implement the steps of the speed estimation method of the target according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: Used to store a computer program; wherein, when the computer program is executed by a processor, the steps of the speed estimation method of the target according to any one of claims 1 to 7 are implemented.
