Speed measurement method and device, electronic equipment and computer readable storage medium

Abstract

The invention discloses a speed measurement method and device, electronic equipment and a computer readable storage medium. The method comprises the following steps: acquiring a first image frame and a second image frame which are acquired by a vehicle at a first moment and a second moment and contain a target object; the first moment is earlier than the second moment; determining a first scale factor of the target object at the second moment based on the first image frame and the second image frame; based on a second scale factor and a third scale factor of the target object at a third moment and the first scale factor, determining a fourth scale factor of the target object at the second moment; the third moment is earlier than the second moment; and determining the first speed of the target object at the second moment based on the fourth scale factor, the inter-frame interval, the longitudinal distance between the vehicle and the target object at the second moment and the vehicle speed. According to the invention, speed measurement of the target object can be realized.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent driving technology, and in particular to a speed measurement method, device, electronic device, and computer-readable storage medium. Background Art

[0002] In vision-based intelligent driving solutions, vehicles use cameras mounted on them to collect real-time image data to detect, identify, and track surrounding dynamic and static obstacles. Because the images captured by the cameras are two-dimensional, the vehicle cannot directly determine the speed of dynamic obstacles (such as motor vehicles, non-motor vehicles, and pedestrians). Therefore, a method for measuring the speed of dynamic obstacles in vision-based intelligent driving scenarios is urgently needed. Summary of the Invention

[0003] In order to solve the above technical problems, the present disclosure provides a speed measurement method, device, electronic device, and computer-readable storage medium to achieve speed measurement of a target object in a vision-based intelligent driving scenario.

[0004] A first embodiment of the present disclosure provides a speed measurement method, comprising:

[0005] Acquire a first image frame and a second image frame containing a target object captured by the vehicle at a first moment and a second moment, wherein the first moment is earlier than the second moment;

[0006] determining a first scale factor of the target object at the second moment based on the first image frame and the second image frame;

[0007] determining a fourth scale factor of the object at the second moment based on the second scale factor and the third scale factor of the object at a third moment and the first scale factor, wherein the third moment is earlier than the second moment;

[0008] A first speed of the target object at the second moment is determined based on the fourth scale factor, an inter-frame interval, a longitudinal distance between the ego vehicle and the target object at the second moment, and a speed of the ego vehicle.

[0009] A second embodiment of the present disclosure provides a speed measuring device, comprising:

[0010] An image acquisition module, configured to acquire a first image frame and a second image frame containing a target object, captured by the vehicle at a first moment and a second moment; the first moment is earlier than the second moment;

[0011] a first scale factor determining module, configured to determine a first scale factor of the target object at the second moment based on the first image frame and the second image frame;

[0012] a second scale factor determining module, configured to determine a fourth scale factor of the target object at a second moment based on the second scale factor and the third scale factor of the target object at a third moment, and the first scale factor; the third moment being earlier than the second moment;

[0013] The first speed determination module is configured to determine a first speed of the target object at the second moment based on the fourth scale factor, an inter-frame interval, a longitudinal distance between the ego vehicle and the target object at the second moment, and a speed of the ego vehicle.

[0014] A third aspect of the present disclosure provides a computer-readable storage medium storing a computer program for executing the speed measurement method provided in the first aspect.

[0015] An embodiment of the fourth aspect of the present disclosure provides an electronic device, comprising: a processor; a memory for storing executable instructions of the processor; the processor for reading the executable instructions from the memory and executing the instructions to implement the speed measurement method provided in the first aspect above.

[0016] The fifth embodiment of the present disclosure provides a computer program product. When an instruction processor in the computer program product executes, the speed measurement method provided by the first aspect of the present disclosure is executed.

[0017] In an embodiment of the present disclosure, an electronic device obtains a first image frame and a second image frame containing a target object captured by a self-vehicle at a first moment and a second moment. The first moment is earlier than the second moment. Then, the electronic device determines the first scale factor of the target object at the second moment based on the first image frame and the second image frame, and determines the fourth scale factor of the target object at the second moment based on the second scale factor and the third scale factor of the target object at the third moment, as well as the first scale factor. The third moment is earlier than the second moment. Thereafter, the electronic device determines the first speed of the target object at the second moment based on the fourth scale factor, the inter-frame interval, the longitudinal distance between the self-vehicle and the target object at the second moment, and the self-vehicle speed. In this way, the electronic device can measure the speed of the target object in a vision-based intelligent driving scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flowchart of a speed measurement method provided by an exemplary embodiment of the present disclosure.

[0019] Figure 2 It is a flowchart of a speed measurement method provided by another exemplary embodiment of the present disclosure.

[0020] Figure 3 It is a flowchart of a speed measurement method provided by another exemplary embodiment of the present disclosure.

[0021] Figure 4 It is a flowchart of a speed measurement method provided by another exemplary embodiment of the present disclosure.

[0022] Figure 5 2 is a schematic structural diagram of a speed measuring device provided by an exemplary embodiment of the present disclosure.

[0023] Figure 6 2 is a schematic structural diagram of a speed measuring device provided by an exemplary embodiment of the present disclosure.

[0024] Figure 7 2 is a schematic structural diagram of a speed measuring device provided by an exemplary embodiment of the present disclosure.

[0025] Figure 8 2 is a schematic structural diagram of a speed measuring device provided by an exemplary embodiment of the present disclosure.

[0026] Figure 9 is a structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] To explain the present disclosure, example embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. It should be understood that the present disclosure is not limited to the example embodiments.

[0028] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

[0029] Application Overview

[0030] The intelligent driving described in the present disclosure may cover multiple fields such as autonomous driving, assisted driving, and robotic systems. Autonomous driving technology is committed to achieving fully autonomous driving of intelligent vehicles in various complex road conditions without human intervention. It is also called unmanned driving and is an advanced form of intelligent driving. Assisted driving provides drivers with real-time road condition information, warnings, and partial driving operation support, such as automatic parking and adaptive cruise control, through a series of sensors and algorithms, aiming to improve driving safety and convenience. Robotic systems further expand intelligent driving technology to areas such as service robots and industrial robots, enabling robots to autonomously navigate, avoid obstacles, and complete specific tasks in complex environments, such as logistics distribution, warehouse management, etc., demonstrating the wide application potential of intelligent driving technology in different scenarios.

[0031] In vision-based intelligent driving solutions, vehicles use cameras mounted on the vehicle to collect real-time image data to detect, identify, and track surrounding dynamic and static obstacles. However, because the images collected by the cameras are two-dimensional, the vehicle cannot directly obtain the speed of dynamic obstacles (such as motor vehicles, non-motor vehicles, and pedestrians).

[0032] In an embodiment of the present disclosure, an electronic device obtains a first image frame and a second image frame containing a target object captured by a self-vehicle at a first moment and a second moment. The first moment is earlier than the second moment. Then, the electronic device determines the first scale factor of the target object at the second moment based on the first image frame and the second image frame, and determines the fourth scale factor of the target object at the second moment based on the second scale factor and the third scale factor of the target object at the third moment, as well as the first scale factor. The third moment is earlier than the second moment. Thereafter, the electronic device determines the first speed of the target object at the second moment based on the fourth scale factor, the inter-frame interval, the longitudinal distance between the self-vehicle and the target object at the second moment, and the self-vehicle speed. In this way, the electronic device can measure the speed of the target object in a vision-based intelligent driving scenario.

[0033] Exemplary Methods

[0034] Figure 1 FIG. 1 is a flow chart of a speed measurement method provided by an exemplary embodiment of the present disclosure. This embodiment can be applied to electronic devices, such as Figure 1 As shown, the following steps are included:

[0035] Step 101: Acquire a first image frame and a second image frame containing a target object captured by the vehicle at a first moment and a second moment, wherein the first moment is earlier than the second moment.

[0036] For example, while the vehicle is driving, a camera mounted on the vehicle can capture real-time images of the environment. For each moment in the environment, the electronic device can detect objects in the environment and obtain a detection frame for each object. For each object, the electronic device can capture an ROI (Region of Interest) image corresponding to the object from the environment image based on the detection frame. The objects are traffic participants (e.g., motor vehicles, non-motor vehicles, pedestrians, cyclists, etc.) in the driving scene in which the vehicle is located.

[0037] When the vehicle needs to measure the speed of a certain target object, the electronic device can obtain the first image frame and the second image frame containing the target object collected by the vehicle at the first moment and the second moment from the ROI image of the target object at each moment. The first moment is earlier than the second moment, the first moment can be a historical moment, the second moment can be the current moment, or both the first moment and the second moment can be historical moments, which is not limited in the embodiment of the present disclosure. The embodiment of the present disclosure is introduced by taking the first moment as the historical moment and the second moment as the current moment as an example. Other situations are similar and will not be described in detail in the embodiment of the present disclosure. The first moment and the second moment can differ by an inter-frame interval (i.e., the second moment is T n moment, the first moment is T n-1 The first image frame and the second image frame are adjacent image frames), and the first moment and the second moment may also differ by multiple inter-frame intervals (ie, the second moment is T n moment, the first moment is T n-k At the time point, k>1, the first image frame and the second image frame are non-adjacent image frames), which is not limited in the embodiment of the present disclosure.

[0038] Step 102 : Determine a first scale factor of the target object at a second moment based on the first image frame and the second image frame.

[0039] For example, since the target object moves relative to the vehicle, the size of the target object in the first image frame and the size of the target object in the second image frame will also change accordingly. After the electronic device acquires the first image frame and the second image frame of the target object, it can further perform optical flow tracking on the target object based on the first image frame and the second image frame to determine the first scale factor of the target object at the second moment. The scale factor is used to characterize the degree of change in the relative size of the target object between the first image frame and the second image frame. Accordingly, the first scale factor of the target object at the second moment is the scale factor of the target object between the first image frame and the second image frame, that is, the first scale factor of the target object at the second moment is the scale factor of the target object between the image frames at moments adjacent to the second moment.

[0040] Step 103: Determine a fourth scale factor of the target object at the second moment based on the second scale factor and the third scale factor of the target object at the third moment, and the first scale factor, wherein the third moment is earlier than the second moment.

[0041] For example, even if the target object rapidly accelerates or decelerates, its speed will not change dramatically in a very short period of time. Similarly, the scale factor of the target object will not change dramatically in a very short period of time. Therefore, to ensure that the scale factor of the target object is as stable as possible, after the electronic device determines the first scale factor (original scale factor) of the target object at the second moment, it can obtain the second scale factor (original scale factor) of the target object at a third moment before the second moment. The electronic device can then perform a linear fit on the second scale factor of the target object at the third moment and the first scale factor at the second moment to obtain a scale factor fitting curve for the target object. The scale factor fitting curve for the target object can characterize the trend or pattern of changes in the scale factor of the target object over time. The electronic device can then determine an estimated value of the first scale factor of the target object at the second moment based on the scale factor fitting curve. Finally, the electronic device can determine a fourth scale factor of the target object at the second moment based on the third scale factor of the target object at the third moment and the estimated value of the first scale factor at the second moment. The third scale factor of the target object at the third moment is an estimated value of the second scale factor of the target object at the third moment determined by the electronic device based on a scale factor fitting curve obtained by linearly fitting the second scale factor of the target object at the third moment and the scale factor before the third moment. The third moment may be multiple moments, and the third moment may include the first moment.

[0042] Step 104 : Determine a first velocity of the target object at the second moment based on the fourth scale factor, the inter-frame interval, the longitudinal distance between the ego vehicle and the target object at the second moment, and the ego vehicle velocity.

[0043] For example, after obtaining the fourth scale factor of the target at the second moment, the electronic device can determine the change in the target's movement distance between image frames based on the fourth scale factor and the longitudinal distance between the ego vehicle and the target at the second moment. The electronic device can then determine the target's relative speed relative to the ego vehicle at the second moment based on the change in the target's movement distance between image frames and the inter-frame interval. The electronic device can then determine the target's first speed at the second moment based on the target's relative speed relative to the ego vehicle at the second moment and the ego vehicle's speed at the second moment. The longitudinal distance between the ego vehicle and the target at the second moment can be calculated based on the target's detection frame height, camera intrinsic parameters, and extrinsic parameters in the environmental image captured by the ego vehicle at the second moment; the ego vehicle's speed at the second moment can be acquired by the ego vehicle's speed sensor.

[0044] In an embodiment of the present disclosure, an electronic device obtains a first image frame and a second image frame containing a target object captured by a self-vehicle at a first moment and a second moment. The first moment is earlier than the second moment. Then, the electronic device determines the first scale factor of the target object at the second moment based on the first image frame and the second image frame, and determines the fourth scale factor of the target object at the second moment based on the second scale factor and the third scale factor of the target object at the third moment, as well as the first scale factor. The third moment is earlier than the second moment. Thereafter, the electronic device determines the first speed of the target object at the second moment based on the fourth scale factor, the inter-frame interval, the longitudinal distance between the self-vehicle and the target object at the second moment, and the self-vehicle speed. In this way, the electronic device can measure the speed of the target object in a vision-based intelligent driving scenario.

[0045] like Figure 2 As shown in the above Figure 1 Based on the embodiment shown, step 102 may include the following steps:

[0046] Step 1021 : extract feature points from the first image frame to obtain first feature points of the first image frame.

[0047] Exemplarily, after an electronic device acquires a first image frame containing a target object, it may continuously downsample the first image frame to obtain an image pyramid corresponding to the first image frame. The image pyramid includes multiple image levels, each image level corresponding to an image of a certain size, and the lower the image level, the larger the image size of the corresponding image. The electronic device then determines a target image level and a feature point extraction step size based on the image size of the target object, determines a target image from the image pyramid based on the target image level, and extracts the first feature point of the first image frame from the target image based on the feature point extraction step size.

[0048] Step 1022: Based on the first image frame and the second image frame, perform optical flow tracking on the first feature point to obtain a second feature point corresponding to the first feature point in the second image frame.

[0049] Exemplarily, after the electronic device obtains the first feature point of an image frame, for each first feature point, it can determine the descriptor of the feature point based on the brightness of the feature point and its surrounding pixels. The descriptor of the feature point can characterize the local image feature of the feature point. Then, based on the coordinates of the feature point, the electronic device determines the local image area centered on the feature point in the second image frame, and for each pixel in the local image area, determines the descriptor of the pixel based on the brightness of the pixel and its surrounding pixels. Afterwards, the electronic device matches the descriptor of the feature point with the descriptors of each pixel in the local image area, thereby determining the second feature point corresponding to the first feature point in the second image frame. The i-th first feature point in the first image frame can be represented as A i , the i-th first feature point A in the first image frame i The corresponding second feature point in the second image frame can be expressed as B i .

[0050] It should be noted that the processing process of determining the second feature point corresponding to the first feature point in the second image frame based on the optical flow tracking method in the above embodiment is only an exemplary description. Those skilled in the art can also use other methods to perform optical flow tracking to determine the second feature point corresponding to the first feature point in the second image frame, and the present disclosed embodiment is not limited to this.

[0051] Step 1023 : Determine a first scale factor of the target object at a second moment based on the first feature point and the second feature point.

[0052] For example, after acquiring the first feature points of the first image frame and the second feature points of the second image frame, the electronic device may arbitrarily select two first feature points from each of the first feature points to form a first feature point pair, and then select the two second feature points corresponding to the two first feature points in the second image frame to form a second feature point pair. The electronic device may then determine a first scale factor of the target object between the first image frame and the second image frame based on the first feature point distance of the first feature point pair and the second feature point distance of the second feature point pair.

[0053] In the disclosed embodiment, optical flow tracking can quickly and accurately determine the first feature points of the first image frame and the second feature points of the second image frame, providing a basis for subsequently calculating and determining the first scale factor of the target object at the second moment based on the first feature points and the second feature points. Simultaneously, the electronic device arbitrarily selects two first feature points from each first feature point to form a first feature point pair, and then forms a second feature point pair with the corresponding second feature points of the two first feature points in the second image frame. Subsequently, the electronic device determines the first scale factor of the target object at the second moment based on the first feature point distance of the first feature point pair and the second feature point distance of the second feature point pair, thereby improving the accuracy of the scale factor calculation.

[0054] In the above Figure 2 Based on the illustrated embodiment, step 1021 may include the following steps:

[0055] Step 1: Perform multi-scale processing on the first image frame to generate an image pyramid of the first image frame, wherein the image pyramid includes multiple image levels.

[0056] For example, after acquiring a first image frame, the electronic device may continuously downsample the first image frame to obtain a series of images of gradually decreasing image sizes. The electronic device may then generate an image pyramid based on the first image frame and the series of images of gradually decreasing image sizes, in descending order of image size. The image pyramid includes multiple image levels, with each image level corresponding to an image of a certain image size. The lower the image level, the larger the image size of the corresponding image. For example, assume the image size of the first image frame is (H, W), where H represents height and W represents width. First, the electronic device may use the first image frame as an image at the first image level in the image pyramid, where the image size of the image at the first image level is (H, W). The first image level is the bottom layer of the image pyramid, i.e., the lowest image level. The electronic device may then downsample the image at the first image level by a factor of 2 and use it as an image at the second image level in the image pyramid, where the image size of the image at the second image level is (H / 2, W / 2). The second image level is the image level immediately above the first image level. The electronic device can then downsample the image at the second image level by a factor of 2 and use it as an image at the third image level in the image pyramid. The image size of the image at the third image level is (H / 4, W / 4). The third image level is the previous image level of the second image level. Similarly, the electronic device generates an image pyramid for the first image frame.

[0057] Step 2: Based on the image size of the target object, determine the target image level and feature point extraction step size.

[0058] For example, the closer the target is to the vehicle, the larger its image size in the environment image captured by the vehicle, and the farther the target is from the vehicle, the smaller its image size in the environment image captured by the vehicle. The target's image size can be the size of the target's detection frame. Given the same resolution, the larger the target's image size, the more pixels its corresponding first image frame contains, requiring more computation and taking longer to process. Similarly, the smaller the target's image size, the fewer pixels its corresponding first image frame contains, requiring less computation and taking less time to process. To reduce computational complexity while ensuring accuracy, the electronic device can pre-divide the target's minimum to maximum image size into multiple image size intervals based on the number of image levels in the image pyramid. Each image size interval corresponds to an image level and a feature point extraction step size. The larger the image size interval, the higher the image level and the larger the feature point extraction step size. The feature point extraction step size represents the number of rows of pixels between each feature point. For example, the minimum image size of the target object is 16*16, the maximum image size is 96*96, and the number of image levels in the image pyramid is 5. Then the image size interval [16, 32] corresponds to image level 1, and the corresponding feature point extraction step is 1; the image size interval (32, 48] corresponds to image level 2, and the corresponding feature point extraction step is 2; the image size interval (48, 64] corresponds to image level 3, and the corresponding feature point extraction step is 3; the image size interval (64, 80] corresponds to image level 4, and the corresponding feature point extraction step is 4; the image size interval (80, 96] corresponds to image level 5, and the corresponding feature point extraction step is 5. Among them, image level 1 is the bottom layer and image level 5 is the highest layer.

[0059] Furthermore, the electronic device can query the target image level and feature point extraction step size corresponding to the target object's image size from the pre-stored correspondence between image size ranges, image levels, and feature point extraction step sizes. For example, if the target object's image size is 96*96, the target image level and feature point extraction step size corresponding to the target object are 5 and 5, respectively.

[0060] It should be noted that the unit of the image size described in the embodiments of the present disclosure is usually pixel (px). For example, if the image size of the target object is 96*96, it means that the image size of the target object is 96px*96px.

[0061] Step three: based on the target image level, determine the target image from the image pyramid, and extract the first feature point of the first image frame in the target image based on the feature point extraction step size.

[0062] Exemplarily, after the electronic device determines the target image level and feature point extraction step corresponding to the target object, it can first determine the target image from the image pyramid based on the target image level. Then, the electronic device can perform gradient processing on the target image to obtain the gradient value of each pixel point contained in the target image. The gradient value of the pixel point represents the intensity change and direction of the image at the pixel point. The larger the gradient value of the pixel point, the more drastic the image change at the pixel point, that is, the more obvious the edge or texture. Further, the electronic device can start from the 1st row, first select the pixel point with the largest gradient value among the pixels in the 1st row as the first feature point corresponding to the 1st row of the first image frame. Then, the electronic device determines the k+1th row based on the feature point extraction step length k, and selects the pixel point with the largest gradient value among the pixels in the k+1th row as the first feature point corresponding to the k+1th row of the first image frame. The electronic device then determines the 2k+1th row based on the feature point extraction step size k, and selects the pixel with the largest gradient value among the pixels in the 2k+1th row as the first feature point corresponding to the 2k+1th row of the first image frame. This continues in this manner, ultimately obtaining the first feature point of the first image frame. For example, if the feature point extraction step size is 5, the electronic device selects the pixel with the largest gradient value in the 1st row, the 6th row, the 11th row, and so on, as the first feature point of the first image frame.

[0063] In an embodiment of the present application, an electronic device continuously downsamples a first image frame to obtain an image pyramid corresponding to the first image frame. The electronic device then determines a target image level and a feature point extraction step size based on the target object's image size. Based on the target image level, the electronic device determines a target image from the image pyramid and extracts first feature points from the target image based on the feature point extraction step size. As the target object's image size increases, the corresponding target image level increases, and the feature point extraction step size increases. A higher target image level has fewer pixels, requiring less computation and shortening computation time. A larger feature point extraction step size increases the number of pixel rows spaced apart, requiring less computation and shortening computation time, thereby reducing computation time and shortening computation time. Furthermore, as the target object's image size increases, the corresponding first image frame also increases in size. Even after multiple downsampling steps, the number of pixels in the image at the corresponding target image level remains high, thereby reducing computation time and ensuring computation accuracy.

[0064] In some embodiments, extracting the first feature point of the first image frame from the target image based on the feature point extraction step size includes the following steps:

[0065] Step 1: Determine the cropping ratio based on the target type of the target object.

[0066] Exemplarily, the first image frame of a target object typically includes a background image, and the proportion of the background image in the first image frame varies for different types of targets. For regularly shaped targets, the proportion of the background image in the corresponding first image frame is relatively small, while for irregularly shaped targets, the proportion of the background image in the corresponding first image frame is relatively large. For example, if the target object is a car, the proportion of the background image in the corresponding first image frame is relatively small; whereas, if the target object is a pedestrian, the proportion of the background image in the corresponding first image frame is relatively large. Based on this, the electronic device can pre-store a correspondence between target type and cropping ratio. For example, if the target object type is a pedestrian, the cropping ratio is 20%; if the target object type is a bicycle, motorcycle, or electric scooter, the cropping ratio is 15%; and if the target object type is a car, the cropping ratio is 10%. Furthermore, the electronic device can query the pre-stored correspondence between target type and cropping ratio to determine the cropping ratio corresponding to the target type.

[0067] Step 2: Crop the target image based on the cropping ratio to obtain a cropped target image.

[0068] For example, after the electronic device determines the target image from the image pyramid based on the target image level, it can further crop the target image based on the determined cropping ratio to obtain a cropped target image. The electronic device can crop the height of the target image based on the cropping ratio, or it can crop the width of the target image based on the cropping ratio, or it can crop both the height and width of the target image based on the cropping ratio, which is not limited in this disclosure. Different target types have different proportions of the background image in the height and width directions in their corresponding target images. Therefore, for targets of different target types, the electronic device can crop the height and / or width of the target image based on the proportions of the background image in the height and width directions in their corresponding target images, based on the cropping ratio. For example, if the target object is a pedestrian, and the background image in its corresponding target image has a smaller proportion in the height direction and a larger proportion in the width direction, the electronic device can crop the width of the target image based on the cropping ratio.

[0069] It should be noted that for different target types, the height and width ratios of the background image in the corresponding target image can be pre-set by a technician based on experience. Of course, those skilled in the art can also perform semantic segmentation on the target image of the target object to determine the height and width ratios of the background image in the corresponding target image, which is not limited in the present embodiment.

[0070] Step three: extracting the first feature point of the first image frame from the cropped target image based on the feature point extraction step size.

[0071] Exemplarily, after the electronic device obtains the cropped target image, it can start from the 1st row and first select the pixel with the largest gradient value among the pixels in the 1st row as the first feature point corresponding to the 1st row of the first image frame. Then, the electronic device determines the k+1th row based on the feature point extraction step length k, and selects the pixel with the largest gradient value among the pixels in the k+1th row as the first feature point corresponding to the k+1th row of the first image frame. After that, the electronic device determines the 2k+1th row based on the feature point extraction step length k, and selects the pixel with the largest gradient value among the pixels in the 2k+1th row as the first feature point corresponding to the 2k+1th row of the first image frame, and so on, to finally obtain the first feature point of the first image frame. For example, if the feature point extraction step length is 5, the electronic device selects the pixel with the largest gradient value in the 1st row, the 6th row, the 11th row, etc. as the first feature point of the first image frame.

[0072] In the disclosed embodiment, the electronic device determines a cropping ratio based on the target object's type, crops the target image based on the cropping ratio, and extracts a first feature point of the first image frame from the cropped target image based on the feature point extraction step size. Thus, by cropping the target image, the proportion of the background image within the target image is reduced, thereby reducing interference from the background image on feature point extraction and improving feature point extraction accuracy.

[0073] In some embodiments, the electronic device further performs the following steps:

[0074] In response to the number of first feature points in the first image frame being greater than or equal to a preset number threshold, the first feature points at the background position are deleted, and / or the first feature points whose feature point distance to other first feature points is less than or equal to a preset distance threshold are deleted.

[0075] For example, if too many first feature points are extracted from the first image frame, subsequent computations will be large and time-consuming. Therefore, if the number of first feature points in the first image frame is greater than or equal to a preset threshold, the electronic device may delete first feature points located in the background, and / or delete first feature points whose feature point distance from other first feature points is less than or equal to a preset distance threshold. Specifically, the electronic device may perform semantic segmentation on the target image to determine the background area and the target object area in the target image. Then, for each first feature point, the electronic device may determine whether the first feature point is located in the background area based on the coordinates of the first feature point. If the first feature point is located in the background area, the electronic device may delete the first feature point. If the first feature point is located in the target object area, the electronic device may retain the first feature point. At the same time, for any two first feature points, the electronic device may combine the two first feature points into a feature point pair. Then, for each feature point pair, the electronic device may determine the feature point distance of the feature point pair based on the coordinates of the two first feature points included in the feature point pair. Afterwards, the electronic device may determine a preset number of target feature point pairs with smaller feature point distances in ascending order of feature point distances, and delete the first feature point included in the target feature point pairs.

[0076] In the disclosed embodiment, when a first image frame contains a large number of first feature points, the electronic device deletes first feature points located in the background and / or deletes first feature points whose distance from other first feature points is less than or equal to a preset distance threshold. This effectively reduces the number of first feature points, thereby reducing the amount and time required for subsequent computations. Furthermore, deleting first feature points located in the background and those that are too close to each other effectively reduces noise interference from the feature points, thereby improving the accuracy of subsequent scale calculations.

[0077] In the above Figure 2 Based on the illustrated embodiment, step 1023 may include the following steps:

[0078] Step 1: for each first feature point, determine the displacement of the first feature point based on the coordinates of the first feature point and the coordinates of the second feature point corresponding to the first feature point.

[0079] Exemplarily, after the electronic device determines the first feature point of the first image frame and the second feature point of the second image frame, the electronic device can further determine, for each first feature point, the displacement of the first feature point between the first image frame and the second image frame based on the coordinates of the first feature point and the coordinates of the second feature point corresponding to the first feature point.

[0080] Step 2: In response to the absolute value of the difference between the displacement of the first feature point and the average displacement being greater than or equal to a preset displacement threshold, the first feature point is deleted from each first feature point, and the second feature point corresponding to the first feature point is deleted from each second feature point.

[0081] Exemplarily, after the electronic device determines the displacement of each first feature point between the first image frame and the second image frame, it can determine the average displacement based on the displacement of each first feature point. Then, for each first feature point, the electronic device can calculate the absolute value of the difference between the displacement of the first feature point and the average displacement, and determine whether the absolute value of the difference is greater than or equal to a preset displacement threshold. If the absolute value of the difference is greater than or equal to the preset displacement threshold, it means that the first feature point has jumped between the first image frame and the second image frame. Accordingly, the electronic device can delete the first feature point from each first feature point and delete the second feature point corresponding to the first feature point from each second feature point. If the absolute value of the difference is less than the preset displacement threshold, it means that the first feature point has not jumped between the first image frame and the second image frame. Accordingly, the electronic device can retain the first feature point and the second feature point corresponding to the first feature point.

[0082] Step three: determining a first scale factor of the target object at a second moment based on the first feature point and the second feature point.

[0083] For example, after eliminating the first feature point that experiences a jump and the second feature point corresponding to the first feature point, the electronic device can arbitrarily select two first feature points from each first feature point to form a first feature point pair, and then select the two second feature points corresponding to the two first feature points in the second image frame to form a second feature point pair. The electronic device can then determine a first scale factor of the target object between the first image frame and the second image frame based on the first feature point distance of the first feature point pair and the second feature point distance of the second feature point pair.

[0084] In the disclosed embodiment, for each first feature point, the electronic device determines the displacement of the first feature point based on the coordinates of the first feature point and the coordinates of the second feature point corresponding to the first feature point. If the absolute value of the difference between the displacement of the first feature point and the average displacement is greater than or equal to a preset displacement threshold, the electronic device deletes the first feature point and the second feature point corresponding to the first feature point. In this way, the electronic device can eliminate the first feature point that has experienced a jump and the second feature point corresponding to the first feature point before determining the first scale factor of the target object at the second moment based on the first feature point and the second feature point, thereby improving the accuracy of the scale factor calculation.

[0085] In the above Figure 2Based on the illustrated embodiment, step 1023 may include the following steps:

[0086] Step 1: Determine at least one first feature point pair in each first feature point, and determine at least one second feature point pair in each second feature point, wherein the two first feature points included in the first feature point pair correspond to the two second feature points included in the second feature point pair.

[0087] For example, the i-th first feature point in the first image frame can be represented as A i , the i-th first feature point A in the first image frame i The corresponding second feature point in the second image frame can be expressed as B i After the electronic device determines the first feature point of the first image frame and the second feature point of the second image frame, it can further arbitrarily select two first feature points A from each first feature point. i and A j , forming a first feature point pair. Among them, the two first feature points A i and A j The first feature point pair composed of can be expressed as CA ij For example, if the number of first feature points in the first image frame is 100, then the number of first feature point pairs is 4950. The first feature point pairs are (A1, A2), (A1, A3) ... (A1, A 100 ), (A2, A3), (A2, A4)... (A2, A 100 )……(A 99 , A 100 ).

[0088] Furthermore, the first feature point A i The corresponding second feature point B in the second image frame i and the first feature point A j The corresponding second feature point B in the second image frame j A second feature point pair can also be formed. Among them, two second feature points B i and B j The second feature point pair composed of can be expressed as CB ij The first feature point pair CA ij With the second feature point CB ij A set of feature point pairs with corresponding relationships can be formed. For example, if the number of second feature points in the second image frame is 100, then the number of second feature point pairs is also 4950. The second feature point pairs are (B1, B2), (B1, B3) ... (B1, B 100 ), (B2, B3), (B2, B4)... (B2, B 100 )……(B 99, B 100 ).

[0089] Step 2: Determine a first scale factor of the target object at a second moment based on the first feature point distance of the first feature point pair and the second feature point distance of the second feature point pair.

[0090] Exemplarily, the electronic device determines the first feature point pair CA ij After that, CA can be based on the first feature point ij The two first feature points A i and A j Coordinates of the first feature point pair CA ij The first feature point distance DA ij Among them, the first feature point is DA ij The first feature point pair CA ij The first feature point A i and A j Similarly, the electronic device determines the distance between the second feature point pair CB ij After that, CB can also be based on the second feature point ij The two second feature points B i and B j Coordinates of the second feature point pair CB ij The second feature point distance DB ij Among them, the second feature point is away from DB ij The second feature point pair CB ij The second feature point B i and B j Then, the electronic device can calculate the distance between the second feature point pair CB ij The second feature point distance DB ij CA with the first feature point ij The first feature point distance DA ij Afterwards, the electronic device calculates the ratio of the feature point distances of each corresponding feature point pair and further determines the median of each ratio as the first scale factor of the target object between the first image frame and the second image frame, that is, the first scale factor of the target object at the second moment.

[0091] It should be noted that the electronic device may also determine the average value of each ratio as the first scale factor of the target object between the first image frame and the second image frame, or may determine the weighted average value of each ratio as the first scale factor of the target object between the first image frame and the second image frame, which is not limited in the embodiments of the present disclosure.

[0092] In the embodiment of the present disclosure, in the embodiment of the present disclosure, the electronic device determines the median value of the feature point distance between two corresponding feature point pairs in the first image frame and the second image frame of non-adjacent frames as the first scale factor of the target object at the second moment, thereby improving the calculation accuracy of the scale factor and thus improving the accuracy of speed measurement.

[0093] like Figure 3 As shown in the above Figure 1 Based on the embodiment shown, step 103 may include the following steps:

[0094] Step 1031 : Perform linear fitting on the second scale factor and the first scale factor to obtain a scale factor fitting curve.

[0095] For example, even if the target object is rapidly accelerated or decelerated, its speed will not jump in a very short time. Similarly, the scale factor of the target object will not jump in a very short time. Therefore, in order to ensure that the scale factor of the target object is as stable as possible, the electronic device determines the first scale factor (original scale factor) of the target object at the second moment, and then obtains the second scale factor (original scale factor) of the target object at the third moment before the second moment. Among them, similar to the first scale factor of the target object at the second moment, the second scale factor of the target object at the third moment is also the original scale factor. Then, the electronic device can perform a linear fit on the second scale factor of the target object at the third moment and the first scale factor at the second moment to obtain the scale factor fitting curve of the target object. Among them, the scale factor fitting curve of the target object can characterize the trend or law of the scale factor of the target object changing over time.

[0096] Step 1032: Determine a first scale factor estimate based on the scale factor fitting curve.

[0097] For example, after obtaining the scale factor fitting curve of the target object, the electronic device may determine an estimated first scale factor value (corrected scale factor) of the target object at the second moment based on the scale factor fitting curve. Similarly, after determining the second scale factor (original scale factor) of the target object at the third moment, the electronic device may perform a linear fit on the second scale factor of the target object at the third moment and the scale factor before the third moment to obtain a scale factor fitting curve for the target object, and determine an estimated second scale factor value (i.e., the third scale factor (corrected scale factor)) of the target object at the third moment based on the scale factor fitting curve.

[0098] Step 1033 : Determine a fourth scale factor of the target object at the second moment based on the third scale factor and the estimated value of the first scale factor.

[0099] For example, after obtaining the estimated value of the first scale factor of the target object at the second moment, the electronic device may calculate an average scale factor with the third scale factor of the target object at the third moment (the corrected scale factor, the estimated value of the second scale factor of the target object at the third moment), and determine the average scale factor as the fourth scale factor of the target object at the second moment. Of course, those skilled in the art may also determine the median of the estimated value of the first scale factor of the target object at the second moment and the third scale factor of the target object at the third moment as the fourth scale factor of the target object at the second moment, or may determine the weighted value of the estimated value of the first scale factor of the target object at the second moment and the third scale factor of the target object at the third moment as the fourth scale factor of the target object at the second moment, and the embodiments of the present disclosure are not limited thereto.

[0100] In the disclosed embodiment, the electronic device performs a linear fit on the second scale factor (original scale factor) of the target object at the third moment and the first scale factor (original scale factor) at the second moment to obtain a scale factor fitting curve. The electronic device then determines an estimated first scale factor (corrected scale factor) based on the scale factor fitting curve. Subsequently, the electronic device determines a fourth scale factor of the target object at the second moment based on the third scale factor (corrected scale factor) and the estimated first scale factor (corrected scale factor) of the target object at the third moment. This makes the scale factor of the target object at the second moment determined by the electronic device more stable, thereby ensuring that the speed subsequently calculated based on the scale factor is also more stable.

[0101] like Figure 4 As shown in the above Figure 1 Based on the embodiment shown, step 104 may include the following steps:

[0102] Step 1041 : Determine the relative speed between the target object and the ego vehicle based on the fourth scale factor, the inter-frame interval, and the longitudinal distance between the ego vehicle and the target object at the second moment.

[0103] For example, the ratio of the height h2 of the detection frame of the target at the second moment (i.e., moment T2) to the height h1 of the detection frame of the target at the first moment (i.e., moment T1) is the fourth scale factor S of the target at the second moment, i.e., h2 / h1=S. Based on the optical imaging principle of the camera, the height h2 of the detection frame of the target at moment i is i The actual height H of the target, the relative distance between the target and the camera at the i-th moment (i.e., the longitudinal distance between the target and the vehicle at the i-th moment) X i And the focal length f of the camera. Specifically, the detection frame height h of the target object at the i-th moment i It is proportional to the actual height H of the target and the focal length f of the camera, and is also proportional to the relative distance between the target and the camera at the i-th moment (i.e., the longitudinal distance between the target and the vehicle at the i-th moment) Xi Inversely proportional. Therefore, the height of the detection frame of the target at the first moment (i.e., moment T1) is h1 = H*f / X1, and the height of the detection frame of the target at the second moment (i.e., moment T2) is h2 = H*f / X2. Therefore, S = h2 / h1 = (H*f / X2) / (H*f / X1) = (H*f / X2)*(X1 / (H*f)) = X1 / X2, that is, the ratio of the longitudinal distance X1 between the target and the vehicle at the first moment (i.e., moment T1) and the longitudinal distance X2 between the target and the vehicle at the second moment (i.e., moment T2) is the fourth scale factor S of the target at the second moment, i.e., X1 / X2 = S. The relative speed V of the target and the vehicle at the second moment (i.e., moment T2) is 相 V is the ratio of the difference between the longitudinal distance X2 between the target object and the vehicle at the second moment (i.e., time T2) and the longitudinal distance X1 between the target object and the vehicle at the first moment (i.e., time T1) (i.e., the relative movement distance of the target object between time T2 and time T1 relative to the vehicle) to the inter-frame interval dt, i.e., V 相 =(X2-X1) / dt. Substituting X1 / X2=S into the above formula, we can get the relative speed V between the target object and the vehicle at the second moment (i.e., time T2). 相 =(X2-X2*S) / dt=X2(1-S) / dt.

[0104] Step 1042: Determine a first velocity of the target object at a second moment based on the relative velocity and the vehicle velocity.

[0105] For example, the electronic device determines the relative speed V between the target object and the vehicle at the second moment (ie, moment T2). 相 After that, the relative speed V between the target object and the vehicle at the second moment (i.e., moment T2) can be calculated. 相 and the vehicle's speed V at the second moment 自 The sum of the values ​​is determined as the first velocity V of the target at the second moment. 目 =V 相 +V 自 =X2(1-S) / dt+V 自 .

[0106] In the disclosed embodiment, the electronic device can calculate the relative speed of the target object and the ego vehicle using the fourth scale factor and the longitudinal distance between the target object and the ego vehicle at the second moment, and further determine the first speed of the target object at the second moment based on the relative speed between the target object and the ego vehicle and the ego vehicle's speed at the second moment, thereby greatly improving the real-time performance of speed measurement.

[0107] In the above Figure 4 Based on the illustrated embodiment, step 1041 may include the following steps:

[0108] The product of the longitudinal distance and the fourth scale factor is determined; the difference between the longitudinal distance and the product is determined; and the ratio of the difference to the inter-frame interval is determined as the relative speed between the target object and the ego vehicle.

[0109] For example, the ratio of the longitudinal distance X1 between the target object and the ego vehicle at the first moment (i.e., moment T1) to the longitudinal distance X2 between the target object and the ego vehicle at the second moment (i.e., moment T2) is the fourth scale factor S of the target object at the second moment, i.e., X1 / X2=S. The relative speed V of the target object to the ego vehicle at the second moment (i.e., moment T2) is 相 V is the ratio of the difference between the longitudinal distance X2 between the target object and the vehicle at the second moment (i.e., time T2) and the longitudinal distance X1 between the target object and the vehicle at the first moment (i.e., time T1) (i.e., the relative movement distance of the target object between time T2 and time T1 relative to the vehicle) to the inter-frame interval dt, i.e., V 相 =(X2-X1) / dt. Substituting X1 / X2=S into the above formula, we can get the relative speed V between the target object and the vehicle at the second moment (i.e., time T2). 相 =(X2-X2*S) / dt=X2(1-S) / dt.

[0110] In the disclosed embodiment, the electronic device can calculate the relative speed between the target object and the vehicle using the fourth scale factor and the longitudinal distance between the target object and the vehicle at the second moment, thereby greatly improving the real-time performance of speed measurement.

[0111] In the above Figure 1 Based on the embodiment shown, the electronic device further performs the following steps:

[0112] Step 1: Obtain a second velocity of the target object at a fourth moment, wherein the fourth moment is earlier than the second moment.

[0113] For example, even if the target object accelerates or decelerates rapidly, its speed will not jump in a very short time. Therefore, in order to ensure that the speed of the target object is as stable as possible, after the electronic device determines the first speed of the target object at the second moment, it can also smooth and stabilize the first speed of the target object at the second moment based on the second speed at the fourth moment before the second moment. Therefore, the electronic device can obtain the second speed of the target object at the fourth moment. It should be noted that the fourth moment is the moment before the second moment, the fourth moment can be multiple moments, and the fourth moment can include the first moment.

[0114] Step 2: Determine a speed fitting curve of the target object based on the second speed and the first speed.

[0115] For example, after obtaining the second speed of the target object at the fourth moment, the electronic device may further perform a linear fit on the second speed of the target object at the fourth moment and the first speed of the target object at the second moment to obtain a speed fitting curve for the target object. The speed fitting curve for the target object may represent a trend or pattern in which the speed of the target object changes over time.

[0116] Step three: determine the third speed of the target object at the second moment based on the speed fitting curve.

[0117] Illustratively, after the electronic device determines the speed fitting curve of the target object, it may determine a third speed of the target object at the second moment based on the speed fitting curve.

[0118] In the embodiment of the present disclosure, the first speed of the target object at the second moment is smoothed and stabilized by the second speed of the target object at the fourth moment before the second moment, thereby ensuring the stability of the speed of the target object at the second moment.

[0119] In the above Figure 1 Based on the embodiment shown, the electronic device further performs the following steps:

[0120] A confidence level of a third speed is determined based on the second speed, the first speed, and the speed fitting curve.

[0121] Exemplarily, after the electronic device determines the speed fitting curve of the target object, it can determine the speed error of the target object at the fourth moment based on the speed fitting curve and the second speed of the target object at the fourth moment. Similarly, the electronic device can also determine the speed error of the target object at the second moment based on the speed fitting curve and the first speed of the target object at the second moment. Then, the electronic device can determine the average speed error based on the speed error of the target object at the fourth moment and the speed error at the second moment. Afterwards, the electronic device can calculate the sum of the average speed error and a preset error constant, and determine the ratio of the error constant to the sum as the confidence level of the third speed of the target object at the second moment. Among them, the fourth moment is the moment before the second moment, the fourth moment can be multiple moments, and the fourth moment can include the first moment. Those skilled in the art can set the error constant based on actual conditions, and the embodiments of the present disclosure are not limited thereto.

[0122] In the embodiment of the present disclosure, the electronic device determines the average speed error based on the second speed, the first speed and the speed fitting curve, and further determines the confidence level of the third speed of the target object at the second moment based on the average speed error, thereby evaluating the accuracy of the speed estimation and ensuring the safety of intelligent driving.

[0123] Exemplary devices

[0124] Figure 5FIG. 1 is a schematic diagram of the structure of an image processing device provided by an exemplary embodiment of the present disclosure. Figure 5 As shown, the speed measuring device 500 includes an image acquisition module 510 , a first scale factor determination module 520 , a second scale factor determination module 530 and a first speed determination module 540 .

[0125] An image acquisition module 510 is configured to acquire a first image frame and a second image frame containing a target object captured by the vehicle at a first moment and a second moment, wherein the first moment is earlier than the second moment;

[0126] A first scale factor determination module 520 is configured to determine a first scale factor of the target object at a second moment based on the first image frame and the second image frame;

[0127] A second scale factor determination module 530 is configured to determine a fourth scale factor of the target object at a second moment based on the second scale factor and the third scale factor of the target object at a third moment, and the first scale factor; the third moment is earlier than the second moment;

[0128] The first speed determination module 540 is configured to determine a first speed of the target object at the second moment based on the fourth scale factor, the inter-frame interval, the longitudinal distance between the ego vehicle and the target object at the second moment, and the ego vehicle speed.

[0129] In some embodiments, as Figure 6 As shown, the first scale factor determination module 520 includes:

[0130] The feature point extraction unit 521 is used to extract feature points from the first image frame to obtain first feature points of the first image frame;

[0131] An optical flow tracking unit 522 is configured to perform optical flow tracking on the first feature point based on the first image frame and the second image frame to obtain a second feature point corresponding to the first feature point in the second image frame;

[0132] The scale factor determining unit 523 is configured to determine a first scale factor of the target object at a second moment based on the first feature point and the second feature point.

[0133] In some embodiments, the feature point extraction unit 521 is specifically configured to:

[0134] Performing multi-scale processing on the first image frame to generate an image pyramid for the first image frame; the image pyramid includes a plurality of image levels;

[0135] Based on the image size of the target object, determine the target image level and feature point extraction step size;

[0136] A target image is determined from an image pyramid based on a target image level, and a first feature point of a first image frame is extracted from the target image based on a feature point extraction step size.

[0137] In some embodiments, the feature point extraction unit 521 is specifically configured to:

[0138] Determine the cropping ratio based on the target type of the target object;

[0139] Based on the cropping ratio, the target image is cropped to obtain a cropped target image;

[0140] Based on the feature point extraction step size, first feature points of the first image frame are extracted from the cropped target image.

[0141] In some embodiments, further comprising:

[0142] A feature point deletion module is used to delete the first feature points in the background position in response to the number of first feature points in the first image frame being greater than or equal to a preset number threshold, and / or to delete the first feature points whose feature point distance to other first feature points is less than or equal to a preset distance threshold.

[0143] In some embodiments, the scale factor determining unit 523 is specifically configured to:

[0144] For each first feature point, determining a displacement of the first feature point based on a coordinate of the first feature point and a coordinate of a second feature point corresponding to the first feature point;

[0145] In response to an absolute value of a difference between a displacement of the first feature point and an average displacement being greater than or equal to a preset displacement threshold, deleting the first feature point from each first feature point, and deleting a second feature point corresponding to the first feature point from each second feature point;

[0146] A first scale factor of the target object at a second moment is determined based on the first feature point and the second feature point.

[0147] In some embodiments, the scale factor determining unit 523 is specifically configured to:

[0148] Determine at least one first feature point pair in each first feature point, and determine at least one second feature point pair in each second feature point; the two first feature points included in the first feature point pair correspond to the two second feature points included in the second feature point pair respectively;

[0149] A first scale factor of the target object at a second moment is determined based on a first feature point distance of the first feature point pair and a second feature point distance of the second feature point pair.

[0150] In some embodiments, as Figure 7As shown, the second scale factor determination module 530 includes:

[0151] A linear fitting unit 531 is configured to perform a linear fitting on the second scale factor and the first scale factor to obtain a scale factor fitting curve;

[0152] a scale estimation unit 532 for determining a first scale factor estimate based on the scale factor fitting curve;

[0153] The scale factor determining unit 533 is configured to determine a fourth scale factor of the target object at the second moment based on the third scale factor and the estimated value of the first scale factor.

[0154] In some embodiments, as Figure 8 As shown, the first speed determination module 540 includes:

[0155] a relative speed determining unit 541 for determining a relative speed between the target object and the ego vehicle based on the fourth scale factor, the inter-frame interval, and the longitudinal distance between the ego vehicle and the target object at the second moment;

[0156] The speed determination unit 542 is configured to determine a first speed of the target object at a second moment based on the relative speed and the vehicle speed.

[0157] In some embodiments, the relative speed determination unit 541 is specifically configured to:

[0158] Determine the product of the longitudinal distance and the fourth scale factor;

[0159] Determine the difference between the longitudinal distance and the product;

[0160] The ratio of the difference to the inter-frame interval is determined as the relative speed between the target object and the vehicle.

[0161] In some embodiments, further comprising:

[0162] A speed acquisition module is used to acquire a second speed of the target object at a fourth moment; the fourth moment is earlier than the second moment;

[0163] A speed fitting module, configured to determine a speed fitting curve of the target object based on the second speed and the first speed;

[0164] The second speed determining module is configured to determine a third speed of the target object at a second moment based on the speed fitting curve.

[0165] In some embodiments, further comprising:

[0166] The confidence determination module is configured to determine the confidence of the third speed based on the second speed, the first speed, and the speed fitting curve.

[0167] The beneficial technical effects corresponding to the exemplary embodiment of this device can be found in the corresponding beneficial technical effects of the above exemplary method part, which will not be repeated here.

[0168] Exemplary electronic devices

[0169] Figure 9 A structural diagram of an electronic device provided in an embodiment of the present disclosure includes at least one processor 11 and a memory 12.

[0170] The processor 11 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0171] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), a hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute the one or more computer program instructions to implement the speed measurement method and / or other desired functions of the various embodiments of the present disclosure described above.

[0172] In one example, the electronic device 10 may further include an input device 13 and an output device 14 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0173] The input device 13 may also include, for example, a keyboard, a mouse, etc.

[0174] The output device 14 can output various information to the outside, and may include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0175] Of course, to simplify, Figure 9 Only some of the components related to the present disclosure in the electronic device 10 are shown, and components such as buses, input / output interfaces, etc. are omitted. In addition, the electronic device 10 may further include any other appropriate components according to specific application scenarios.

[0176] Exemplary computer program products and computer-readable storage media

[0177] In addition to the above methods and devices, embodiments of the present disclosure may also provide a computer program product, including computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the speed measurement method of various embodiments of the present disclosure described in the above "Exemplary Method" section.

[0178] The computer program product may be written in any combination of one or more programming languages ​​to implement the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0179] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the processor executes the steps of the speed measurement method of various embodiments of the present disclosure described in the above “Exemplary Method” section.

[0180] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium is, for example, but not limited to, a system, device or component comprising electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0181] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, the advantages, strengths, and effects mentioned in this disclosure are merely illustrative and not restrictive, and should not be considered as essential to each embodiment of the present disclosure. Furthermore, the specific details disclosed above are provided for illustrative purposes and to facilitate understanding, rather than as limitations. These details do not limit the present disclosure to necessarily being implemented using these specific details.

[0182] Those skilled in the art may make various changes and modifications to the present disclosure without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is intended to include these modifications and variations.

Claims

1. A speed measurement method, comprising: Acquire a first image frame and a second image frame containing the target object captured by the vehicle at a first moment and a second moment; The first moment is earlier than the second moment; determining a first scale factor of the target object at the second moment based on the first image frame and the second image frame; determining a fourth scale factor of the object at the second moment based on the second scale factor and the third scale factor of the object at the third moment and the first scale factor; The third moment is earlier than the second moment; A first speed of the target object at the second moment is determined based on the fourth scale factor, an inter-frame interval, a longitudinal distance between the ego vehicle and the target object at the second moment, and a speed of the ego vehicle.

2. The method according to claim 1, wherein The determining, based on the first image frame and the second image frame, a first scale factor of the target object at the second moment includes: Extracting feature points from the first image frame to obtain first feature points of the first image frame; Based on the first image frame and the second image frame, performing optical flow tracking on the first feature point to obtain a second feature point corresponding to the first feature point in the second image frame; A first scale factor of the target object at the second moment is determined based on the first feature point and the second feature point.

3. The method according to claim 2, wherein: The extracting feature points from the first image frame to obtain first feature points of the first image frame includes: performing multi-scale processing on the first image frame to generate an image pyramid for the first image frame; the image pyramid comprising a plurality of image levels; Determining the target image level and feature point extraction step size based on the image size of the target object; A target image is determined from the image pyramid based on the target image level, and first feature points of the first image frame are extracted from the target image based on the feature point extraction step size.

4. The method according to claim 3, wherein: Extracting the first feature point of the first image frame from the target image based on the feature point extraction step size includes: Determining a cropping ratio based on the target type of the target object; Cropping the target image based on the cropping ratio to obtain a cropped target image; Based on the feature point extraction step size, first feature points of the first image frame are extracted from the cropped target image.

5. The method according to claim 3, further comprising: In response to the number of first feature points of the first image frame being greater than or equal to a preset number threshold, the first feature points at the background position are deleted, and / or the first feature points whose feature point distance to other first feature points is less than or equal to a preset distance threshold are deleted.

6. The method according to claim 2, wherein: The determining, based on the first feature point and the second feature point, a first scale factor of the target object at the second moment includes: For each first feature point, determining a displacement of the first feature point based on a coordinate of the first feature point and a coordinate of a second feature point corresponding to the first feature point; In response to an absolute value of a difference between a displacement of the first feature point and an average displacement being greater than or equal to a preset displacement threshold, deleting the first feature point from each of the first feature points, and deleting a second feature point corresponding to the first feature point from each of the second feature points; A first scale factor of the target object at the second moment is determined based on the first feature point and the second feature point.

7. The method according to claim 2 or 6, wherein: The determining, based on the first feature point and the second feature point, a first scale factor of the target object at the second moment includes: Determine at least one first feature point pair in each of the first feature points, and determine at least one second feature point pair in each of the second feature points; the two first feature points included in the first feature point pair correspond to the two second feature points included in the second feature point pair, respectively; A first scale factor of the target object at a second moment is determined based on a first feature point distance of the first feature point pair and a second feature point distance of the second feature point pair.

8. The method according to claim 1, wherein The determining, based on the second scale factor and the third scale factor of the object at the third moment and the first scale factor, of the object at the second moment includes: performing a linear fit on the second scale factor and the first scale factor to obtain a scale factor fitting curve; determining the first scale factor estimate based on the scale factor fitting curve; A fourth scale factor of the target object at the second moment is determined based on the third scale factor and the first scale factor estimate.

9. The method according to claim 1, wherein Determining the first speed of the target object at the second moment based on the fourth scale factor, the inter-frame interval, the longitudinal distance between the vehicle and the target object at the second moment, and the vehicle speed includes: determining a relative speed between the target object and the ego vehicle based on the fourth scale factor, the inter-frame interval, and a longitudinal distance between the ego vehicle and the target object at the second moment; A first speed of the target object at the second moment is determined based on the relative speed and the vehicle speed.

10. The method according to claim 9, wherein: The determining, based on the fourth scale factor, the inter-frame interval, and the longitudinal distance between the ego vehicle and the target object at the second moment, of the relative speed between the target object and the ego vehicle includes: determining a product of the longitudinal distance and the fourth scale factor; determining a difference between the longitudinal distance and the product; The ratio of the difference to the inter-frame interval is determined as the relative speed between the target object and the vehicle.

11. The method according to claim 1 , further comprising: Obtaining a second velocity of the target object at a fourth moment; The fourth moment is earlier than the second moment; determining a velocity fitting curve of the target object based on the second velocity and the first velocity; Based on the speed fitting curve, a third speed of the target object at the second moment is determined.

12. The method according to claim 11, further comprising: A confidence level of the third speed is determined based on the second speed, the first speed, and the speed fitting curve.

13. A speed measuring device comprising: An image acquisition module is used to acquire a first image frame and a second image frame containing a target object captured by the vehicle at a first moment and a second moment; The first moment is earlier than the second moment; a first scale factor determining module, configured to determine a first scale factor of the target object at the second moment based on the first image frame and the second image frame; a second scale factor determining module, configured to determine a fourth scale factor of the object at the second moment based on the second scale factor and the third scale factor of the object at the third moment and the first scale factor; The third moment is earlier than the second moment; The first speed determination module is configured to determine a first speed of the target object at the second moment based on the fourth scale factor, an inter-frame interval, a longitudinal distance between the ego vehicle and the target object at the second moment, and a speed of the ego vehicle.

14. A computer-readable storage medium storing a computer program, wherein the computer program is used to execute the speed measurement method according to any one of claims 1 to 12.

15. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the speed measurement method described in any one of claims 1-12 above.

Images