A camera speed measurement method and related apparatus

Abstract

The application provides a camera speed measurement method and a related device. The method comprises the following steps: determining the external parameter of a target camera; acquiring a plurality of first ground images from an image acquisition module; determining feature point position information according to the plurality of first ground images and the external parameter; determining a first moving distance of a target vehicle in a transverse direction and a longitudinal direction according to the feature point position information; and determining a target vehicle speed of the target vehicle according to the first moving distance and an image acquisition time. Through a CMS camera lens, a static feature point on the ground is extracted, the feature point position information is determined through the camera external parameter and the matching feature point, the transverse and longitudinal moving distances of the vehicle are measured, the vehicle speed is further determined, the vehicle speed can be acquired by the rear-mounted CMS camera device, and the accuracy of the vehicle speed acquired by the camera is improved.

Description

Technical Field

[0001] This application belongs to the field of electronic rearview mirrors, specifically relating to a camera speed measurement method and related devices. Background Technology

[0002] Currently, vehicle speed is typically obtained by measuring wheel speed using Hall effect sensors in conjunction with GPS (Global Positioning System). However, aftermarket CMS (Camera Monitor System) devices cannot connect to the CAN (Controller Area Network) bus to obtain vehicle speed information. Furthermore, when obtaining vehicle speed using the aforementioned methods, the speed readings can be distorted when the vehicle travels on bumpy roads or when the wheels slip. Additionally, GPS positioning accuracy is not high, and there are errors in the positioning process, resulting in lag and inaccuracies in the vehicle speed obtained via GPS. Summary of the Invention

[0003] This application provides a camera speed measurement method and related device. By using a CMS camera lens, static feature points on the ground are extracted. The location information of the feature points is determined by the camera's extrinsic parameters and matching feature points. The lateral and longitudinal movement distances of the vehicle are measured, and the vehicle speed is further determined. This enables the aftermarket CMS camera device to acquire vehicle speed, which helps improve the accuracy of the camera in acquiring vehicle speed.

[0004] In a first aspect, embodiments of this application provide a camera speed measurement method, applied to a control module in a target camera. The target camera includes the control module and an image acquisition module. The control module is connected to the image acquisition module, and the image acquisition module is used to acquire images of the ground area in front of or below the target vehicle. The method includes:

[0005] Determine the extrinsic parameters of the target camera;

[0006] Acquire multiple frames of the first ground image from the image acquisition module;

[0007] Based on the multiple frames of the first ground image and the extrinsic parameters, the feature point location information is determined;

[0008] Based on the location information of the feature points, determine the first horizontal and vertical movement distance of the target vehicle;

[0009] The target speed of the target vehicle is determined based on the first moving distance and the image acquisition time.

[0010] In one possible example, the feature point location information includes first coordinates and second coordinates, and the static feature points include primary road surface feature points and secondary road surface feature points; determining the feature point location information based on the multiple frames of the first ground image and the extrinsic parameters includes:

[0011] Extract multiple primary road surface feature points from the first ground image of each frame based on the image acquisition area;

[0012] Extract multiple secondary road surface feature points from the first ground image of each frame based on the image acquisition area;

[0013] Remove interfering feature points from multiple primary road surface feature points and multiple secondary road surface feature points respectively;

[0014] Determine the first pixel coordinates of the primary road surface feature points and the second pixel coordinates of the secondary road surface feature points;

[0015] Based on the first pixel coordinates, intrinsic parameters, and extrinsic parameters, determine the first coordinates of the vehicle-associated feature point in the world coordinate system;

[0016] Based on the intrinsic and extrinsic parameters of the second pixel coordinates, the second coordinates of the road surface feature point in the world coordinate system are determined.

[0017] In one possible example, determining the first lateral and longitudinal movement distance of the target vehicle based on the feature point location information includes:

[0018] Determine the matching constraints for primary road surface feature points and secondary road surface feature points, wherein the matching constraints include topological invariance constraints and kinematic constraints;

[0019] Feature point matching is performed on the primary road surface feature points of two adjacent frames according to the matching constraints;

[0020] Feature point matching is performed on the secondary road surface feature points of two adjacent frames according to the matching constraints;

[0021] Based on the feature point location information, determine the first coordinate change value of the matched first-level road feature point;

[0022] Based on the feature point location information, determine the second coordinate change value of the matched secondary road surface feature point;

[0023] Based on the first coordinate change value and the second coordinate change value, the first horizontal and vertical movement distance of the target vehicle is determined.

[0024] In one possible example, determining the first coordinate change value of the matched primary road surface feature point based on the feature point location information includes:

[0025] The target second-level road feature point that matches the first-level road feature point of the current frame is used as the reference point, and other second-level road feature points other than the target second-level road feature point are normalized.

[0026] Based on the benchmark points, select multiple third-level pavement feature points from multiple second- and second-level pavement feature points;

[0027] Determine the confidence level of each third-level road feature point;

[0028] Determine the third coordinate change value between the first-level road feature point and each third-level road feature point in the current frame;

[0029] The first coordinate change value is determined based on the confidence level and the third coordinate change value.

[0030] In one possible example, determining the first lateral and longitudinal movement distance of the target vehicle based on the first coordinate change value and the second coordinate change value includes:

[0031] Obtain the state prediction value and Kalman gain determined based on historical movement distance data;

[0032] Determine the observation matrix;

[0033] The first horizontal and vertical movement distances of the target vehicle are determined based on the first coordinate change value, the second coordinate change value, the state prediction value, the Kalman gain, the observation matrix, and the preset movement distance formula.

[0034] In one possible example, before performing feature point matching on the primary road surface feature points of two adjacent frames according to the matching constraints, the method further includes:

[0035] If it is determined that the image acquisition module is acquiring the area below the target vehicle, the first descriptor and the second descriptor of the static feature points are normalized respectively. The first descriptor is a descriptor based on the gradient histogram, and the second descriptor is a binary descriptor.

[0036] Determine a first weight for the first descriptor and a second weight for the second descriptor, wherein the first weight is greater than the second weight;

[0037] A first fusion descriptor is generated based on the first weight, the second weight, the first descriptor, and the second descriptor;

[0038] If it is determined that the image acquisition module is acquiring the area in front of the target vehicle, the first descriptor and the second descriptor of the static feature points are normalized respectively. The first descriptor is a descriptor based on the gradient histogram, and the second descriptor is a binary descriptor.

[0039] Determine the third weight of the first descriptor and the fourth weight of the second descriptor, wherein the fourth weight is greater than the third weight;

[0040] A second fusion descriptor is generated based on the third weight, the fourth weight, the first descriptor, and the second descriptor.

[0041] In one possible example, determining the extrinsic parameters of the target camera includes:

[0042] Acquire the calibration board image from the image acquisition module and the intrinsic parameters of the target camera;

[0043] Determine the three-dimensional coordinates of the corner points of the calibration plate in the world coordinate system;

[0044] Based on the calibration board image, determine the two-dimensional pixel coordinates of the corner points;

[0045] The extrinsic parameters of the target camera are determined based on the intrinsic parameters, the three-dimensional coordinates of the corner points, and the two-dimensional pixel coordinates of the corner points.

[0046] Secondly, embodiments of this application provide a camera speed measuring device, applied to a control module in a target camera. The target camera includes the control module and an image acquisition module. The control module is connected to the image acquisition module, which is used to acquire data about the ground area in front of or below the target vehicle. The camera speed measuring device includes an acquisition unit, a judgment unit, a determination unit, an execution unit, and a transmission unit.

[0047] The determining unit is used to determine the extrinsic parameters of the target camera;

[0048] The acquisition unit is used to acquire multiple frames of first ground images from the image acquisition module;

[0049] The determining unit is further configured to determine the feature point location information based on the multiple frames of the first ground image and the external parameters;

[0050] The determining unit is further configured to determine the first horizontal and vertical movement distance of the target vehicle based on the feature point position information;

[0051] The determining unit is further configured to determine the target speed of the target vehicle based on the first moving distance and the image acquisition time.

[0052] A third aspect of this application provides an electronic device including: a processor and a memory; and one or more programs stored in the memory and configured to be executed by the processor, the programs including instructions for some or all of the steps as described in the first aspect.

[0053] A fourth aspect of this application provides a computer-readable storage medium for storing a computer program that causes a computer to perform some or all of the steps described in the first aspect of this application.

[0054] A fifth aspect of this application provides a computer program product, comprising a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps described in the first aspect of this application. This computer program product may be a software installation package.

[0055] As can be seen from the embodiments of this application, the control module first acquires multiple frames of first ground images from the image acquisition module to determine the extrinsic parameters of the target camera. Then, based on the multiple frames of first ground images and the extrinsic parameters, it determines the feature point location information. Next, based on the feature point location information, it determines the first horizontal and vertical movement distance of the target vehicle. Finally, based on the first movement distance and the image acquisition time, it determines the target vehicle speed. By extracting static ground feature points through the CMS camera lens, determining the feature point location information through camera extrinsic parameters and matching feature points, and measuring the horizontal and vertical movement distance of the vehicle, the vehicle speed is further determined. This enables the aftermarket CMS camera device to acquire vehicle speed, which helps improve the accuracy of the camera in acquiring vehicle speed. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This is a schematic diagram of the architecture of a target camera provided in an embodiment of this application;

[0058] Figure 2 This is a schematic flowchart of a camera speed measurement method provided in an embodiment of this application;

[0059] Figure 3 This is a schematic diagram of a process for determining location information provided in an embodiment of this application;

[0060] Figure 4 This is a flowchart illustrating the determination of a travel distance provided in an embodiment of this application;

[0061] Figure 5 This is a flowchart illustrating the determination of coordinate change values ​​provided in an embodiment of this application;

[0062] Figure 6 This is another flowchart for determining the travel distance provided in an embodiment of this application;

[0063] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0064] Figure 8 This is a block diagram of the functional units of a camera speed measuring device provided in an embodiment of this application. Detailed Implementation

[0065] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0066] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0067] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0068] In the embodiments of this application, "and / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent the following three situations: A exists alone; A and B exist simultaneously; B exists alone. Among them, A and B can be singular or plural.

[0069] In this embodiment, the symbol " / " can indicate that the preceding and following objects are in an "or" relationship. Alternatively, the symbol " / " can also represent a division sign, i.e., performing a division operation. For example, A / B can mean A divided by B.

[0070] In the embodiments of this application, "at least one item" or its similar expression refers to any combination of these items, including any combination of a single item or a plurality of items. "One or more" means one or more, while "multiple" means two or more. For example, "at least one item" of a, b, or c can represent the following seven cases: a, b, c; a and b; a and c; b and c; a, b, and c. Each of a, b, and c can be an element or a set containing one or more elements.

[0071] In the embodiments of this application, "equal to" can be used with "greater than" and is applicable to technical solutions used when "greater than" is used; it can also be used with "less than" and is applicable to technical solutions used when "less than" is used. When "equal to" is used with "greater than", it is not used with "less than"; when "equal to" is used with "less than", it is not used with "greater than".

[0072] To better understand the solutions of the embodiments of this application, the electronic devices, related concepts and background that may be involved in the embodiments of this application will be introduced below.

[0073] The electronic device in this application embodiment is a device with wireless communication capabilities, and may be referred to as a terminal, user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal device, vehicle-mounted terminal device, industrial control terminal device, UE unit, UE station, mobile station, remote station, remote terminal device, mobile device, UE terminal device, wireless communication device, UE agent, or UE device, etc. The terminal device can be fixed or mobile. It should be noted that the terminal device can support at least one wireless communication technology, such as LTE, New Radio (NR), Wideband Code Division Multiple Access (WCDMA), etc. For example, terminal devices can be mobile phones, tablets, desktop computers, laptops, all-in-one computers, in-vehicle terminals, virtual reality (VR) terminal devices, augmented reality (AR) terminal devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, electronic devices or other processing devices connected to a wireless modem, wearable devices, terminal devices in future mobile communication networks, or terminal devices in future evolved public land mobile networks (PLMNs), etc.

[0074] Please see Figure 1 , Figure 1 This is a schematic diagram of the architecture of a target camera provided in an embodiment of this application. The target camera 1 includes a control module 10 and an image acquisition module 20, and the control module 10 is connected to the image acquisition module 20.

[0075] The target camera is a CMS camera. The aftermarket CMS camera is mounted on the target vehicle at a suitable angle and position, ensuring the camera lens can clearly capture the ground area in front of or below the vehicle. The camera's tilt and horizontal angles are adjusted so that the ground occupies a suitable area in the camera's field of view, while avoiding factors that affect image quality, such as obstructions or reflections.

[0076] The control module 10 can be an electronic rearview mirror chip, which includes an ISP image signal processor and an NPU neural network processing unit.

[0077] The image acquisition module 20 can be a camera, which can acquire raw image data and transmit the raw image data to the control module 10.

[0078] In one possible example, the control module 10 first acquires multiple frames of first ground images from the image acquisition module 20 to determine the extrinsic parameters of the target camera. Then, based on the multiple frames of first ground images and the extrinsic parameters, the control module 10 determines the feature point location information. Next, based on the feature point location information, the control module 10 determines the first lateral and longitudinal movement distance of the target vehicle. Finally, based on the first movement distance and the image acquisition time, the control module 10 determines the target vehicle speed. By extracting static ground feature points through the CMS camera lens, determining the feature point location information through camera extrinsic parameters and matching feature points, and measuring the lateral and longitudinal movement distance of the vehicle, the vehicle speed is further determined. This enables the aftermarket CMS camera device to acquire vehicle speed, which helps improve the accuracy of the camera in acquiring vehicle speed.

[0079] Please see Figure 2 , Figure 2 This is a flowchart illustrating a camera speed measurement method provided in an embodiment of this application. The method is applied to a control module in a target camera, which includes the control module and an image acquisition module. The control module is connected to the image acquisition module, which is used to acquire images of the ground area in front of or below the target vehicle. The method includes:

[0080] Step S201: Determine the extrinsic parameters of the target camera.

[0081] Among them, a calibration plate with known three-dimensional coordinates can be used to calibrate the extrinsic parameters of the target camera.

[0082] Step S202: Acquire multiple frames of the first ground image from the image acquisition module.

[0083] The image acquisition module transmits the acquired ground images to the control module.

[0084] Step S203: Determine the feature point location information based on the multiple frames of the first ground image and the extrinsic parameters.

[0085] Among them, static feature points of the first ground image can be extracted, and the coordinates of the static feature points in the world coordinate system can be determined based on extrinsic parameters, that is, the feature point location information is obtained.

[0086] Step S204: Determine the first horizontal and vertical movement distance of the target vehicle based on the feature point position information.

[0087] In this process, feature points in two adjacent frames are matched, and the first horizontal and vertical movement distance of the target vehicle is determined based on the coordinate changes between the two matched feature points.

[0088] The first moving distance includes the lateral moving distance and the longitudinal moving distance of the target vehicle.

[0089] Step S205: Determine the target vehicle speed based on the first moving distance and image acquisition time.

[0090] Specifically, the vehicle speed is calculated based on the first horizontal and vertical distances measured by the camera and the time interval between image acquisitions. The target vehicle speed is calculated as the first moving distance plus the image acquisition time.

[0091] As can be seen from the embodiments of this application, the control module first acquires multiple frames of first ground images from the image acquisition module to determine the extrinsic parameters of the target camera. Then, based on the multiple frames of first ground images and the extrinsic parameters, it determines the feature point location information. Next, based on the feature point location information, it determines the first horizontal and vertical movement distance of the target vehicle. Finally, based on the first movement distance and the image acquisition time, it determines the target vehicle speed. By extracting static ground feature points through the CMS camera lens, determining the feature point location information through camera extrinsic parameters and matching feature points, and measuring the horizontal and vertical movement distance of the vehicle, the vehicle speed is further determined. This enables the aftermarket CMS camera device to acquire vehicle speed, which helps improve the accuracy of the camera in acquiring vehicle speed.

[0092] Please see Figure 3 , Figure 3 This is a flowchart illustrating a method for determining location information according to an embodiment of this application. The feature point location information includes a first coordinate and a second coordinate. Determining the feature point location information based on the multiple frames of the first ground image and the extrinsic parameters includes:

[0093] Step S301: Extract multiple primary road surface feature points from each frame of the first ground image based on the image acquisition area.

[0094] Among them, static feature points can be classified based on spatial stability, texture recognition, and anti-interference ability to obtain primary road surface feature points and secondary road surface feature points.

[0095] Among them, the primary road surface feature points corresponding to the images collected in front of the vehicle include the intersection of white / yellow road markings, the endpoints of the markings, and the bottom connection points of the curb stones, etc. The primary road surface feature points corresponding to the images collected below the vehicle include the inflection points of the anti-skid texture of the road surface, the corner points of the manhole cover edges, and the intersection points of the fixed cracks in the road surface, etc.

[0096] Step S302: Extract multiple secondary road surface feature points from each frame of the first ground image based on the image acquisition area.

[0097] The secondary road surface feature points corresponding to the image area collected in front of the vehicle include uniform road surface texture points and shoulder edge points. The secondary road surface feature points corresponding to the image area collected below the vehicle include fixed ground protrusions and drainage ditch edge points.

[0098] Step S303: Remove interfering feature points from multiple primary road surface feature points and multiple secondary road surface feature points respectively.

[0099] Interference points include, but are not limited to, road surface water stains, shadows, fallen leaves, drifting objects, chassis obstructions, and ground reflections. Interference points can affect the accuracy of subsequent vehicle speed determination.

[0100] Step S304: Determine the first pixel coordinates of the primary road surface feature points and the second pixel coordinates of the secondary road surface feature points.

[0101] Wherein, the first pixel coordinates are the two-dimensional pixel coordinates of the primary road surface feature points, and the second pixel coordinates are the two-dimensional pixel coordinates of the secondary road surface feature points.

[0102] Step S305: Determine the first coordinates of the primary road surface feature point in the world coordinate system based on the first pixel coordinates, the intrinsic parameters, and the extrinsic parameters.

[0103] Step S306: Determine the second coordinates of the secondary road surface feature point in the world coordinate system based on the intrinsic and extrinsic parameters of the second pixel coordinates.

[0104] Among them, the first coordinate and the second coordinate are three-dimensional coordinates.

[0105] As can be seen in this example, by extracting static road surface feature points in a hierarchical manner and assigning different weights to feature points of different levels, it is beneficial to improve the accuracy of vehicle speed determination.

[0106] Please see Figure 4 , Figure 4This is a flowchart of a method for determining a moving distance according to an embodiment of this application. The method involves determining the first moving distance of the target vehicle in both the lateral and longitudinal directions based on the feature point location information. The method may include the following steps:

[0107] Step S401: Determine the matching constraints between primary road feature points and secondary road feature points, wherein the matching constraints include topological invariance constraints and kinematic constraints;

[0108] The topological invariance constraints include Euclidean distance constraints and angle constraints. Euclidean distance constraints mean that the relative Euclidean distance deviation between two feature points is less than or equal to a first preset distance threshold. Angle constraints mean that the angle deviation between two feature points is less than or equal to a preset angle threshold. The first preset distance threshold and the preset angle threshold can be set manually or by system default, and are not limited here.

[0109] Among them, kinematic constraints refer to the lateral pixel displacement of two feature points being less than or equal to a second preset distance threshold and the vertical pixel displacement being less than or equal to a third preset distance threshold.

[0110] Only two feature points that simultaneously satisfy both topological invariance constraints and kinematic constraints can be successfully matched.

[0111] Step S402: Perform feature point matching on the primary road surface feature points of two adjacent frames according to the matching constraints.

[0112] The method for finding the corresponding matching second-level road surface feature point of the previous frame for the first-level road surface feature point of each current frame is as follows: calculate the matching distance between the current first-level road surface feature point and each second-level road surface feature point of the previous frame, select the second-level road surface feature point with the largest matching distance, and if the matching distance of the second-level road surface feature point is greater than the preset similarity threshold, then determine the current first-level road surface feature point and the second-level road surface feature point.

[0113] Among them, the matching distance is related to the similarity of feature descriptors, the spatial topological consistency of feature points, the gray-level correlation of pixel neighborhoods, and the scale / direction consistency of feature points.

[0114] Among them, the relationship between matching distance and feature descriptor similarity is that the smaller the descriptor distance, the higher the overlap of texture features, the stronger the similarity, and the higher the matching distance.

[0115] The relationship between matching distance and spatial topological consistency of feature points is that the smaller the relative Euclidean distance deviation and the smaller the relative angle deviation, the higher the matching distance.

[0116] Among them, the relationship between matching distance and gray-level correlation of pixel neighborhood is that the higher the cross-correlation coefficient of neighborhood gray levels, the stronger the overlap of gray-level distribution, and the smaller the deviation of gray-level mean / variance, the higher the matching distance.

[0117] Among them, the relationship between matching distance and scale / direction consistency of feature points is that the smaller the difference in scale levels and the smaller the deviation in the main direction angle, the stronger the consistency and the higher the matching distance.

[0118] Optionally, different weights can be assigned to the four dimensions of feature descriptor similarity, spatial topological consistency of feature points, gray-level correlation of pixel neighborhood, and scale / direction consistency of feature points. For example, the weights assigned to feature descriptor similarity, spatial topological consistency of feature points, gray-level correlation of pixel neighborhood, and scale / direction consistency of feature points are 50%, 25%, 15%, and 10%, respectively.

[0119] Step S403: Perform feature point matching on the secondary road surface feature points of two adjacent frames according to the matching constraints.

[0120] The method for finding the corresponding second-level road surface feature point of the previous frame for each first-level road surface feature point in the current frame is as follows: calculate the matching distance between the current first-level road surface feature point and each second-level road surface feature point in the previous frame, select the second-level road surface feature point with the largest matching distance, and if the matching distance of the second-level road surface feature point is greater than the preset similarity threshold, then determine the current first-level road surface feature point and the second-level road surface feature point.

[0121] Step S404: Determine the first coordinate change value of the matched first-level road surface feature point based on the feature point location information.

[0122] Step S405: Determine the second coordinate change value of the matched secondary road surface feature point based on the feature point location information.

[0123] Step S406: Determine the first horizontal and vertical movement distance of the target vehicle based on the first coordinate change value and the second coordinate change value.

[0124] Different weights are assigned to the first coordinate change value and the second coordinate change value. For example, the weights assigned to the first coordinate change value and the second coordinate change value are 70% and 30%, respectively.

[0125] Considering that there are multiple road surface feature points and multiple coordinate change values, the average of the first coordinate change value and the second coordinate change value can be calculated first, and then the first movement distance can be calculated according to the weight.

[0126] As can be seen, in this example, matching constraints can be set and feature point matching can be performed, which helps to improve the accuracy of determining the movement distance.

[0127] Please see Figure 5 In determining the first coordinate change value of the matched vehicle-associated feature point, the steps include:

[0128] Step S501: Take the target second-level road feature point that matches the first-level road feature point in the current frame as the reference point, and normalize the other second-level road feature points other than the target second-level road feature point.

[0129] The normalization is performed according to the following formula:

[0130]

[0131] in, This represents the normalized matching distance. This indicates the matching distance of the currently processed primary road surface feature points. This represents the largest matching distance among multiple matching distances for multiple primary road surface feature points.

[0132] Step S502: Based on the reference point, select multiple third-level pavement feature points from multiple second- and second-level pavement feature points.

[0133] Specifically, second-level road feature points that are within a fifth preset distance from the reference point in the world coordinate system can be selected as third-level road feature points. The fifth preset distance can be set manually or by system default, and is not limited here.

[0134] Step S503: Determine the confidence level of each third-level road feature point.

[0135] The confidence level is related to the distance between the third-level pavement feature point and the reference point. The greater the distance, the lower the confidence level, and the smaller the distance, the greater the confidence level.

[0136] Step S504: Determine the third coordinate change value between the first-level road feature point and each third-level road feature point in the current frame.

[0137] The third coordinate change value is the specific numerical value of the coordinate deviation between the first-level road surface feature point and the third-level road surface feature point.

[0138] Among them, the target coordinate change value between the first-level road feature point and the target first-level road feature point in the current frame can be determined.

[0139] Step S505: Determine the first coordinate change value based on the confidence level and the third coordinate change value.

[0140] Specifically, based on the confidence level, weights are assigned to the target coordinate change value and the third coordinate change value; the higher the confidence level, the greater the weight assigned to the third coordinate change value. Thus, a more accurate first coordinate change value is calculated by weighting the target coordinate change value corresponding to the second-level road feature point and the third coordinate change values ​​of multiple third-level road feature points.

[0141] Similarly, the change value of the second coordinate is calculated in the manner described above.

[0142] As can be seen, in this example, calculating the coordinate change value by using a multi-feature point fusion filtering method is beneficial to improving the accuracy of the coordinate change value.

[0143] Please see Figure 6 Regarding determining the first longitudinal and lateral movement distance of the target vehicle based on the first coordinate change value and the second coordinate change value, the above method may include the following steps:

[0144] Step S601: Obtain the state prediction value and Kalman gain determined based on historical travel distance data.

[0145] Among them, the current frame's movement distance can be predicted based on historical movement distance data, i.e., the state prediction value.

[0146] Step S602: Determine the observation matrix.

[0147] Step S603: Determine the first horizontal and vertical movement distance of the target vehicle based on the first coordinate change value, the second coordinate change value, the state prediction value, the Kalman gain, the observation matrix, and the preset movement distance formula.

[0148] The weights of the first and second coordinate changes can be assigned, and the weighted calculation is performed to obtain the fourth coordinate change value.

[0149] The formula for the preset movement distance is as follows:

[0150]

[0151] in, Indicates the first distance traveled. Indicates the predicted state value. Indicates Kalman gain, H represents the change value of the fourth coordinate, and H represents the observation matrix.

[0152] As can be seen, in this example, using the Kalman filter algorithm to calculate the first lateral and longitudinal movement distance of the vehicle helps to improve the accuracy of determining the movement distance.

[0153] In one possible example, before performing feature point matching on the primary road surface feature points of two adjacent frames according to the matching constraints, the above method may include the following steps: If it is determined that the image acquisition module has acquired the area below the target vehicle, normalize the first descriptor and the second descriptor of the static feature points respectively, wherein the first descriptor is a gradient histogram-based descriptor and the second descriptor is a binary descriptor; determine the first weight of the first descriptor and the second weight of the second descriptor, wherein the first weight is greater than the second weight; generate a first fused descriptor based on the first weight, the second weight, the first descriptor and the second descriptor; If it is determined that the image acquisition module has acquired the area in front of the target vehicle, normalize the first descriptor and the second descriptor of the static feature points respectively, wherein the first descriptor is a gradient histogram-based descriptor and the second descriptor is a binary descriptor; determine the third weight of the first descriptor and the fourth weight of the second descriptor, wherein the fourth weight is greater than the third weight; generate a second fused descriptor based on the third weight, the fourth weight, the first descriptor and the second descriptor.

[0154] The first, second, third, and fourth weights can be preset, with the first weight being greater than the second weight and the fourth weight being greater than the third weight.

[0155] The first fusion descriptor is obtained by weighting the first descriptor and the second descriptor based on the first weight and the second weight, and the second fusion descriptor is obtained by weighting the first descriptor and the second descriptor based on the third weight and the fourth weight.

[0156] The first descriptor is the DAISY (Densely Applicable Invariant and Scalable) gradient histogram descriptor, and the second descriptor is the ORB binary descriptor. Considering that the area below the vehicle is a low-light region, while the area in front of the vehicle is a relatively well-lit region, the first descriptor is weighted greater than the second, for example, set to 60% and 40% respectively. The first descriptor dominates (60%) to ensure recognition under weak textures, while the second descriptor assists (40%) to compensate for the slow matching speed of DAISY and improve overall efficiency. The fourth descriptor is weighted greater than the third, for example, set to 30% and 70% respectively. Since the area in front has relatively sufficient lighting and clear textures, the second descriptor dominates (70%), while the first descriptor assists (30%) to ensure matching speed.

[0157] As can be seen, in this example, setting different descriptor weights for different acquisition areas is beneficial to improving the accuracy and efficiency of feature point matching in the future.

[0158] In one possible example, determining the extrinsic parameters of the target camera includes: acquiring a calibration board image from the image acquisition module and the intrinsic parameters of the target camera; determining the three-dimensional coordinates of the calibration board corner points in the world coordinate system; determining the two-dimensional pixel coordinates of the corner points based on the calibration board image; and determining the extrinsic parameters of the target camera based on the intrinsic parameters, the three-dimensional coordinates of the corner points, and the two-dimensional pixel coordinates of the corner points.

[0159] As can be seen, in this example, calibrating the camera's extrinsic parameters helps to improve the accuracy of determining the vehicle's movement distance.

[0160] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, which is a control module applied in a target camera. The target camera includes the control module and an image acquisition module. The control module is connected to the image acquisition module, and the image acquisition module is used to acquire images of the ground area in front of or below a target vehicle. Figure 7 As shown, the electronic device includes a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs are configured to be executed by the processor according to the following instructions:

[0161] Determine the extrinsic parameters of the target camera;

[0162] Acquire multiple frames of the first ground image from the image acquisition module;

[0163] Based on the multiple frames of the first ground image and the extrinsic parameters, the feature point location information is determined;

[0164] Based on the location information of the feature points, determine the first horizontal and vertical movement distance of the target vehicle;

[0165] The target speed of the target vehicle is determined based on the first moving distance and the image acquisition time.

[0166] As can be seen, in this embodiment, the electronic device first acquires multiple frames of first ground images from the image acquisition module to determine the extrinsic parameters of the target camera. Then, based on the multiple frames of first ground images and the extrinsic parameters, it determines the feature point location information. Next, based on the feature point location information, it determines the first horizontal and vertical movement distance of the target vehicle. Finally, based on the first movement distance and the image acquisition time, it determines the target vehicle speed. By extracting static ground feature points through the CMS camera lens, determining the feature point location information through camera extrinsic parameters and matching feature points, and measuring the horizontal and vertical movement distance of the vehicle, the vehicle speed is further determined. This enables the aftermarket CMS camera device to acquire vehicle speed, which helps improve the accuracy of the camera in acquiring vehicle speed.

[0167] The above primarily describes the solutions of the embodiments of this application from the perspective of the method execution process. It is understood that, in order to achieve the above functions, the electronic device includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments provided herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0168] This application embodiment can divide the electronic device into functional units according to the above method example. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0169] When dividing each function into modules according to its corresponding function. Figure 8 A functional block diagram of a camera speed measuring device is provided, which is applied to the control module of a target camera. The target camera includes the control module and an image acquisition module. The control module is connected to the image acquisition module, which is used to acquire images of the ground area in front of or below the target vehicle. Figure 8 As shown, the camera speed measuring device includes a determining unit 801 and an acquiring unit 802; wherein,

[0170] The determining unit 801 is used to determine the extrinsic parameters of the target camera;

[0171] The acquisition unit 802 is used to acquire multiple frames of first ground images from the image acquisition module;

[0172] The determining unit 801 is further configured to determine the feature point location information based on the multiple frames of the first ground image and the external parameters;

[0173] The determining unit 801 is further configured to determine the first horizontal and vertical movement distance of the target vehicle based on the feature point position information.

[0174] The determining unit 801 is further configured to determine the target speed of the target vehicle based on the first moving distance and the image acquisition time.

[0175] As can be seen from the embodiments of this application, the camera speed measuring device first acquires multiple frames of first ground images from the image acquisition module to determine the extrinsic parameters of the target camera. Then, based on the multiple frames of first ground images and the extrinsic parameters, it determines the feature point location information. Next, based on the feature point location information, it determines the first horizontal and vertical movement distance of the target vehicle. Finally, based on the first movement distance and the image acquisition time, it determines the target vehicle speed. By extracting static feature points on the ground through the CMS camera lens, determining the feature point location information through the camera extrinsic parameters and matching feature points, and measuring the horizontal and vertical movement distance of the vehicle, the vehicle speed is further determined. This enables the aftermarket CMS camera device to acquire vehicle speed, which helps improve the accuracy of the camera in acquiring vehicle speed.

[0176] It should be noted that all relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0177] The electronic device provided in this embodiment is used to execute the above-described camera speed measurement method, and therefore can achieve the same effect as the above-described implementation method.

[0178] When using integrated units, the electronic device may include a processing module, a storage module, and a communication module. The processing module can be used to control and manage the actions of the electronic device; for example, it can support the electronic device in executing the steps performed by the aforementioned functional units. The storage module can support the electronic device in executing stored program code and data. The communication module can support communication between the electronic device and other devices.

[0179] The processing module can be a processor or a controller. It can implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module can be a memory. The communication module can specifically be a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, or other devices that interact with other electronic devices.

[0180] This application also provides a computer storage medium storing a computer program for electronic data interchange, which causes a computer to perform some or all of the steps of any of the methods described in the above method embodiments, wherein the computer includes an electronic device.

[0181] This application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments. The computer program product may be a software installation package, and the computer includes a control platform.

[0182] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0183] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0184] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0185] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0186] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0187] If the aforementioned integrated units are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0188] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage device, which may include: a flash drive, a read-only memory, a random access memory, a magnetic disk, or an optical disk, etc.

[0189] The embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A camera speed measurement method, characterized in that, A control module is applied in a target camera, the target camera including the control module and an image acquisition module, the control module being connected to the image acquisition module, the image acquisition module being used to acquire ground areas in front of or below the target vehicle; the method includes: Determine the extrinsic parameters of the target camera; Acquire multiple frames of the first ground image from the image acquisition module; Based on the multiple frames of the first ground image and the external parameters, the feature point location information is determined. The static feature points include primary road surface feature points and secondary road surface feature points. The feature point location information includes the first coordinates of the primary road surface feature points and the second coordinates of the secondary road surface feature points among the static feature points. Based on the location information of the feature points, determine the first horizontal and vertical movement distance of the target vehicle; The target speed of the target vehicle is determined based on the first moving distance and the image acquisition time. The step of determining the first horizontal and vertical movement distance of the target vehicle based on the feature point location information includes: Determine the matching constraints for primary road surface feature points and secondary road surface feature points, wherein the matching constraints include topological invariance constraints and kinematic constraints; Feature point matching is performed on the primary road surface feature points of two adjacent frames according to the matching constraints; Feature point matching is performed on the secondary road surface feature points of two adjacent frames according to the matching constraints; The target second-level road feature point that matches the first-level road feature point of the current frame is used as the reference point, and other second-level road feature points other than the target second-level road feature point are normalized. Based on the benchmark points, select multiple third-level pavement feature points from multiple second-level pavement feature points; Determine the confidence level of each third-level road feature point; Determine the third coordinate change value between the first-level road feature point and each third-level road feature point in the current frame; The first coordinate change value is determined based on the confidence level and the third coordinate change value; Based on the feature point location information, determine the second coordinate change value of the matched secondary road surface feature point; Based on the first coordinate change value and the second coordinate change value, the first horizontal and vertical movement distance of the target vehicle is determined.

2. The method according to claim 1, characterized in that, Based on the multiple frames of the first ground image and the extrinsic parameters, the feature point location information is determined, including: Extract multiple primary road surface feature points from the first ground image of each frame based on the image acquisition area; Extract multiple secondary road surface feature points from the first ground image of each frame based on the image acquisition area; Remove interfering feature points from multiple primary road surface feature points and multiple secondary road surface feature points respectively; Determine the first pixel coordinates of the primary road surface feature points and the second pixel coordinates of the secondary road surface feature points; Based on the first pixel coordinates, camera intrinsic parameters, and the extrinsic parameters, determine the first coordinates of the first-level road feature point in the world coordinate system; Based on the second pixel coordinates, the camera intrinsic parameters, and the extrinsic parameters, the second coordinates of the secondary road surface feature points in the world coordinate system are determined.

3. The method according to claim 1, characterized in that, Based on the first coordinate change value and the second coordinate change value, the first lateral and longitudinal movement distance of the target vehicle is determined, including: Obtain the state prediction value and Kalman gain determined based on historical movement distance data; Determine the observation matrix; The first horizontal and vertical movement distances of the target vehicle are determined based on the first coordinate change value, the second coordinate change value, the state prediction value, the Kalman gain, the observation matrix, and the preset movement distance formula.

4. The method according to claim 1, characterized in that, Before performing feature point matching on the primary road surface feature points of two adjacent frames according to the matching constraints, the method further includes: If it is determined that the image acquisition module is acquiring the area below the target vehicle, then the first and second descriptors of the primary road surface feature points are normalized, and the first and second descriptors of the secondary road surface feature points are also normalized. The first descriptor is a descriptor based on the gradient histogram, and the second descriptor is a binary descriptor. Determine a first weight for the first descriptor and a second weight for the second descriptor, wherein the first weight is greater than the second weight; A first fusion descriptor is generated based on the first weight, the second weight, the first descriptor, and the second descriptor; If it is determined that the image acquisition module is acquiring the area in front of the target vehicle, then the first descriptor and the second descriptor of the primary road surface feature points are normalized, and the first descriptor and the second descriptor of the secondary road surface feature points are also normalized. The first descriptor is a descriptor based on the gradient histogram, and the second descriptor is a binary descriptor. Determine the third weight of the first descriptor and the fourth weight of the second descriptor, wherein the fourth weight is greater than the third weight; A second fusion descriptor is generated based on the third weight, the fourth weight, the first descriptor, and the second descriptor.

5. The method according to claim 1, characterized in that, Determining the extrinsic parameters of the target camera includes: Acquire the calibration board image from the image acquisition module and the camera intrinsic parameters of the target camera; Determine the three-dimensional coordinates of the corner points of the calibration plate in the world coordinate system; Based on the calibration board image, determine the two-dimensional pixel coordinates of the corner points; The extrinsic parameters of the target camera are determined based on the camera intrinsic parameters, the three-dimensional coordinates of the corner points, and the two-dimensional pixel coordinates of the corner points.

6. A camera speed measuring device, characterized in that, A control module is applied in a target camera, the target camera including the control module and an image acquisition module, the control module being connected to the image acquisition module, the image acquisition module being used to acquire images of the ground area in front of or below the target vehicle; the camera speed measuring device includes an acquisition unit and a determination unit; wherein... The determining unit is used to determine the extrinsic parameters of the target camera; The acquisition unit is used to acquire multiple frames of first ground images from the image acquisition module; The determining unit is further configured to determine feature point location information based on the multiple frames of the first ground image and the external parameters. The static feature points include primary road surface feature points and secondary road surface feature points. The feature point location information includes the first coordinates of the primary road surface feature points and the second coordinates of the secondary road surface feature points among the static feature points. The determining unit is further configured to determine the first lateral and longitudinal movement distance of the target vehicle based on the feature point position information; the determination of the first lateral and longitudinal movement distance of the target vehicle based on the feature point position information includes: determining matching constraints for primary road feature points and secondary road feature points, the matching constraints including topological invariance constraints and kinematic constraints; performing feature point matching on primary road feature points in two adjacent frames based on the matching constraints; performing feature point matching on secondary road feature points in two adjacent frames based on the matching constraints; taking the target second primary road feature point that matches the first primary road feature point in the current frame as a reference point, and taking the target second... Normalize the second-level road surface feature points other than the first-level road surface feature points; select multiple third-level road surface feature points from the multiple second-level road surface feature points based on the reference point; determine the confidence level of each third-level road surface feature point; determine the third coordinate change value between the first-level road surface feature point in the current frame and each third-level road surface feature point; determine the first coordinate change value based on the confidence level and the third coordinate change value; determine the second coordinate change value of the matched second-level road surface feature point based on the feature point position information; determine the first horizontal and vertical movement distance of the target vehicle based on the first coordinate change value and the second coordinate change value. The determining unit is further configured to determine the target speed of the target vehicle based on the first moving distance and the image acquisition time.

7. An electronic device, characterized in that, It includes a processor and a memory, the memory being used to store one or more programs and configured to be executed by the processor, the programs including instructions for performing the steps of the method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, A computer program for storing electronic data interchange is provided, wherein the computer program causes a computer to perform the method as described in any one of claims 1-5.

Images