Motor vehicle fixed-point target ranging method, device and computer-readable storage medium

Abstract

An embodiment of the present invention provides a method, device and computer-readable storage medium for measuring the distance of a fixed-point target of a motor vehicle. The method includes: obtaining real-time image frames from the original video images captured by an in-vehicle camera; using a pre-stored target detection model to detect obstacles around the motor vehicle from each frame of the image frames, and selecting the fixed-point targets to be measured in two adjacent frames of the image frames; respectively calculating the photosensitive surface coordinates of the same position points in the target frames of two adjacent frames of the image frames on the photosensitive surface of the in-vehicle camera; deriving the lens imaging coordinates of the position points corresponding to the two adjacent frames of the image frames on the lens of the in-vehicle camera based on the camera imaging principle and the geometric relationship derivation method; calculating and outputting the current actual distance of the fixed-point target to be measured relative to the motor vehicle in combination with the change amount P of the wheel speed pulses within the frame difference time of the original video images. This embodiment can effectively reduce the detection cost and improve the detection efficiency.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of motor vehicle assisted driving, and in particular to a method, device and computer-readable storage medium for measuring the distance to a fixed-point target of a motor vehicle. Background Art

[0002] At present, motor vehicles are usually equipped with a vision ranging device connected to an in-vehicle camera for detecting the distance to obstacles around the motor vehicle based on the original video images provided by the in-vehicle camera. To accurately calculate the distance between the obstacle and the motor vehicle, the existing vision ranging methods usually first obtain the video images of the surrounding environment of the motor vehicle through the in-vehicle camera, then construct a three-dimensional scene map of the current environment of the motor vehicle through the video images, and finally calculate and determine the relative distance between the obstacle and the motor vehicle through the three-dimensional map. However, the inventor found during specific implementation that when constructing a three-dimensional scene map using the traditional method, it is necessary to calculate the image depth, which requires a relatively high hardware computing power cost, the process is complex, and ultimately the overall ranging efficiency is also relatively low. Summary of the Invention

[0003] The technical problem to be solved by the embodiments of the present invention is to provide a method for measuring the distance to a fixed-point target of a motor vehicle, which can effectively reduce the detection cost and improve the detection efficiency.

[0004] The technical problem to be further solved by the embodiments of the present invention is to provide a device for measuring the distance to a fixed-point target of a motor vehicle, which can effectively reduce the detection cost and improve the detection efficiency.

[0005] The technical problem to be further solved by the embodiments of the present invention is to provide a computer-readable storage medium for storing a computer program that can effectively reduce the detection cost and improve the detection efficiency.

[0006] To solve the above technical problems, the embodiments of the present invention first provide the following technical solution: A method for measuring the distance to a fixed-point target of a motor vehicle, comprising the following steps:

[0007] Obtain real-time image frames from the original video images captured by the in-vehicle camera;

[0008] Use a pre-stored target detection model to detect obstacles around the motor vehicle frame by frame starting from the first frame, mark the obstacles that exist in both adjacent image frames and are stationary themselves as the fixed-point targets to be measured, and use a target box of a predetermined standard size to frame the fixed-point targets to be measured in the adjacent two image frames;

[0009] Calculate the photosensitive surface coordinates corresponding to the same position points within the target boxes of the adjacent two image frames on the photosensitive surface of the in-vehicle camera respectively;

[0010] Based on the coordinates of the photosensitive surface, the lens imaging coordinates corresponding to the position points of the adjacent two image frames on the lens of the vehicle-mounted camera are derived by the method of derivation based on the camera imaging principle and geometric relationship; and

[0011] Combined with the change amount P of the wheel speed pulse within the frame difference time of the original video image and the lens imaging coordinates, calculate and output the current actual distance of the to-be-determined point target relative to the motor vehicle.

[0012] Furthermore, the specific steps of combining the change amount P of the wheel speed pulse within the frame difference time of the original video image and the lens imaging coordinates to calculate and output the current actual distance of the to-be-determined point target relative to the motor vehicle include:

[0013] Calculate the lens imaging coordinates corresponding to multiple pairs of different position points within the target box of the adjacent two image frames;

[0014] Combined with the change amount P of the wheel speed pulse within the frame difference time of the original video image and each pair of the lens imaging coordinates, respectively calculate multiple current estimated distances of the to-be-determined point target relative to the motor vehicle; and

[0015] Take the weighted average value of the current estimated distances corresponding to each pair of lens imaging coordinates as the current actual distance for output.

[0016] Furthermore, the current estimated distance where P is the change amount of the wheel speed pulse of the motor vehicle within the frame difference time of the original video image, f is the focal length of the vehicle-mounted camera, and X2 and X1 are the abscissa values of a pair of corresponding lens imaging coordinates in the adjacent two image frames, respectively.

[0017] Furthermore, the position point is the coordinate point on the bottom edge of the target box.

[0018] Furthermore, the target detection model is an SSD algorithm model based on deep learning.

[0019] Furthermore, obtaining real-time image frames from the original video image captured by the vehicle-mounted camera specifically refers to obtaining real-time image frames from the original video image captured by the vehicle-mounted monocular camera.

[0020] On the other hand, to solve the above further technical problems, the embodiments of the present invention further provide the following technical solution: A vehicle fixed-point target ranging device is respectively connected to a vehicle-mounted camera for shooting video images of the surrounding environment of the vehicle and providing original video images, and an information display device for displaying the current actual distance output by the vehicle fixed-point target ranging device. The vehicle fixed-point target ranging device includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the vehicle fixed-point target ranging method as described in any one of the above.

[0021] On yet another aspect, to solve the above further technical problems, the embodiments of the present invention further provide the following technical solution: A computer-readable storage medium includes a stored computer program. When the computer program runs, it controls the device where the computer-readable storage medium is located to execute the vehicle fixed-point target ranging method as described in any one of the above.

[0022] After adopting the above technical solutions, the embodiments of the present invention have at least the following beneficial effects: After obtaining real-time image frames from the original video images captured by the vehicle-mounted camera, and detecting obstacles around the vehicle from each frame of the image frames, the obstacles that exist in both adjacent frames of the image frames and are stationary by themselves are marked as the to-be-determined fixed-point targets, and the to-be-determined fixed-point targets in the adjacent two frames of the image frames are framed by a target frame of a predetermined standard size. Then, the photosensitive surface coordinates corresponding to the same position points in the target frames of the adjacent two frames of the image frames on the photosensitive surface of the vehicle-mounted camera are calculated. Further, the lens imaging coordinates of the imaging on the lens of the vehicle-mounted camera are deduced based on the photosensitive surface coordinates. The calculation process is relatively simpler. Finally, by combining the change amount P of the wheel speed pulses within the frame difference time of the original video images and the lens imaging coordinates, the current actual distance of the to-be-determined fixed-point target relative to the vehicle can be calculated and output. There is no need for complex pixel point matching, and the vehicle-mounted camera can achieve ranging without using a relatively expensive binocular camera compared with the traditional technology, with lower costs. Description of the Drawings

[0023] Figure 1 It is a flowchart of the steps of an optional embodiment of the vehicle fixed-point target ranging method of the present invention.

[0024] Figure 2 It is a specific flowchart of step S5 of an optional embodiment of the vehicle fixed-point target ranging method of the present invention.

[0025] Figure 3 It is a schematic diagram of the imaging principle of the to-be-determined fixed-point target of an optional embodiment of the vehicle fixed-point target ranging method of the present invention.

[0026] Figure 4 This is a schematic block diagram of an alternative embodiment of the fixed-point target ranging device for a motor vehicle according to the present invention.

[0027] Figure 5 This is a functional module diagram of an alternative embodiment of the fixed-point target ranging device for a motor vehicle according to the present invention. Detailed implementation manners

[0028] The following further describes the present application in detail with reference to the accompanying drawings and specific embodiments. It should be understood that the following illustrative embodiments and descriptions are only used to explain the present invention and are not intended to limit the present invention. Moreover, in the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0029] As Figure 1 shown, an alternative embodiment of the present invention provides a method for ranging a fixed-point target of a motor vehicle, including the following steps:

[0030] S1: Obtain real-time image frames from the original video images captured by the in-vehicle camera 1;

[0031] S2: Detect obstacles around the motor vehicle from each frame of the image frames by using a pre-stored target detection model, mark the obstacles that exist in both adjacent frames of the image frames and are fixed by themselves as the to-be-determined fixed-point targets, and respectively frame out the to-be-determined fixed-point targets from the adjacent frames of the image frames by using target frames of the same size;

[0032] S3: Calculate the photosensitive surface coordinates corresponding to the same position points within the target frames of the adjacent frames of the image frames on the photosensitive surface of the in-vehicle camera 1 respectively;

[0033] S4: Derive the lens imaging coordinates of the imaging of the position points of the adjacent frames on the lens of the in-vehicle camera 1 based on the camera imaging principle and the geometric relationship derivation method according to the photosensitive surface coordinates; and

[0034] S5: Calculate and output the current actual distance of the to-be-determined fixed-point target relative to the motor vehicle in combination with the change amount P of the wheel speed pulses within the frame difference time of the original video images and the lens imaging coordinates.

[0035] In an embodiment of the present invention, after obtaining real-time image frames from the original video images captured by the vehicle-mounted camera 1, detecting obstacles around the motor vehicle from each frame of the image frames, marking the obstacles that exist in both adjacent frames of the image frames and are stationary by themselves as the to-be-determined fixed-point targets, using a target frame of a predetermined standard size to frame the to-be-determined fixed-point targets in the two adjacent frames of the image frames, then calculating the photosensitive surface coordinates of the same position points in the target frames of the two adjacent frames of the image frames corresponding to the photosensitive surface of the vehicle-mounted camera, and further deriving the lens imaging coordinates of the imaging on the lens of the vehicle-mounted camera 1 according to the photosensitive surface coordinates. The calculation process is relatively simpler. Finally, by combining the change amount P of the wheel speed pulses within the frame difference time of the original video image and the lens imaging coordinates, the current actual distance of the to-be-determined fixed-point target relative to the motor vehicle can be calculated and output. There is no need for complex pixel point matching, and the vehicle-mounted camera 1 can achieve ranging without using a relatively expensive binocular camera compared with the traditional technology, with lower cost.

[0036] In an alternative embodiment of the present invention, as Figure 2 shown, step S5 specifically includes:

[0037] S51: Calculate the lens imaging coordinates corresponding to multiple pairs of different position points within the target frames of the two adjacent frames of the image frames;

[0038] S52: Combine the change amount P of the wheel speed pulses within the frame difference time of the original video image and each pair of the lens imaging coordinates to calculate multiple current estimated distances of the to-be-determined fixed-point target relative to the motor vehicle; and

[0039] S53: Output the weighted average value of the current estimated distances corresponding to each pair of the lens imaging coordinates as the current actual distance.

[0040] In this embodiment, by calculating the lens imaging coordinates corresponding to multiple pairs of different position points, thus calculating a current estimated distance according to each pair of the lens imaging coordinates, and finally, by obtaining the weighted average value of each current estimated distance to determine the current actual distance, the system error can be effectively reduced and the accuracy of the distance calculation can be improved. In specific implementation, the specific number of the lens imaging coordinates can be determined by the shooting frame rate of the specific vehicle-mounted camera 1, the driving speed of the motor vehicle, etc.

[0041] In an alternative embodiment of the present invention, as Figure 3 shown, the current estimated distance wherein, P is the change amount of the wheel speed pulses of the motor vehicle within the frame difference time of the original video image, f is the focal length of the vehicle-mounted camera 1, and X2 and X1 are respectively the abscissa values of a pair of corresponding lens imaging coordinates in the two adjacent frames of the image frames. In this embodiment, as Figure 3As shown in the figure, since the ordinate of the object's image on the lens of the vehicle-mounted camera 1 is always constant, point A represents the vertex target point to be measured. From the photosensitive surface coordinates O1(u1, v1) and O2(u2, v2), a pair of corresponding lens imaging coordinates B(X1, Y1) and C(X2, Y1) in the adjacent two image frames can be calculated. The displacement of the motor vehicle within the frame difference time is represented by the change in the wheel speed pulse within the frame difference time. Using Figure 3 the similarity of triangle ABC and triangle AO1O2 in Figure 3 , the following formula can be obtained:

[0042] Z / (Z - f) = P / |X2 - X1| (Formula 1)

[0043] From the above, the expression of the current estimated distance Z can be derived.

[0044] In an alternative embodiment of the present invention, the position point is the coordinate point on the bottom edge of the target box. In this embodiment, the coordinate point on the bottom edge of the target box is used as the position point for coordinate calculation, which is convenient for realizing the extraction and operation of coordinates. In specific implementation, the OPENCV function is used to extract the photosensitive surface coordinates.

[0045] In an alternative embodiment of the present invention, the target detection model is an SSD algorithm model based on deep learning. In this embodiment, the SSD algorithm model based on deep learning is adopted. Through a large amount of pre-learning and training, in the actual application process, the target box of the to-be-determined point target can be detected quickly and accurately.

[0046] In an alternative embodiment of the present invention, obtaining the real-time image frame from the original video image captured by the vehicle-mounted camera 1 specifically refers to obtaining the real-time image frame from the original video image captured by the vehicle-mounted monocular camera. In this embodiment, the vehicle-mounted camera 1 adopts a monocular camera, which has a relatively lower cost and can effectively realize the distance detection of the fixed-point target.

[0047] On the other hand, as Figure 4 shown, the embodiment of the present invention further provides a vehicle fixed-point target ranging device 3, which is respectively connected to a vehicle-mounted camera 1 for capturing the video image of the vehicle's surrounding environment and providing the original video image, and an information display device 5 for displaying the current actual distance output by the vehicle fixed-point target ranging device 3. The vehicle fixed-point target ranging device 3 includes a processor 30, a memory 32, and a computer program stored in the memory 32 and configured to be executed by the processor. When the processor 30 executes the computer program, it implements the vehicle fixed-point target ranging method described in any one of the above.

[0048] Exemplarily, the computer program may be divided into one or more modules / units, and the one or more modules / units are stored in the memory 42 and executed by the processor to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program in the reticle repair control device. For example, the computer program may be divided into Figure 5 function modules in the motor vehicle fixed-point target ranging device 3 described above. Among them, the image frame extraction module 41, the target box extraction module 42, the photosensitive surface coordinate calculation module 43, the lens imaging coordinate derivation module 44, and the distance calculation module 45 respectively execute the above steps S1 - step S5.

[0049] The motor vehicle fixed-point target ranging device 3 may be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The motor vehicle fixed-point target ranging device 3 may include, but is not limited to, a processor 30 and a memory 32. Those skilled in the art can understand that the schematic diagram is only an example of the motor vehicle fixed-point target ranging device 3, and does not constitute a limitation on the motor vehicle fixed-point target ranging device 3. It may include more or fewer components than shown, or combine certain components, or different components. For example: The motor vehicle fixed-point target ranging device 3 may further include input / output devices, network access devices, a bus, etc.

[0050] The so-called processor 30 may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The processor 30 is the control center of the motor vehicle fixed-point target ranging device 3, and connects various parts of the entire motor vehicle fixed-point target ranging device 3 through various interfaces and lines.

[0051] The memory 32 can be used to store the computer program and / or module. By running or executing the computer program and / or module stored in the memory 32, and invoking the data stored in the memory 32, the processor 30 realizes various functions of the vehicle fixed-point target ranging device 3. The memory 32 mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a graphic recognition function, a graphic stacking function, etc.); the data storage area can store data created according to the use of the ranging device (such as graphic data, etc.). In addition, the memory 32 can include high-speed random access memory, and can also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0052] If the functions described in the embodiments of the present invention are implemented in the form of software function modules or units and sold or used as independent products, they can be stored in a storage medium readable by a computing device. Based on such an understanding, to implement all or part of the processes in the above-mentioned method embodiments, the present invention embodiments can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor 30, the steps of the above-mentioned method embodiments can be realized. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disc, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0053] On the other hand, the embodiments of the present invention further provide a computer-readable storage medium, which includes a stored computer program. When the computer program runs, it controls the device where the computer-readable storage medium is located to execute the vehicle fixed-point target ranging method described in any one of the above.

[0054] In the present specification, the various embodiments are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the various embodiments, reference may be made to each other.

[0055] The embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, the present invention is not limited to the above specific embodiments. The above specific embodiments are merely illustrative rather than restrictive. Under the inspiration of the present invention, those of ordinary skill in the art can also make many forms without departing from the spirit of the present invention and the scope protected by the claims. All of these fall within the protection scope of the present invention.

Claims

1. A method for measuring the distance to a fixed point target of a motor vehicle, characterized in that, The method includes the following steps: Obtain real-time image frames from the original video images captured by the in-vehicle monocular camera; Use a pre-stored target detection model to detect obstacles around the motor vehicle frame by frame starting from the first frame in the image frames, and mark the obstacles that exist in both adjacent image frames and are stationary by themselves as the to-be-determined fixed-point targets, and use a target box of a predetermined standard size to frame the to-be-determined fixed-point targets in the adjacent two image frames; Calculate the photosensitive surface coordinates corresponding to the same position points within the target boxes of the adjacent two image frames on the photosensitive surface of the in-vehicle monocular camera respectively; Based on the photosensitive surface coordinates, deduce the lens imaging coordinates of the position points corresponding to the adjacent two image frames when imaging on the lens of the in-vehicle monocular camera by using the camera imaging principle and the geometric relationship derivation method; Calculate the lens imaging coordinates corresponding to multiple pairs of different position points within the target boxes of the adjacent two image frames; Combined with the change amount P of the wheel speed pulse within the frame difference time of the original video image, and corresponding to each pair of the imaging coordinates of the lenses, a plurality of current estimated distances Z of the to-be-determined point target relative to the motor vehicle are calculated, wherein the current estimated distance , where P is the change amount of the wheel speed pulse of the motor vehicle within the frame difference time of the original video image, f is the focal length of the vehicle-mounted monocular camera, and are respectively the abscissa values of a pair of corresponding lens imaging coordinates in two adjacent image frames; and Output the weighted average value of the current estimated distance Z corresponding to each pair of lens imaging coordinates as the current actual distance of the to-be-determined fixed-point target relative to the motor vehicle.

2. The motor vehicle fixed-point target ranging method according to claim 1, characterized in that, The position point is the coordinate point on the bottom edge of the target box.

3. The motor vehicle fixed-point target ranging method according to claim 1, characterized in that, The target detection model is an SSD algorithm model based on deep learning.

4. A vehicle fixed-point target ranging device is connected to a vehicle-mounted monocular camera for shooting video images of the surrounding environment of the vehicle and providing original video images, and an information display device for displaying the current actual distance output by the vehicle fixed-point target ranging device. It is characterized in that, The motor vehicle fixed-point target ranging device includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the motor vehicle fixed-point target ranging method described in any one of claims 1 to 3.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein when the computer program runs, it controls the device where the computer-readable storage medium is located to execute the motor vehicle fixed-point target ranging method described in any one of claims 1 to 3.

Images