Motion measurement method and device, terminal equipment and storage medium

Abstract

The invention discloses a motion measurement method and device, terminal equipment and a storage medium. The method comprises the following steps: acquiring a target image acquired by a current frame of a camera; identifying to obtain identification information of a candidate indication identifier in the target image; matching the identification information with a preset reference information database to obtain identification information of the candidate indication identification; based on the candidate indication identifier, performing distance projection on an obstacle in the target image to obtain position information and / or relative distance information of the obstacle in the current frame; and tracking the position and / or distance change of the obstacle in combination with the information of the obstacle in the historical frame to complete the measurement of the movement information of the obstacle. The method is applied to the field of automatic driving, an existing traffic indication sign is used as a reference object, an object motion measurement result can be prevented from being affected by shielding and a multipath effect without an additional sensor, the cost is reduced, the object motion measurement precision is improved, and the method has wide application prospects and economic benefits.

Description

Technical Field

[0001] The present invention relates to the fields of traffic management, intelligent traffic systems and autonomous driving technology, and in particular to a motion measurement method, device, terminal equipment and storage medium based on traffic indication signs. Background Art

[0002] Traffic management and traffic safety are crucial issues in modern society. In order to guide drivers, provide traffic information and keep roads open, various traffic signs are set up on the roads, such as traffic lights, signs and markings. These traffic signs play an important role in road traffic.

[0003] At the same time, accurately measuring the motion information of vehicles or objects is also of great significance to fields such as traffic management, intelligent transportation systems, and autonomous driving technology. Traditional motion measurement methods mainly rely on technologies such as vehicle sensors, global positioning systems (GPS), and inertial navigation systems (INS). However, these traditional methods have some problems. For example, there may be errors in sensor positioning, especially in complex road environments; GPS signals may be affected by occlusion and multipath effects, resulting in inaccurate position measurements; and these traditional methods are costly and are not suitable for all vehicles and application scenarios. Summary of the invention

[0004] In view of the deficiencies in the above-mentioned prior art, the present invention provides a motion measurement method, device, terminal equipment and storage medium based on traffic signs, which can accurately measure the motion information of objects on the road and has the characteristics of low cost and wide applicability, and solves the technical problems of inaccurate positioning, low precision and high cost in traditional motion measurement methods.

[0005] To achieve the above object, the present invention provides a motion measurement method, comprising the following steps:

[0006] Step 1, obtaining a target image captured by a current frame of a camera, wherein the target image has at least one candidate indicator mark to be identified;

[0007] Step 2, performing image processing on the target image to identify and obtain identification information of the candidate indicator in the target image;

[0008] Step 3, matching the identification information with a preset reference information database to obtain identification information of a candidate indicator;

[0009] Step 4, based on the candidate indicator having the identification information, performing distance projection on the obstacle in the target image to obtain the position information and / or relative distance information of the obstacle in the current frame;

[0010] Step 5: Track the position and / or distance change of the obstacle in combination with the position information and / or relative distance information of the obstacle in the historical frame, that is, complete the measurement of the motion information of the obstacle.

[0011] In one embodiment, in step 3, the reference information database contains reference information of the indicator, and the reference information includes but is not limited to the physical size, position coordinates and orientation angle of the indicator.

[0012] In one embodiment, step 4 includes:

[0013] Based on the internal and external parameters of the camera and the identification information, the candidate indicator is associated with the actual indicator in the world coordinate system, and the relative distance information of the actual indicator is calculated;

[0014] Based on the relative distance information of the actual indication mark, distance projection is performed on the obstacles in the target image, the obstacles in the target image are mapped into a world coordinate system, and the position information and / or relative distance information of the obstacles in the world coordinate system are calculated.

[0015] In one embodiment, performing distance projection on obstacles in the target image is specifically:

[0016] Perform distance projection on obstacles located near the lane line in the target image.

[0017] In one embodiment, in step 5, tracking the position and / or distance change of the obstacle in combination with the position information and / or relative distance information of the obstacle in the historical frame is specifically:

[0018] The position information and / or relative distance information of the obstacle in the historical frame and the current frame are combined, and a time observation window is used to track the position and / or distance changes of the obstacle, wherein the length of the time observation window has an adaptive adjustment function.

[0019] In one embodiment, in step 5, while tracking the position and / or distance change of the obstacle in combination with the position information and / or relative distance information of the obstacle in the historical frame, the speed information and / or direction information of the obstacle is also obtained.

[0020] In one embodiment, when there are more than one candidate indicator in the target image, the operations of step 2 to step 4 are performed for each candidate indicator to obtain the motion information of the obstacle under the reference of each candidate indicator.

[0021] Then, the motion information of the obstacle under the reference of each candidate indicator mark is weighted or the best one is selected based on the accuracy to obtain the optimal motion information of the obstacle.

[0022] To achieve the above object, the present invention further provides a motion measurement device, using the above method, the motion measurement device comprises:

[0023] An image acquisition unit, used to acquire a target image captured by a current frame of the camera, wherein the target image has at least one candidate indicator mark to be identified;

[0024] An image processing unit, used to perform image processing on the target image to identify and obtain identification information of the candidate indicator in the target image;

[0025] An identification matching unit, used to match the identification information with a preset reference information database to obtain identification information of a candidate indication identification;

[0026] A distance projection unit, configured to perform distance projection on the obstacle in the target image according to the candidate indication mark having the identification information, so as to obtain the position information and / or relative distance information of the obstacle in the current frame;

[0027] The motion measurement unit is used to track the position and / or distance change of the obstacle in combination with the position information and / or relative distance information of the obstacle in the historical frame, that is, to complete the motion information measurement of the obstacle.

[0028] To achieve the above object, the present invention further provides a terminal device, wherein the terminal device is provided with:

[0029] Memory, used to store programs;

[0030] The processor is used to execute the program stored in the memory. When the program is executed, the processor is used to execute part or all of the steps of the motion measurement method as described above.

[0031] To achieve the above objectives, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions; when the computer-executable instructions are executed by a processor, they are used to implement some or all of the steps of the motion measurement method as described above.

[0032] Compared with the prior art, the present invention has the following beneficial technical effects:

[0033] 1. The present invention collects target images through a camera, uses traffic signs as reference objects, and uses image recognition technology, combined with reference information, distance projection and multi-frame observation information, to achieve accurate measurement of object movement;

[0034] 2. The present invention utilizes existing traffic signs as reference objects in the process of measuring object motion, and can ensure that the object motion measurement results are not affected by occlusion and multipath effects without the need for additional sensors or equipment, thereby reducing costs and improving the measurement accuracy of object motion, and has broad application prospects and economic benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0036] Figure 1 is a flow chart of a motion measurement method according to an embodiment of the present invention;

[0037] Figure 2 is a structural block diagram of a motion measurement device in an embodiment of the present invention;

[0038] Figure 3 4 is a structural block diagram of a terminal device in an embodiment of the present invention.

[0039] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0042] In addition, in the present invention, descriptions such as "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0043] In the present invention, unless otherwise clearly specified and limited, the terms "connection", "fixation", etc. should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, a physical connection, or a wireless communication connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0044] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0045] Example 1

[0046] This embodiment discloses a motion measurement method, the core idea of ​​which is to use a vehicle-mounted camera to obtain image data on the road, and identify the information of candidate indicator signs in the image data through an image recognition method, compare it with the reference information such as the physical size, position and angle of the pre-stored indicator signs to obtain the identification information of the candidate indicator signs in the image data, and then combine the identification information with the internal and external parameters of the camera to calculate the relative distance information of the candidate indicator signs as the identification information. Next, based on the candidate indicator signs as a reference, the obstacles near the lane line are projected for distance to obtain a set of obstacle descriptions containing relative distance information. At the same time, by using multi-frame observation information and tracking the change of distance through the time observation window, the motion description of the obstacle can be obtained, thereby realizing the measurement of the object's motion.

[0047] refer to Figure 1 The motion measurement method in this embodiment specifically includes the following steps:

[0048] Step 1, obtaining a target image captured by the current frame of the camera, wherein the camera is a high-resolution vehicle-mounted camera capable of obtaining clear road image data, and the position and angle of the camera on the vehicle can cover key information such as traffic signs and lane lines on the road, so that the target image has at least one candidate sign to be identified, and the sign can be a traffic light, a signboard, a marking, etc.;

[0049] Step 2: using machine vision technology or deep learning technology to perform image processing operations such as target detection, feature extraction and / or image classification on the target image, and identifying the identification information of the candidate indicator in the target image, such as the aspect ratio, outline and angular orientation of the candidate indicator;

[0050] Step 3, matching the identification information with a preset reference information database to obtain identification information of the candidate indicator, wherein the identification information is the correspondence between the candidate indicator and the reference information, and the reference information database has reference information of several different indicator, and the reference information includes but is not limited to the physical size, position coordinates and orientation angle of the indicator;

[0051] Step 4, associating the candidate indicator with identification information with the object in the world coordinate system, and then performing distance projection on the obstacle in the target image based on the position of the candidate indicator with identification information in the world coordinate system to obtain the position information and / or relative distance information of the obstacle in the current frame;

[0052] Step 5: Track the position and / or distance change and / or speed information and / or direction information of the obstacle in combination with the position information and / or relative distance information of the obstacle in the historical frame, so as to obtain the motion description of the obstacle, that is, complete the motion information measurement of the obstacle, wherein the speed information of the obstacle includes the speed and acceleration of the obstacle.

[0053] In a specific application process, the reference information in the reference information database can be obtained through manual measurement or measurement using other measurement equipment such as laser radar, and the measured reference information is stored in the system for subsequent use.

[0054] In the specific implementation process, the process of performing distance projection on obstacles in the target image is as follows:

[0055] First, based on the pre-calibrated internal and external parameters and identification information of the camera, the candidate indicator is associated with the actual indicator in the world coordinate system, and the relative distance information of the actual indicator, that is, the distance between the actual indicator and the camera, is calculated;

[0056] Based on the relative distance information of the actual indication mark, the obstacle in the target image is projected, that is, the obstacle in the target image is mapped to the world coordinate system, and the position information and / or relative distance information of the obstacle in the world coordinate system are calculated, that is, the world coordinates of the obstacle and / or the distance relative to the camera are obtained.

[0057] As a preferred implementation, in the process of distance projection of obstacles in the target image, only obstacles located near the lane line in the target image are distance projected, thereby reducing the amount of calculation, wherein the range near the lane line is the width of the left and right lanes of the current vehicle driving lane.

[0058] In the specific implementation process of step 5, when tracking the position and / or distance changes of the obstacle in combination with the position information and / or relative distance information of the obstacle in the historical frame, the time observation window is used to track the position and / or distance changes of the obstacle, wherein preferably, the length of the time observation window has an adaptive adjustment function, that is, the system dynamically adjusts the length of the time observation window according to the actual situation. For example, for scenes where vehicles travel faster, such as highways, the time observation window can be set longer to better track the movement of obstacles; and in the case of slower driving speeds, such as urban roads, the time observation window can be set shorter to reduce the amount of calculation and delay. Such adaptive performance makes the system more flexible and efficient.

[0059] It is worth noting that in the specific application process, there may be a situation where there are multiple candidate indicator markers in the target image. In this regard, the present embodiment is preferred. When the number of candidate indicator markers in the target image is more than one, steps 2 to 4 are performed for each candidate indicator marker to obtain the motion information of the obstacle under each candidate indicator marker benchmark; the motion information of the obstacle under each candidate indicator marker benchmark is then weighted or the best is selected based on accuracy to obtain the optimal motion information of the obstacle.

[0060] Specifically, the process of weighting the motion information of the obstacle under each candidate indicator reference is as follows:

[0061] According to the distance from the obstacle to each candidate indicator benchmark, different operation weight coefficients are assigned from near to far, and the weight coefficient of the closer distance is larger. Finally, the motion information of each candidate indicator benchmark is merged as the operation information of the obstacle.

[0062] The process of selecting the best obstacle motion information under each candidate indicator reference based on accuracy is as follows:

[0063] Based on the running information of multiple frames, the length, width, height and image information of the obstacle in the motion information are judged to determine whether the obstacle continuously meets the characteristics, thereby reducing the occurrence of false alarms.

[0064] In addition, if there is no candidate indicator in the target image captured by the current frame, the obstacle is projected into the global coordinate system, and the motion information of the obstacle in multiple frames is tracked to determine whether it is a moving target.

[0065] As a preferred implementation, the motion measurement method in this embodiment can also be combined with the vehicle-mounted sensor data measured by the millimeter wave radar and / or the laser radar to further improve the measurement accuracy of the motion information of the obstacle. The specific implementation process is as follows:

[0066] The motion information of obstacles can be obtained through millimeter wave radar and / or lidar, the motion information obtained by the above sensors and the motion information detected by the image are fused, and the motion state of the obstacle is estimated more accurately through the filter estimation algorithm (EKF / UEK).

[0067] To sum up, the motion measurement method in this embodiment, by utilizing existing traffic signs as reference objects and through image recognition technology, combined with a combination of reference information, distance projection and multi-frame observation information, can make the object motion measurement results unaffected by occlusion and multipath effects without the need for additional sensors or equipment, thereby reducing costs while improving the measurement accuracy of object motion, and can bring broad application prospects and economic benefits in the fields of traffic management, intelligent transportation systems and autonomous driving.

[0068] Example 2

[0069] Based on the motion measurement method in Embodiment 1, this embodiment discloses a motion measurement device. Figure 2 The motion measurement device includes an image acquisition unit, an image processing unit, a marker matching unit, a distance projection unit and a motion measurement unit. The motion measurement device is used to perform some or all steps of the motion measurement method in Example 1, thereby realizing the motion information measurement of the obstacle. Specifically:

[0070] An image acquisition unit, used to acquire a target image captured by a current frame of the camera, wherein the target image has at least one candidate indicator mark to be identified;

[0071] An image processing unit, used to perform image processing on a target image to identify and obtain identification information of a candidate indicator in the target image;

[0072] An identification matching unit, used to match the identification information with a preset reference information database to obtain identification information of a candidate indication identification;

[0073] A distance projection unit, configured to perform distance projection on obstacles in the target image according to the candidate indicating mark having identification information, so as to obtain position information and / or relative distance information of the obstacles in the current frame;

[0074] The motion measurement unit is used to track the position and / or distance change of the obstacle in combination with the position information and / or relative distance information of the obstacle in the historical frame, that is, to complete the motion information measurement of the obstacle.

[0075] In this embodiment, the specific working process and working principle of the image acquisition unit, image processing unit, identification matching unit, distance projection unit and motion measurement unit are the same as those in Embodiment 1, so they are not described in detail in this embodiment.

[0076] Example 3

[0077] like Figure 3 The terminal device disclosed in this embodiment is shown, including a transmitter, a receiver, a memory and a processor. The transmitter is used to send instructions and data, the receiver is used to receive instructions and data, the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions stored in the memory to implement some or all of the steps performed by the motion measurement method in the above-mentioned embodiment 1. The specific implementation process is the same as the motion measurement method in the above-mentioned embodiment 1.

[0078] It should be noted that the above memory can be independent or integrated with the processor. When the memory is independently provided, the terminal device further includes a bus for connecting the memory and the processor.

[0079] In the specific application process, the terminal device is a computer, an unmanned vehicle, a drone, an unmanned driving device or a mobile robot, etc.

[0080] Example 4

[0081] This embodiment discloses a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, part or all of the steps executed by the motion measurement method in the above-mentioned embodiment 1 are implemented.

[0082] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the inventive concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A motion measurement method, characterized in that: The steps include: Step 1, obtaining a target image captured by a current frame of a camera, wherein the target image has at least one candidate indicator mark to be identified; Step 2, performing image processing on the target image to identify and obtain identification information of the candidate indicator in the target image; Step 3, matching the identification information with a preset reference information database to obtain identification information of a candidate indicator; Step 4, based on the candidate indicator having the identification information, performing distance projection on the obstacle in the target image to obtain the position information and / or relative distance information of the obstacle in the current frame; Step 5: Track the position and / or distance change of the obstacle in combination with the position information and / or relative distance information of the obstacle in the historical frame, that is, complete the measurement of the motion information of the obstacle.

2. The motion measurement method according to claim 1, characterized in that: In step 3, the reference information database contains reference information of the indicator, and the reference information includes but is not limited to the physical size, position coordinates and orientation angle of the indicator.

3. The motion measurement method according to claim 1, characterized in that: Step 4 includes: Based on the internal and external parameters of the camera and the identification information, the candidate indicator is associated with the actual indicator in the world coordinate system, and the relative distance information of the actual indicator is calculated; Based on the relative distance information of the actual indication mark, distance projection is performed on the obstacles in the target image, the obstacles in the target image are mapped into a world coordinate system, and the position information and / or relative distance information of the obstacles in the world coordinate system are calculated.

4. The motion measurement method according to claim 3, characterized in that: Perform distance projection on the obstacles in the target image, specifically: Perform distance projection on obstacles located near the lane line in the target image.

5. The motion measurement method according to any one of claims 1 to 4, characterized in that: In step 5, the position and / or distance change of the obstacle is tracked in combination with the position information and / or relative distance information of the obstacle in the historical frame, specifically: The position information and / or relative distance information of the obstacle in the historical frame and the current frame are combined, and a time observation window is used to track the position and / or distance changes of the obstacle, wherein the length of the time observation window has an adaptive adjustment function.

6. The motion measurement method according to any one of claims 1 to 4, characterized in that: In step 5, while tracking the position and / or distance change of the obstacle in combination with the position information and / or relative distance information of the obstacle in the historical frame, the speed information and / or direction information of the obstacle is also obtained.

7. The motion measurement method according to any one of claims 1 to 4, characterized in that: When there is more than one candidate indicator in the target image, the operations of step 2 to step 4 are performed for each candidate indicator to obtain the motion information of the obstacle under the reference of each candidate indicator; Then, the motion information of the obstacle under the reference of each candidate indicator mark is weighted or the best one is selected based on the accuracy to obtain the optimal motion information of the obstacle.

8. A motion measurement device, characterized in that: According to any one of claims 1 to 7, the motion measurement device comprises: An image acquisition unit, used to acquire a target image captured by a current frame of the camera, wherein the target image has at least one candidate indicator mark to be identified; An image processing unit, used to perform image processing on the target image to identify and obtain identification information of the candidate indicator in the target image; An identification matching unit, used to match the identification information with a preset reference information database to obtain identification information of a candidate indication identification; A distance projection unit, configured to perform distance projection on the obstacle in the target image according to the candidate indication mark having the identification information, so as to obtain the position information and / or relative distance information of the obstacle in the current frame; The motion measurement unit is used to track the position and / or distance change of the obstacle in combination with the position information and / or relative distance information of the obstacle in the historical frame, that is, to complete the motion information measurement of the obstacle.

9. A terminal device, characterized in that: The terminal device is provided with: Memory, used to store programs; A processor is used to execute the program stored in the memory. When the program is executed, the processor is used to execute part or all of the steps of the motion measurement method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions; when the computer-executable instructions are executed by a processor, they are used to implement part or all of the steps of the motion measurement method according to any one of claims 1 to 7.

Images