Marker-assisted visual positioning method for assisting cooperative management and control of smart tunnel

Abstract

The invention discloses a marker-assisted visual positioning method and system for assisting cooperative management and control of a smart tunnel. The method comprises the following steps: acquiring an image; measuring a marker in the image and calculating a coordinate system transformation matrix; and based on the coordinate system transformation matrix, carrying out spatial positioning on the target vehicle in the image. The system comprises an image acquisition module, a matrix calculation module and a positioning module. According to the invention, space positioning can be realized at a low cost in a scene with poor signals. The method can be widely applied to the field of space positioning.

Description

Technical Field

[0001] The present invention relates to the field of spatial positioning, and in particular to a marker-assisted visual positioning method for assisting collaborative management and control of intelligent tunnels. Background Art

[0002] Due to its unique environment and structure, high-speed tunnels have high safety risks. Therefore, it is necessary to strengthen traffic supervision and achieve real-time tracking of vehicles at all times as much as possible, so as to deal with and guide vehicles in time and avoid secondary disasters. At present, vehicle tracking mainly relies on satellite and other positioning systems, but there is a widespread phenomenon of poor signals in mountain tunnels. The only way to obtain accident information in the tunnel is to rely on monitoring personnel to view monitoring videos, on-site patrols, and emergency phone calls. It is difficult to monitor in real time, which brings huge safety hazards to the operation of the road network. Therefore, it is of great significance to achieve spatial positioning and tracking of vehicles based on the existing monitoring system.

[0003] Existing vehicle tracking mainly relies on satellite and other positioning systems, which cannot be applied to mountain tunnels where poor signals are common. The positioning method of radar combined with surveillance cameras not only has high requirements for data quality, but also has high complexity in modal alignment and fusion, and requires high costs. Summary of the invention

[0004] In view of this, in order to solve the technical problems that the existing fusion positioning method used in tunnels requires high-quality data and high costs, the present invention proposes a marker-assisted visual positioning method for assisting the collaborative management of smart tunnels, the method comprising the following steps:

[0005] Acquire a surveillance image including a marker and a target vehicle;

[0006] The marker is measured based on the world coordinate system, and the coordinate system transformation matrix is ​​calculated based on the corresponding pixel coordinates of the marker in the monitoring image;

[0007] Based on the coordinate system transformation matrix, the target vehicle in the image is spatially positioned.

[0008] In some embodiments, the step of measuring the marker in the world coordinate system and calculating the coordinate system conversion matrix based on the corresponding pixel coordinates of the marker in the monitoring image specifically includes:

[0009] Select a world coordinate system and measure the coordinates of the corner points of the marker in the world coordinate system;

[0010] Use visual algorithms to segment the markers in the surveillance image and obtain the coordinates of their corner points in the pixel coordinate system;

[0011] The mapping matrix between the selected world coordinate system and the pixel coordinate system is calculated using the least square method;

[0012] In some embodiments, the step of spatially locating the target vehicle in the image based on the coordinate system conversion matrix specifically includes:

[0013] Identify the target vehicle in the surveillance image and obtain the coordinates of the midpoint of the lower boundary of its bounding box;

[0014] Track the target vehicle based on the Kalman filter algorithm and obtain the coordinates of the target vehicle in the pixel coordinate system in real time;

[0015] Using the calculated coordinate system conversion matrix, the coordinates of the midpoint of the lower boundary of the bounding box are converted from the pixel coordinate system to the world coordinate system to obtain the position of the target vehicle.

[0016] The present invention also proposes a marker-assisted visual positioning system for assisting the coordinated management and control of intelligent tunnels, the system comprising:

[0017] An image acquisition module, used for acquiring images;

[0018] A matrix calculation module, used to measure the markers in the image and calculate the coordinate system transformation matrix;

[0019] A positioning module is used to spatially locate the target vehicle in the image based on the coordinate system conversion matrix.

[0020] Based on the above scheme, the present invention provides a marker-assisted visual positioning method and system for assisting the coordinated management and control of intelligent tunnels. With the help of the monitoring system established in the tunnel, there is no need to upgrade and expand the hardware on a large scale, which is convenient for promotion at the application level. Compared with tracking methods such as satellite positioning, this aspect is more suitable for mountain tunnels where poor signals are common; compared with the positioning method of radar combined with surveillance cameras, the present invention greatly reduces the system complexity and economic cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flowchart of the steps of a marker-assisted visual positioning method for assisting the collaborative management and control of smart tunnels according to the present invention;

[0022] Figure 2 It is a structural block diagram of a marker-assisted visual positioning system for assisting collaborative management and control of intelligent tunnels according to the present invention. DETAILED DESCRIPTION

[0023] In addition to the problems of the existing solutions mentioned in the background technology, the proposed tracking method directly using surveillance cameras is difficult to provide accurate spatial positioning due to the lack of three-dimensional spatial information, which restricts the application of tracking results in motion pattern analysis, anomaly detection and positioning, etc.

[0024] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0025] It should be noted that, for the convenience of description, only the parts related to the relevant invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other.

[0026] It should be understood that the "system", "device", "unit" and / or "module" used in this application is a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the word can be replaced by other expressions.

[0027] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "a kind" and / or "the" do not refer to the singular, but also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of clearly identified steps and elements, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements. The elements defined by the sentence "includes a..." do not exclude the existence of other identical elements in the process, method, commodity or device that includes the elements.

[0028] In the description of the embodiments of the present application, "plurality" means two or more than two. The following terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.

[0029] In addition, flow charts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, the various steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0030] Reference Figure 1 , which is a flow chart of an optional example of a marker-assisted visual positioning method for assisting the collaborative management of intelligent tunnels proposed in the present invention. The method can be applied to computer devices. The visual positioning method proposed in this embodiment may include but is not limited to the following steps:

[0031] Step S1, acquiring an image;

[0032] Step S2, measuring the marker and calculating the coordinate system transformation matrix in combination with the image;

[0033] Step S3: spatially locate the target vehicle in the image based on the coordinate system conversion matrix.

[0034] In some feasible embodiments, the step S1 specifically includes:

[0035] Use Opencv to loop and read each frame of the surveillance camera video stream to obtain a surveillance image containing markers and vehicle targets.

[0036] In some feasible embodiments, the step S2 specifically includes:

[0037] S2.1. Select the world coordinate system and measure the coordinates of the corner points of the marker in the world coordinate system;

[0038] S2.2, segment the marker using a visual algorithm and obtain the coordinates of its corner points in a pixel coordinate system;

[0039] S2.3, calculating the mapping matrix between the selected world coordinate system and the pixel coordinate system using the least square method;

[0040] In some feasible embodiments, the step S3 specifically includes:

[0041] S3.1. Use the YOLOv8 fine-tuned model to identify the vehicle target in the surveillance image and obtain the coordinates of the midpoint of the lower boundary of its bounding box;

[0042] The network structure of the YOLOv8 fine-tuning model includes: Backbone network, Head network and Neck network. The Backbone network uses a series of convolution and deconvolution layers to extract features from the input image, and also uses residual connections and bottleneck structures to reduce the size of the network and improve performance; the Neck network is connected between the backbone network and the head network, and uses multi-scale feature fusion technology for feature fusion to enhance feature representation; the Head network is responsible for the final target detection and classification tasks, and uses a series of convolution layers and deconvolution layers to generate detection results.

[0043] S3.2, use Kalman filter algorithm for target tracking, and obtain the coordinates of the target vehicle in the pixel coordinate system in real time;

[0044] S3.3. Using the coordinate system conversion matrix calculated in real time / timing by the above process, the coordinates of the midpoint of the lower boundary of the bounding box are converted from the pixel coordinate system to the world coordinate system.

[0045] In some feasible embodiments, the S2.1 specifically includes:

[0046] S2.1.1. Select the projection point of the surveillance camera on the ground as the origin to construct a three-dimensional world coordinate system, where the vertical direction between lane lines is the X-axis direction, the vehicle travel direction is the Y-axis direction, and the vertical upward direction of the ground is the Z-axis direction;

[0047] S2.1.2. Actually measure the coordinates of the corner points of the auxiliary object in the world coordinate system.

[0048] In some feasible embodiments, the step S2.2 specifically includes:

[0049] S2.2.1. Find edge information in the image through edge detection, generate rectangular areas that may contain markers as candidates, decode each candidate area, identify potential markers, and confirm their authenticity and validity through geometric verification to obtain the corner coordinates of the auxiliary object in the pixel coordinate system;

[0050] S2.2.2. The coordinates of the corner points of the known markers in the world coordinate system and the pixel coordinate system correspond to each other, and the coordinate system mapping matrix is ​​calculated using the least squares method.

[0051] like Figure 2 As shown, a marker-assisted visual positioning system for assisting the coordinated management and control of intelligent tunnels includes:

[0052] An image acquisition module, used for acquiring images;

[0053] A matrix calculation module, used to measure the markers in the image and calculate the coordinate system transformation matrix;

[0054] A positioning module is used to spatially locate the target vehicle in the image based on the coordinate system conversion matrix.

[0055] The contents of the above method embodiments are all applicable to the present system embodiments. The functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0056] In some feasible embodiments, the matrix calculation module further includes:

[0057] The first coordinate submodule selects a world coordinate system and measures the coordinates of the corner points of the marker in the world coordinate system;

[0058] The second coordinate submodule uses a visual algorithm to segment the marker and obtain the coordinates of its corner points in the pixel coordinate system;

[0059] A mapping submodule calculates the mapping matrix between the selected world coordinate system and the pixel coordinate system using the least square method;

[0060] In some feasible embodiments, the positioning module further includes:

[0061] The recognition submodule uses the YOLOV8 fine-tuned model to identify vehicle targets in the surveillance image and obtain the coordinates of the midpoint of the lower boundary of its bounding box;

[0062] The target tracking submodule uses the Kalman filter algorithm for target tracking and obtains the coordinates of the target vehicle in the pixel coordinate system in real time;

[0063] The coordinate system conversion submodule uses the coordinate system conversion matrix calculated by the above process to convert the coordinates of the midpoint of the lower boundary of the bounding box from the pixel coordinate system to the world coordinate system.

[0064] A marker-assisted visual positioning device for assisting collaborative management of smart tunnels:

[0065] at least one processor;

[0066] at least one memory for storing at least one program;

[0067] When the at least one program is executed by the at least one processor, the at least one processor implements the marker-assisted visual positioning method for assisting the collaborative management and control of smart tunnels as described above.

[0068] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0069] A storage medium storing processor-executable instructions, wherein the processor-executable instructions are used to implement a marker-assisted visual positioning method for assisting the collaborative management and control of a smart tunnel as described above when executed by the processor.

[0070] The contents of the above method embodiments are all applicable to the present storage medium embodiments. The functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0071] The above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A marker-assisted visual positioning method for assisting collaborative management of smart tunnels, characterized in that: The following steps are involved: Get the image; measuring the markers and calculating a coordinate system transformation matrix in combination with the image; Based on the coordinate system transformation matrix, the target vehicle in the image is spatially positioned.

2. According to claim 1, a marker-assisted visual positioning method for assisting the coordinated management and control of intelligent tunnels is characterized in that: Also includes: The images are screened to retain images containing markers and target vehicles.

3. According to claim 1, a marker-assisted visual positioning method for assisting the coordinated management and control of intelligent tunnels is characterized in that: The step of measuring the marker and calculating the coordinate system conversion matrix in combination with the image specifically includes: Selecting a world coordinate system, and measuring the coordinates of the corner points of the marker in the image in the world coordinate system; Obtaining the coordinates of the corner points of the marker in a pixel coordinate system; The mapping matrix between the world coordinate system and the pixel coordinate system is calculated using the least squares method to obtain the coordinate system transformation matrix.

4. According to claim 1, a marker-assisted visual positioning method for assisting the coordinated management and control of intelligent tunnels is characterized in that: The step of spatially locating the target vehicle in the image based on the coordinate system conversion matrix specifically includes: Identify the target vehicle in the image and obtain the coordinates of the midpoint of the lower boundary of its bounding box; The conversion matrix is ​​used to convert the coordinates of the midpoint of the lower boundary of the bounding box from the pixel coordinate system to the world coordinate system to obtain the target vehicle position.

5. According to claim 4, a marker-assisted visual positioning method for assisting the coordinated management and control of intelligent tunnels is characterized in that: Also includes: The target vehicle is tracked based on the Kalman filter algorithm, and the coordinates of the target vehicle in the pixel coordinate system are obtained in real time.

6. According to claim 3, a marker-assisted visual positioning method for assisting the coordinated management and control of intelligent tunnels is characterized in that: The step of selecting a world coordinate system and measuring the coordinates of the corner points of the marker in the image in the world coordinate system specifically includes: A three-dimensional world coordinate system is constructed with the projection point of the camera on the ground as the origin, where the vertical direction between lane lines is the X-axis direction, the vehicle travel direction is the Y-axis direction, and the vertical upward direction of the ground is the Z-axis direction; Measure the coordinates of the corner points of the marker in the world coordinate system.

7. According to claim 3, a marker-assisted visual positioning method for assisting the coordinated management and control of intelligent tunnels is characterized in that: The step of obtaining the coordinates of the corner points of the marker in the pixel coordinate system specifically includes: Using an edge detection algorithm to find edge information in the image and generate a candidate region; The candidate area is decoded, the marker is identified, and its authenticity and validity are confirmed through geometric verification to obtain the corner point coordinates of the marker in the pixel coordinate system.

8. A marker-assisted visual positioning system for assisting the coordinated management and control of intelligent tunnels, characterized in that: include: An image acquisition module, used for acquiring images; A matrix calculation module, used to measure the markers in the image and calculate the coordinate system transformation matrix; A positioning module is used to spatially locate the target vehicle in the image based on the coordinate system conversion matrix.

9. A marker-assisted visual positioning device for assisting the coordinated management and control of intelligent tunnels, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements a marker-assisted visual positioning method for assisting collaborative management of smart tunnels as described in any one of claims 1 to 7.

Images