A lane space occupancy measurement method based on remote sensing images

Abstract

The present invention provides a lane space occupancy measurement method based on remote sensing images, comprising: using ResNet50 as a LinkNet image feature extraction structure, employing three layers of residual units, two 1*1 convolution kernels, and one 3*3 convolution kernel; adding a dilated convolution to the central region of the LinkNet; introducing a binary cross-entropy loss function in the ResNet50 output layer; collecting images containing roads and vehicles as a dataset to train an improved LinkNet; inputting remote sensing images into the trained LinkNet to obtain a pixel grid image that segments vehicles and roads; calculating vehicle lengths and road lengths; and calculating the ratio of the sum of vehicle lengths to the road length to obtain lane space occupancy. By improving the LinkNet network, the present invention achieves higher accuracy in segmenting road and vehicle images, improving measurement performance and computational accuracy, while reducing measurement costs.

Description

Technical Field

[0001] The present invention relates to the technical field of traffic monitoring, and in particular to a method for measuring lane space occupancy based on remote sensing images. Background Art

[0002] As people's living standards continue to improve and technology continues to advance, the number, quality, and appearance of motor vehicles continue to break through and innovate. At the same time, the negative impact of traffic congestion has become increasingly apparent. Traffic congestion not only causes numerous inconveniences to people's daily lives and work, but also severely restricts the economic development of cities.

[0003] To alleviate traffic congestion, we need to identify road traffic conditions and lane occupancy rates to determine which roads are congested. Lane occupancy, which is divided into time occupancy and space occupancy, is a key indicator for measuring road congestion. Traffic density and vehicle space occupancy are important indicators for determining traffic conditions. Currently, there are many methods for collecting these two types of data in the transportation field, such as geomagnetism, induction coils, and microwave GPS / GIS. However, due to cost and external environmental factors, these methods are unable to effectively and comprehensively collect traffic information.

[0004] In recent years, with the rapid development of digital image processing technology, remote sensing image processing technology has been widely studied and applied in the field of transportation. Traffic information collection methods based on remote sensing images can change the current situation of excessively high cost of road traffic information collection and can collect traffic information on a large road network. Figure 3 The flowchart of measuring lane occupancy in the prior art is shown.

[0005] However, the current method for extracting traffic density and lane space occupancy from remote sensing images is to use traditional image processing to calculate road length using Hough transform, see Figure 4 The figure shows the LinkNet network model used in the existing remote sensing image segmentation model, which then uses fuzzy technology to estimate vehicle length. This traditional method is not very versatile, is costly, requires manual feature extraction, and has poor measurement results. Summary of the Invention

[0006] In view of this, the purpose of the present invention is to propose a method for segmenting road and vehicle remote sensing images based on the LinkNet network, and then compare the lengths of the segmented roads and vehicles to obtain the lane space occupancy rate, thereby solving the problems of high cost and insufficient comprehensiveness of the existing methods for collecting lane space occupancy rate.

[0007] The present invention provides a lane space occupancy measurement method based on remote sensing images, comprising the following steps:

[0008] A. Use ResNet50 as the image feature extraction structure of the LinkNet network model. The ResNet50 uses three layers of residual units, two 1*1 convolution kernels and one 3*3 convolution kernel. The three layers of residual units are used to reduce the number of parameters and the amount of computation.

[0009] Calculation shows that the number of parameters required for a two-layer residual unit is 3*3*256*256*2=1179648, while the number of parameters required for a three-layer residual unit according to the present invention is 1*1*256*64+3*3*64*64+1*1*64*256=69632.

[0010] ResNet50 is used instead of ResNet18 in the original LinkNet network to improve the speed of road segmentation;

[0011] B. Add multiple dilated convolutions to the central area of ​​the LinkNet network model to expand the receptive field of the feature map without reducing the resolution of the extracted feature map;

[0012] Adding dilated convolution helps preserve complete spatial information and makes each convolution output contain larger range information;

[0013] C. Introducing binary cross entropy loss function in the output layer of ResNet50;

[0014] The binary cross entropy loss function shows fast convergence in the early stage and slow convergence in the later stage. Compared with the squared error loss function, it is not affected by the inverse of the learning function.

[0015] The binary cross entropy loss function solves the problem of slow learning rate of neurons in the output layer;

[0016] D. Collect images containing roads and vehicles as a dataset, and use the dataset to train the LinkNet network model improved in steps A-C.

[0017] E. Input the remote sensing image into the trained LinkNet network model to obtain a pixel grid image that separates the vehicle and the road;

[0018] F. Calculate the vehicle length and road length respectively using the pixel grid image; the expression for length calculation is:

[0019]

[0020] In formula (1), z is the length, x1 and y1 are the position values ​​of pixel 1, and x2 and y2 are the position values ​​of pixel 2;

[0021] G. Calculate the ratio of the sum of the vehicle lengths to the road length to obtain the lane space occupancy rate. The calculation formula is:

[0022]

[0023] In formula (2), Rs is the lane space occupancy, L is the total length of the observed section, Li is the length of the i-th vehicle, and n is the number of vehicles in the section.

[0024] Furthermore, the number of dilated convolutions added in step B is 4, and the dilation rates of the 4 dilated convolutions are 1, 2, 4, and 8, respectively.

[0025] Furthermore, the multiple dilated convolutions in step B are connected in cascade mode and parallel mode. The dilated convolutions are connected in cascade mode and then superimposed in parallel mode.

[0026] The present invention combines the advantages of the cascade mode and the parallel mode and uses the two modes in combination.

[0027] Furthermore, the method for calculating the vehicle length in step F includes:

[0028] The pixel values ​​of all vehicle images are sequentially checked along a straight line in the pixel grid image; if a pixel element is continuous, then the element point is a vehicle element.

[0029] Furthermore, the method for calculating the road length in step F includes:

[0030] In the pixel grid image, the pixel values ​​of all road images are checked in sequence along a straight line; if a pixel element has a continuous value of 0, this element point is excluded; and finally, the number of pixels with a continuous pixel value of 1 obtained along the straight line is the length of the road to be calculated.

[0031] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned lane space occupancy measurement method based on remote sensing images when the program is executed by a processor.

[0032] The present invention also provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the lane space occupancy measurement method based on remote sensing images as described above is implemented.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] The present invention improves the LinkNet network, thereby achieving higher precision in segmenting road images and vehicle images, improving measurement effects and calculation accuracy, and reducing measurement costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Various other advantages and benefits will become apparent to those skilled in the art by reading the following detailed description of the preferred embodiment.The accompanying drawings are only for the purpose of illustrating the preferred embodiment and are not to be considered as limiting the present invention.

[0036] In the attached figure:

[0037] Figure 1 This is a flow chart of a lane space occupancy measurement method based on remote sensing images according to the present invention;

[0038] Figure 2 A schematic diagram of the structure of a computer device according to an embodiment of the present invention;

[0039] Figure 3 A schematic diagram of a flow chart of measuring lane occupancy in the prior art;

[0040] Figure 4 It is a diagram of the LinkNet network model structure of the prior art;

[0041] Figure 5 A comparison diagram of the number of parameters of the three-layer residual unit used in the embodiment of the present invention and the existing two-layer residual unit;

[0042] Figure 6 This is a connection diagram of the cascade mode and the parallel mode of the dilated convolution according to an embodiment of the present invention;

[0043] Figure 7 A schematic diagram of a pixel grid image of a method for calculating the lengths of a road and a vehicle according to an embodiment of the present invention. DETAILED DESCRIPTION

[0044] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, like numbers in different figures represent like or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of devices and products consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0045] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. As used in this disclosure and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0046] It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining."

[0047] The embodiments of the present invention are further described in detail below with reference to the accompanying drawings.

[0048] The embodiment of the present invention provides a lane space occupancy measurement method based on remote sensing images, see Figure 1 As shown, the following steps are included:

[0049] A. Use ResNet50 as the image feature extraction structure of the LinkNet network model. The ResNet50 uses three layers of residual units, two 1*1 convolution kernels and one 3*3 convolution kernel. The three layers of residual units are used to reduce the number of parameters and the amount of computation.

[0050] See also Figure 5 As shown, it can be seen from the calculation that the number of parameters required for the two-layer residual unit is: 3*3*256*256*2=1179648, while the number of parameters required for the three-layer residual unit in this embodiment is: 1*1*256*64+3*3*64*64+1*1*64*256=69632;

[0051] ResNet50 is used instead of ResNet18 in the original LinkNet network to improve the speed of road segmentation;

[0052] B. Add multiple dilated convolutions to the central area of ​​the LinkNet network model to expand the receptive field of the feature map without reducing the resolution of the extracted feature map;

[0053] Adding dilated convolution helps preserve complete spatial information and makes each convolution output contain larger range information;

[0054] In this embodiment, four dilated convolutions are added, and the dilation rates of the four dilated convolutions are 1, 2, 4, and 8 respectively;

[0055] See also Figure 6 As shown, the multiple dilated convolutions are connected in cascade mode and parallel mode. After the dilated convolutions are connected in cascade mode, they are superimposed in parallel mode.

[0056] This embodiment combines the advantages of the cascade mode and the parallel mode and uses these two modes in combination;

[0057] C. Introducing binary cross entropy loss function in the output layer of ResNet50;

[0058] The binary cross entropy loss function shows fast convergence in the early stage and slow convergence in the later stage. Compared with the squared error loss function, it is not affected by the inverse of the learning function.

[0059] The binary cross entropy loss function solves the problem of slow learning rate of neurons in the output layer;

[0060] D. Collect images containing roads and vehicles as a dataset, and use the dataset to train the LinkNet network model improved in steps A-C.

[0061] E. Input the remote sensing image into the trained LinkNet network model to obtain a pixel grid image that separates the vehicle and the road;

[0062] The method for calculating the vehicle length comprises:

[0063] In the pixel grid image, the pixel values ​​of all vehicle images are sequentially checked along a straight line; if a pixel element is continuous, then the element point is a vehicle element;

[0064] The method for calculating the road length includes:

[0065] In the pixel grid image, the pixel values ​​of all road images are sequentially checked along a straight line; if a pixel element has a continuous value of 0, the element point is excluded; and the number of pixels with a continuous pixel value of 1 along the straight line is the length of the road to be calculated;

[0066] F. Calculate the vehicle length and road length respectively through the pixel grid image; Figure 7 As shown, the expression for length calculation is:

[0067]

[0068] In formula (1), z is the length, x1 and y1 are the position values ​​of pixel 1, and x2 and y2 are the position values ​​of pixel 2;

[0069] G. Calculate the ratio of the sum of the vehicle lengths to the road length to obtain the lane space occupancy rate. The calculation formula is:

[0070]

[0071] In formula (2), Rs is the lane space occupancy, L is the total length of the observed section, Li is the length of the i-th vehicle, and n is the number of vehicles in the section.

[0072] The embodiment of the present invention improves the LinkNet network, thereby achieving higher accuracy in segmenting road images and vehicle images, improving measurement effects and calculation accuracy, and reducing measurement costs.

[0073] An embodiment of the present invention further provides a computer device, Figure 2 This is a schematic diagram of the structure of a computer device provided by an embodiment of the present invention; see the accompanying drawings Figure 2 As shown, the computer device includes: an input device 23, an output device 24, a memory 22 and a processor 21; the memory 22 is used to store one or more programs; when the one or more programs are executed by the one or more processors 21, the one or more processors 21 implement the lane space occupancy measurement method based on remote sensing images as provided in the above embodiment; wherein the input device 23, the output device 24, the memory 22 and the processor 21 can be connected by a bus or other means, Figure 2 The bus connection is taken as an example.

[0074] Memory 22, as a readable and writable storage medium of a computing device, can be used to store software programs and computer executable programs, such as program instructions corresponding to the lane space occupancy measurement method based on remote sensing images described in an embodiment of the present invention. Memory 22 may primarily include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function; the data storage area may store data created based on the use of the device. Furthermore, memory 22 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 22 may further include memory remotely located relative to processor 21, and such remote memory may be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0075] The input device 23 may be used to receive input digital or character information, and generate key signal input related to user settings and function control of the device; the output device 24 may include a display device such as a display screen.

[0076] The processor 21 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 22, that is, realizes the above-mentioned lane space occupancy measurement method based on remote sensing images.

[0077] The computer device provided above can be used to execute the lane space occupancy measurement method based on remote sensing images provided in the above embodiment, and has corresponding functions and beneficial effects.

[0078] Embodiments of the present invention also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the lane space occupancy measurement method based on remote sensing images as provided in the above embodiments. The storage medium is any of various types of memory devices or storage devices, including: installation media, such as CD-ROMs, floppy disks, or tape drives; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (such as hard disks or optical storage); registers or other similar types of memory elements; the storage medium may also include other types of memory or a combination thereof; in addition, the storage medium may be located in a first computer system in which the program is executed, or may be located in a different second computer system, which is connected to the first computer system via a network (such as the Internet); the second computer system may provide program instructions to the first computer for execution. The storage medium includes two or more storage media that may reside in different locations (e.g., in different computer systems connected via a network). The storage medium may store program instructions (e.g., embodied as a computer program) that can be executed by one or more processors.

[0079] Of course, the storage medium containing computer-executable instructions provided in an embodiment of the present invention is not limited to the lane space occupancy measurement method based on remote sensing images as described in the above embodiment, and can also execute related operations in the lane space occupancy measurement method based on remote sensing images provided in any embodiment of the present invention.

[0080] Thus far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will fall within the scope of protection of the present invention.

[0081] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that the present invention is susceptible to various modifications and variations. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A lane space occupancy measurement method based on remote sensing images, characterized in that: The following steps are involved: A. Use ResNet50 as the image feature extraction structure of the LinkNet network model. The ResNet50 uses three layers of residual units, two 1*1 convolution kernels and one 3*3 convolution kernel. The three layers of residual units are used to reduce the number of parameters and the amount of computation. B. Add multiple dilated convolutions to the central area of ​​the LinkNet network model to expand the receptive field of the feature map without reducing the resolution of the extracted feature map; C. Introducing binary cross entropy loss function in the output layer of ResNet50; D. Collect images containing roads and vehicles as a dataset, and use the dataset to train the LinkNet network model improved by steps AC; E. Input the remote sensing image into the trained LinkNet network model to obtain a pixel grid image that separates the vehicle and the road; F. Calculate the vehicle length and road length respectively using the pixel grid image; the expression for length calculation is: （1） In formula (1), z is the length, x1 and y1 are the position values ​​of pixel 1, and x2 and y2 are the position values ​​of pixel 2; G. Calculate the ratio of the sum of the vehicle lengths to the road length to obtain the lane space occupancy rate. The calculation formula is: （2） In formula (2), Rs is the lane space occupancy rate, L is the total length of the observed road section, Li is the length of the i-th vehicle, and n is the number of vehicles in the road section; The multiple dilated convolutions in step B are connected in cascade mode and parallel mode. After the dilated convolutions are connected in cascade mode, they are superimposed in parallel mode.

2. The lane space occupancy measurement method based on remote sensing images according to claim 1, characterized in that: The number of parameters required for the three-layer residual unit in step A is: 1*1*256*64+3*3*64*64+1*1*64*256=69632.

3. The lane space occupancy measurement method based on remote sensing images according to claim 2, characterized in that: The number of dilated convolutions added in step B is 4, and the dilation rates of the 4 dilated convolutions are 1, 2, 4, and 8, respectively.

4. The lane space occupancy measurement method based on remote sensing images according to claim 1, characterized in that: The method for calculating the vehicle length in step F includes: The pixel values ​​of all vehicle images are sequentially checked along a straight line in the pixel grid image; if a pixel element is continuous, then the element point is a vehicle element.

5. The lane space occupancy measurement method based on remote sensing images according to claim 1, characterized in that: The method for calculating the road length in step F includes: In the pixel grid image, the pixel values ​​of all road images are checked in sequence along a straight line; if a pixel element has a continuous value of 0, this element point is excluded; and finally, the number of pixels with a continuous pixel value of 1 obtained along the straight line is the length of the road to be calculated.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the lane space occupancy measurement method based on remote sensing images according to any one of claims 1 to 5 is implemented.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the lane space occupancy measurement method based on remote sensing images as described in any one of claims 1 to 5 is implemented.

Images