Deterioration level acquisition system

Abstract

To provide a technology capable of identifying the degree of paint deterioration with a simple configuration. [Solution] A deterioration level acquisition system is configured that includes a recognition unit that recognizes paint formed on a road, and a deterioration judgment unit that judges that the paint is more deteriorated the larger the perimeter / area of ​​the paint.

Description

[Technical Field]

[0001] The present invention relates to a deterioration level acquisition system. [Background technology]

[0002] Conventionally, there are known techniques for identifying deteriorated paint from among the paint on roads. For example, Patent Document 1 discloses a technique in which light emitted from a LIDAR is received reflected from the paint, reflection intensity data is acquired, and deteriorated features are identified based on the reflection intensity data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-121809 Summary of the Invention [Problem to be solved by the invention]

[0004] In conventional technology, the reflection intensity is statistically analyzed to determine whether the paint has deteriorated based on the average value μ and standard deviation σ. However, it has not been possible to identify the degree of paint deterioration using a simple configuration. The present invention has been made in consideration of the above-mentioned problems, and aims to provide a technology that can identify the degree of paint deterioration with a simple configuration. [Means for solving the problem]

[0005] To achieve the above objective, the deterioration level acquisition system includes a recognition unit that recognizes paint formed on a road, and a deterioration determination unit that determines that the paint is more deteriorated the larger the perimeter / area of ​​the paint.

[0006] That is, immediately after newly applied paint on a road, the shape is relatively simple. However, as time passes after the paint is installed, deterioration progresses at different rates at various points along the boundary, and some parts peel off, leaving holes, resulting in a complex shape. Generally, the more deterioration progresses, the more complex the shape becomes. As paint deteriorates and its shape becomes more complex, the perimeter of the paint becomes longer and the area becomes smaller compared to before deterioration. Therefore, the larger the perimeter / area of ​​the paint, the more deteriorated the paint can be determined to be. As a result, the degree of paint deterioration can be identified with a simple configuration. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a block diagram showing the configuration of a deterioration level acquisition system. [Figure 2] Figure 2A shows an image of undegraded paint, and Figure 2B shows an image of degraded paint. [Figure 3] 10 is a flowchart of a degradation level acquisition process. [Figure 4] A diagram showing an example of obtaining the perimeter / area of ​​a paint. DETAILED DESCRIPTION OF THE INVENTION

[0008] Here, the embodiments of the present invention will be described in the following order. (1) Configuration of the deterioration level acquisition system: (2) Deterioration level acquisition process: (3) Other embodiments:

[0009] (1) Configuration of the deterioration level acquisition system: 1 is a block diagram showing the configuration of a deterioration level acquisition system 10 according to one embodiment of the present invention. In this embodiment, the deterioration level acquisition system 10 is a server capable of communicating with a vehicle 100 and an administrator terminal 101. The vehicle 100 has a function of capturing images including the paint of the monitoring target, and transmits the captured images to the deterioration level acquisition system 10. The deterioration level acquisition system 10 identifies the deterioration level of the paint based on the captured images.

[0010] The administrator terminal 101 is a terminal used by an administrator who performs paint maintenance, etc. The deterioration level acquisition system 10 has a function of notifying the administrator, who is the notification target, of paint whose deterioration level exceeds a threshold. Based on the information transmitted from the deterioration level acquisition system 10, the administrator terminal 101 displays paint whose deterioration level exceeds a threshold on the user I / F. As a result, the administrator can recognize deteriorated paint.

[0011] The vehicle 100 includes a control unit 200, a communication unit 400, a GNSS receiving unit 410, a vehicle speed sensor 420, a gyro sensor 430, and a camera 440. The control unit 200 includes a CPU, RAM, ROM, etc. (not shown), and can perform various controls according to various programs. In this embodiment, the control unit 200 controls the communication unit 400, the GNSS receiving unit 410, the vehicle speed sensor 420, the gyro sensor 430, and the camera 440. The communication unit 400 is a communication circuit for wireless communication with the degradation level acquisition system 10.

[0012] The GNSS receiver 410 is a device that receives signals from the Global Navigation Satellite System. It receives radio waves from navigation satellites and outputs a signal for calculating the current position of the vehicle via an interface (not shown). The control unit 200 acquires this signal to obtain the current position of the vehicle. The vehicle speed sensor 420 outputs a signal corresponding to the rotational speed of the wheels of the vehicle. The control unit 200 acquires this signal via an interface (not shown) to obtain the vehicle speed. The gyro sensor 430 detects the angular acceleration of the vehicle's turning in a horizontal plane and outputs a signal corresponding to the vehicle's orientation. The control unit 200 acquires this signal to obtain the vehicle's traveling direction. The vehicle speed sensor 420, the gyro sensor 430, and the like are used to determine the vehicle's traveling path. In this embodiment, the current position of the vehicle is determined based on the vehicle's starting point and traveling path, and the current position of the vehicle determined based on the starting point and traveling path is corrected based on the output signal of the GNSS receiver 410.

[0013] The camera 440 is an image sensor for capturing images of the road on which the vehicle 100 is traveling. In this embodiment, the camera 440 is attached to the vehicle 100 so that the paint on the road is included in the field of view. The paint may be various markings formed on the road, but here it is assumed that the paint is a white line as a lane boundary line.

[0014] The control unit 200 acquires the current location of the vehicle 100 based on output signals from the GNSS receiving unit 410, the vehicle speed sensor 420, and the gyro sensor 430. The control unit 200 also acquires image information of an image of the paint that is output from the camera 440. The control unit 200 associates the image information with location information that indicates the current location of the vehicle 100, and transmits the image information to the deterioration level acquisition system 10 via the communication unit 400.

[0015] The timing of acquiring and transmitting image information may be various. For example, the camera 440 may take an image each time the vehicle 100 travels a certain distance or a certain time, and the image information of the captured image may be acquired and transmitted. Alternatively, the camera 440 may take an image in a monitoring section or monitoring point designated in advance by the driver or the like, and the image information of the captured image may be acquired and transmitted.

[0016] The deterioration level acquisition system 10 includes a control unit 20, a storage medium 30, and a communication unit 40. The storage medium 30 is a storage medium capable of storing any information. The communication unit 40 is a communication circuit for wireless communication with the vehicle 100 and the administrator terminal 101.

[0017] The control unit 20 includes a CPU, RAM, ROM, etc. (not shown), and can perform various controls in accordance with various programs. In this embodiment, the programs include a deterioration level acquisition program 21. When the deterioration level acquisition program 21 is executed, the control unit 20 functions as a recognition unit 21a, a deterioration determination unit 21b, and a notification unit 21c.

[0018] The recognition unit 21a has a function of recognizing paint formed on the road. Specifically, the control unit 20, using the function of the recognition unit 21a, acquires image information 30a transmitted by the vehicle 100 via the communication unit 40 and stores it in the storage medium 30. Furthermore, using the function of the recognition unit 21a, the control unit 20 references the image information 30a and performs image recognition processing on the image including the paint to extract the image of the paint. Details of the image recognition processing will be described later.

[0019] The deterioration determination unit 21b has a function of determining that the larger the perimeter / area of ​​the paint, the more deteriorated the paint is. Specifically, the control unit 20 uses the function of the deterioration determination unit 21b to acquire the perimeter and area of ​​the paint image based on the image of the paint. Then, the control unit 20 uses the function of the deterioration determination unit 21b to acquire the perimeter / area as the degree of deterioration, associates the position information indicated by the image information 30a with the degree of deterioration, and stores the result in the storage medium 30 as deterioration degree information 30b.

[0020] The paint's perimeter / area increases as the paint deteriorates, so it can be said to indicate the degree of paint deterioration. In other words, the degree of deterioration is an index that increases the numerical value as the paint deteriorates. Therefore, obtaining the degree of deterioration based on the perimeter / area is equivalent to determining that the paint is more deteriorated the larger the degree of deterioration, which is the paint's perimeter / area.

[0021] 2A and 2B are diagrams illustrating how paint deteriorates. FIGS. 2A and 2B show a binarized image of a road including white line paint. In FIG. 2A, the white line paint P is not deteriorated, while in FIG. 2B, the white line paint P is deteriorated. As shown in FIG. 2A, the undeteriorated white line paint is a line with a constant width, and therefore, in the binarized image, it is recognized as a simple rectangle with opposing sides at the top and bottom of the image.

[0022] On the other hand, as shown in Figure 2B, deteriorated white line paint peels off relatively more in areas where wheels are likely to come into contact, exposing the road surface more than in areas where wheels are unlikely to come into contact. This type of deterioration progresses simultaneously in multiple different areas, and in areas where deterioration is likely to progress, the boundary line is no longer straight and the white line may even be broken. While deterioration can cause various shape changes, the boundary line becomes more complex and its area becomes smaller as deterioration occurs.

[0023] Therefore, in terms of the degree of deterioration, which is the ratio of the perimeter of the paint to its area, the perimeter of the molecule increases as the deterioration progresses, and the area of ​​the molecule decreases as the deterioration progresses. Therefore, the greater the degree of deterioration, the more severe the deterioration can be considered. Note that when white line paint splits due to deterioration, the perimeter and area become smaller than before the split. However, compared to simple shapes such as the artificially formed rectangle shown in Figure 2A, the shape after the deterioration is more disordered, and the boundary line is often not a simple straight line. As a result, the perimeter becomes longer compared to simple shapes. Therefore, it is likely that paint that has progressed in deterioration will have a larger perimeter / area than paint that has not progressed in deterioration.

[0024] As described above, in this embodiment, the control unit 20 acquires the perimeter / area of ​​the paint based on the image and regards it as the degree of deterioration. With the above configuration, it is possible to identify the degree of deterioration of the paint with a simple configuration.

[0025] The notification unit 21c has a function of notifying the notification target when the perimeter / area of ​​a paint is equal to or greater than a threshold. That is, when the deterioration level of a paint reaches a level that requires notification, the control unit 20 notifies the administrator, who is the notification target, of the paint. Specifically, the control unit 20, using the function of the notification unit 21c, references the deterioration level information 30b and identifies paint whose deterioration level is equal to or greater than the threshold. Note that the threshold may be an index for determining whether a paint has reached a state that requires notification. In this embodiment, a common threshold is set for all paints, but different thresholds may be set depending on the type, size, position, etc. of the paint.

[0026] If there is paint whose deterioration level is equal to or greater than the threshold, the control unit 20 extracts the paint whose deterioration level is equal to or greater than the threshold. Then, the control unit 20 transmits position information of the extracted paint to the administrator terminal 101 via the communication unit 40. The administrator terminal 101 displays the notified position of the paint on the display unit of the user I / F. As a result, the administrator can recognize the position of the deteriorated paint.

[0027] The administrator terminal 101 includes a control unit 201 , a communication unit 401 , and a user I / F 411 . The communication unit 40 is a communication circuit for wireless communication with the vehicle 100 and the administrator terminal 101. The control unit 201 includes a CPU, RAM, ROM, etc. (not shown), and can perform various controls according to various programs. In this embodiment, the control unit 201 controls the communication unit 401 and the user I / F 411.

[0028] The communication unit 401 is a communication circuit for wireless communication with the degradation degree acquisition system 10. The user I / F 411 is an interface unit for inputting instructions from the user and providing various information to the user, and is equipped with a display unit and input units such as a keyboard and a mouse, which are not shown.

[0029] When the administrator operates the input unit to input a display request for deteriorated paint, the control unit 200 outputs a notification request for deteriorated paint to the deterioration level acquisition system 10 via the communication unit 401. When the notification request is made, the control unit 20 of the deterioration level acquisition system 10 references the deterioration level information 30b and identifies the location information of paint whose deterioration level is equal to or greater than a threshold. The control unit 20 then transmits the location information of the paint via the communication unit 40.

[0030] The control unit 201 receives the location information via the communication unit 401. Upon receiving the location information, the control unit 201 controls the display unit of the user I / F 411 to display the location of the deteriorated paint. There are no limitations on the manner in which the paint is displayed, and various configurations can be employed, such as displaying an icon indicating the deteriorated paint on a map, or sorting the deteriorated paint according to the distance from the current location and displaying it in a list.

[0031] (2) Deterioration level acquisition process: Next, the deterioration level acquisition process executed by the control unit 20 will be described with reference to Fig. 3. The deterioration level acquisition process is executed every time image information is transmitted from the vehicle 100. When the deterioration level acquisition process is started, the control unit 20 acquires image information 30a using the function of the recognition unit 21a (step S100). That is, the control unit 20 acquires the image information 30a transmitted from the vehicle 100 via the communication unit 40 and stores it in the storage medium 30.

[0032] Next, the control unit 20 performs image recognition processing to recognize paint based on the image information 30a using the function of the recognition unit 21a. Specifically, the control unit 20 references the image information 30a and extracts paint based on color (step S105). That is, since the color of the paint of the monitored object is known, the control unit 20 references the color of each pixel indicated by the image information 30a and identifies pixels that match the color of the paint of the monitored object. In this embodiment, since the color of the paint of the monitored object is white, the control unit 20 identifies white pixels. Note that whether a pixel is white or not may be determined using various methods. For example, a configuration in which a pixel whose gradation value for each color component (e.g., red, green, blue) is within a predetermined range is considered to be a white pixel may be employed.

[0033] If the area in the image represented by the image information 30a where the paint to be monitored may be present is known, the target area from which the paint is extracted may be limited. For example, in the example shown in Figures 2A and 2B, the white line paint may appear only in a specific range on the left side of the image. In this case, since the area where the paint may appear is known, processing may be performed to extract the paint from that area in the image.

[0034] When white pixels are extracted, the control unit 20 binarizes the image by marking the extracted pixels as white and the non-extracted pixels as black. Furthermore, the control unit 20 performs edge extraction processing on the binarized image, and regards the edge pixels as the boundary between the paint and the road. As a result, the boundary between the paint and the road is identified, and the white pixels surrounded by the boundary are extracted as paint.

[0035] Next, the control unit 20 extracts blobs using the function of the recognition unit 21a (step S110). Here, a blob means a mass, and refers to an area where white pixels, which is the color of paint, exist consecutively. Various processes can be used to extract blobs. For example, the control unit 20 sets one of the white pixels included in the binarized image as a pixel of interest, and determines whether or not there are white pixels adjacent to the pixel of interest in multiple directions (e.g., eight directions or four directions). If there are white pixels adjacent to the pixel of interest in multiple directions, the control unit 20 considers the adjacent white pixels to be part of the same blob as the pixel of interest.

[0036] If it is determined that white pixels are adjacent to the pixel of interest in multiple directions, the control unit 20 selects one of the white pixels adjacent to the pixel of interest as a new pixel of interest and repeats the same process. If it is not determined that white pixels are adjacent to the pixel of interest in multiple directions, the control unit 20 selects a new pixel of interest from among the white pixels not included in an existing blob and repeats the same process. The control unit 20 repeats the above process until there are no more white pixels to be selected as the pixel of interest. Through the above process, the pixels that make up the paint image are classified into one or more blobs. Therefore, even if linear paint is separated due to degradation, it is defined as a separate, individual blob, and it becomes possible to define the perimeter and area of ​​each blob.

[0037] Next, control unit 20 acquires the perimeter / area of ​​the blob using the function of deterioration determination unit 21b (step S115). That is, control unit 20 acquires the degree of deterioration indicated by the perimeter / area for each of the blobs extracted in step S110. Specifically, control unit 20 determines the perimeter for each blob based on the number of pixels that make up the outermost perimeter, and determines the area based on the number of pixels that make up the portion excluding the outermost perimeter of the blob, and acquires the perimeter / area.

[0038] Next, the control unit 20 uses the function of the deterioration determination unit 21b to store the deterioration level information 30b (step S120). That is, the control unit 20 associates position information indicating the position where the image information 30a was captured with each of the perimeters / areas acquired in step S115, and stores the position information as deterioration level information 30b in the storage medium 30. Note that when deterioration level information 30b is defined for the same position (or two positions whose distance is equal to or less than a threshold), the original deterioration level information 30b can be updated with the latest deterioration level information 30b.

[0039] According to the above process, it is possible to define deterioration level information 30b for each paint included in an image captured by vehicle 100 and store it in storage medium 30. Therefore, by requesting the display of deteriorated paint at any time, the administrator can be notified of deteriorated paint. In addition, the location of deteriorated paint can be identified.

[0040] (3) Other embodiments: The above embodiment is one example for implementing the present invention, and various other embodiments are also possible. For example, at least some of the recognition unit 21a, the deterioration determination unit 21b, and the notification unit 21c that constitute the deterioration level acquisition system may be separated into multiple devices. As an example of such a configuration, for example, a configuration in which some of the functions of each unit are implemented by an on-board device installed in the vehicle 100 is envisioned. Furthermore, the deterioration level acquisition system 10 may be implemented by an on-board device. In this case, the administrator terminal 101 may be implemented by the on-board device, or may be implemented as a terminal other than the on-board device. Of course, various other embodiments may be realized, such as a configuration in which some of the above-described embodiments are omitted, or a configuration in which the order of processing is changed or omitted.

[0041] The recognition unit only needs to be able to recognize paint formed on the road. In other words, it only needs to be able to distinguish between paint on the road and non-painted areas. Recognition can be performed based on information acquired by various sensors. The sensor is not limited to a camera as in the above-described embodiment. For example, a sensor such as a LIDAR (Light Detection and Ranging) equipped with a light source can irradiate detection light onto the road and acquire reflected light to recognize the paint. Furthermore, the camera can typically be a visible light camera capable of detecting multiple colors. However, various cameras, such as monochrome cameras and infrared cameras, that acquire images of the road including the paint illuminated by natural light can also be used.

[0042] The deterioration determination unit should be able to determine that the paint is more deteriorated as the perimeter / area of ​​the paint increases. As the paint deteriorates, part of the paint peels off, exposing the road. As a result, the shape of the boundary between the paint and the road becomes more complex, and holes may form due to the peeling. As this deterioration progresses, the perimeter of the paint increases and the area decreases. Therefore, if the perimeter / area is specified as the deterioration level, it can be determined that the paint is more deteriorated as the deterioration level value increases. The perimeter may be the length of the boundary line between the paint and another feature (typically a road), and may be counted using various methods. The area may be the size of the paint portion inside the boundary line, and may be counted using various methods.

[0043] The degree of deterioration defined by the perimeter / area may be used for comparison with a standard, or may be used to evaluate the deterioration based on the magnitude of the value. In the former case, it is possible to adopt a configuration in which the degree of deterioration is compared with the degree of deterioration of the paint before deterioration or the degree of deterioration of other paints, etc.

[0044] The paint area may be any index capable of evaluating the size of the area inside the boundary between the paint and the road. Therefore, the deterioration assessment unit may extract the area inside the boundary using a predetermined rule, and obtain the area of ​​the extracted portion as the paint area. Various rules may be used as the predetermined rule. For example, a rule may be used in which an area within the boundary line, a certain distance from the boundary line, is extracted, and the area of ​​the extracted portion is measured.

[0045] 4A to 4C are diagrams showing such an example. In FIGS. 4A to 4C, boundary line B between the paint and the road is shown as a solid black line. Inside boundary line B, a range within a certain distance from boundary line B is colored gray. This range can be defined, for example, by aligning the center of a circle with the same radius as the certain distance with boundary line B, and moving the circle along boundary line B while maintaining its center aligned with boundary line B. In other words, the part inside boundary line B that the circle passes through is the range within boundary line B that is within the certain distance from boundary line B.

[0046] Once the area is identified in the above manner, the deterioration level of the paint is obtained based on the perimeter / area of ​​the paint. The above processing prevents the range of possible values ​​of the deterioration level for undegraded paint from becoming excessively large. For example, the deterioration level value shown in FIG. 4C is larger than when the entire area inside boundary line B is obtained as the area, and is closer to the deterioration level value in the example shown in FIG. 4B. Therefore, it is possible to define the deterioration level as the same value for the undegraded but differently sized areas shown in FIGS. 4B and 4C. This makes it easier to evaluate the degree of deterioration based on the deterioration level value.

[0047] The notification unit may notify the notification target when the perimeter / area of ​​the paint is equal to or greater than a threshold. The threshold may be a criterion for determining whether the deterioration level indicated by the perimeter / area is high enough to require notification. The threshold may be determined by various methods, for example, depending on the type of paint (lane boundary line, stop line, speed marking, etc.) or the size of the paint (area, etc.). The notification may be performed in various ways. For example, information indicating the deterioration level may be transmitted to an administrator terminal, and information that allows the deterioration level to be visually recognized (such as a numerical value or bar graph indicating the deterioration level, or an icon colored according to the deterioration level) may be displayed on the administrator terminal.

[0048] Furthermore, the degree of deterioration may be defined by the perimeter / area and other indices. For example, in the configuration shown in Fig. 1, the recognition unit 21a may recognize an area (blob) where pixels of the paint color exist consecutively as paint, and the deterioration determination unit 21b may determine that the paint is more deteriorated as the number of areas increases.

[0049] That is, as deterioration progresses, the paint may become fragmented as shown in FIG. 2B. Therefore, when deterioration occurs, the number of independently existing blobs is likely to increase compared to when there is no deterioration. Therefore, after step S110 of the deterioration level acquisition process shown in FIG. 3 described above, the control unit 20 counts the number of extracted blobs. Then, in step S120, the control unit 20 defines the deterioration level information 30b so that the value increases as the number of blobs increases.

[0050] Various methods may be used to define the degradation level information 30b so that the greater the number of blobs, the greater the value. For example, a term may be added to the degradation level obtained from the perimeter / area, such that the greater the number of blobs, the greater the value. Alternatively, the degradation level obtained from the perimeter / area may be multiplied by a number greater than or equal to 1, such that the greater the number of blobs, the greater the value. Alternatively, the control unit 20 may determine the degradation level for each of multiple blobs based on the perimeter / area, and define the degradation level determined based on the number of blobs as a common value for all of the multiple blobs that are located nearby. In either case, evaluating the degradation level based on the number of blobs enables more accurate determination of the degradation level.

[0051] Furthermore, the method of determining the degree of paint deterioration based on the perimeter / area of ​​the paint, as in the present invention, can also be applied as a program or method. The above-described system, program, and method may be realized as a standalone device or may be realized using components shared with various parts of the vehicle, and include various aspects. Furthermore, they can be modified as appropriate, such as being partly software and partly hardware. Furthermore, the invention can also be realized as a recording medium for a program that controls the system. Of course, the recording medium for the program may be a magnetic recording medium or a semiconductor memory, and any recording medium developed in the future can be considered in the same way. [Explanation of symbols]

[0052] 10...deterioration level acquisition system, 20...control unit, 21...deterioration level acquisition program, 21a...recognition unit, 21b...deterioration determination unit, 21c...notification unit, 30...storage medium, 30a...image information, 30b...deterioration level information, 40...communication unit, 100...vehicle, 101...administrator terminal, 200...control unit, 201...control unit, 400...communication unit, 401...communication unit, 410...GNSS receiving unit, 411...user I / F, 420...vehicle speed sensor, 430...gyro sensor, 440...camera

Claims

1. a recognition unit that recognizes paint formed on a road; a deterioration determination unit that determines that the paint is more deteriorated as the perimeter / area of ​​the paint increases; A deterioration level acquisition system comprising:

2. The recognition unit Recognizing the paint based on an image of a road captured by a camera; The deterioration level acquisition system according to claim 1 .

3. The recognition unit A region where pixels of the paint color are continuously present is recognized as the paint, The deterioration determination unit The greater the number of the regions, the more deteriorated the paint is determined to be. The deterioration level acquisition system according to claim 2 .

4. a notification unit that notifies a notification target person when the perimeter / area of ​​the paint is equal to or greater than a threshold value; The deterioration level acquisition system according to any one of claims 1 to 3.

Images