Method and apparatus for determining retroreflective brightness coefficient, electronic device and program product

Abstract

The present application discloses a method for determining the retroreflective brightness coefficient, which includes: obtaining a first image and a second image corresponding to a detected position in the marking detection, where the first image is an image acquired under the condition of single-wavelength laser projection, and the second image is an image acquired under the condition of no single-wavelength laser projection; using a given image segmentation algorithm to identify a first marking area in the first image; using the given image segmentation algorithm to identify a second marking area in the second image; extracting a first gray value of the first marking area in the first image; extracting a second gray value of the second marking area in the second image; calculating the retroreflective brightness coefficient of the marking at the detected position according to the first gray value, the second gray value, and a given standard value of the retroreflective brightness coefficient. The present application also discloses a retroreflective brightness coefficient inspection and evaluation device, an electronic device, and a program product.

Description

Technical Field

[0001] The present application relates to the field of road traffic, and specifically to a method and device for determining the retroreflective brightness coefficient, an electronic device, and a program product. Background Art

[0002] Road traffic markings are key traffic safety facilities applied on the road surface, aiming to channel traffic flow, induce vehicle alignment, and ensure traffic safety. These markings have multiple functions of defining the boundaries of driving lanes, regulating and managing drivers' driving behaviors, guiding drivers' lines of sight, indicating and predicting road conditions ahead, and clarifying the use of road rights.

[0003] After being rolled over for a long time, the visibility of road traffic markings decreases, resulting in a decline in the functions of the markings. At this time, maintenance is required. An important evaluation criterion for judging whether road markings need maintenance is the retroreflective brightness coefficient of road traffic markings, which can also be simply referred to as the retroreflective brightness coefficient.

[0004] In China, a handheld retroreflective measuring instrument is commonly used to detect the retroreflective brightness coefficient of markings. Static inspection and evaluation require road closure, which has a great social impact and immeasurable costs. Dynamic inspection and evaluation is fast and has no impact on traffic, but the detection is easily affected by strong environmental light interference, dynamic geometric angle changes, etc., and often involves complex settings and strict operating conditions, resulting in an inability to accurately determine the retroreflective brightness coefficient of the markings.

[0005] The content described in this background art is only for facilitating the understanding of the related technologies in this field and is not regarded as an admission of the prior art. Summary of the Invention

[0006] Therefore, the embodiments of the present application hope to provide a solution for determining the retroreflective brightness coefficient that can at least partially solve the problems described above.

[0007] In a first aspect, a method for determining the retroreflective brightness coefficient is provided, including:

[0008] Obtaining a first image and a second image corresponding to a detected position in the marking detection, where the first image is an image collected under the condition of single-wavelength laser projection, and the second image is an image collected under the condition of no single-wavelength laser projection;

[0009] Identifying a first marking area in the first image by using a given image segmentation algorithm;

[0010] Identifying a second marking area in the second image by using a given image segmentation algorithm;

[0011] Extracting a first gray value of the first marking area in the first image;

[0012] Extracting a second gray value of the second marking area in the second image;

[0013] Calculate the retroreflective brightness coefficient of the marking line at the detected position according to the first gray value, the second gray value, and the given standard value of the retroreflective brightness coefficient.

[0014] Optionally, calculating the retroreflective brightness coefficient of the marking line at the detected position according to the first gray value, the second gray value, and the given standard value of the retroreflective brightness coefficient includes:

[0015] Determine the gray difference of the marking line between the first gray value and the second gray value;

[0016] Obtain the standard value of the retroreflective brightness coefficient corresponding to the given marking standard plate;

[0017] Obtain the standard gray difference corresponding to the given marking standard plate;

[0018] Determine the retroreflective brightness coefficient of the marking line at the detected position according to the proportional relationship among the gray difference of the marking line, the standard gray difference, and the standard value of the retroreflective brightness coefficient.

[0019] Optionally, when or after calculating the retroreflective brightness coefficient of the marking line at the detected position, the method further includes:

[0020] Correct the retroreflective brightness coefficient of the marking line to obtain the retroreflective brightness coefficient corrected by ambient light, specifically including:

[0021] Obtain the ambient light parameters during the marking line detection;

[0022] Determine the ambient light correction coefficient according to the ambient light parameters, the gray difference of the marking line, and the standard value of the retroreflective brightness coefficient of the marking line at the detected position;

[0023] Correct the retroreflective brightness coefficient according to the ambient light correction coefficient to obtain the retroreflective brightness coefficient corrected by ambient light.

[0024] Optionally, obtaining the standard gray difference corresponding to the given marking standard plate includes:

[0025] Obtain the third image and the fourth image corresponding to the given marking standard plate, where the third image is an image collected under the condition of single-wavelength laser projection, and the fourth image is an image collected under the condition of no single-wavelength laser projection;

[0026] Use a given image segmentation algorithm to identify the third marking area in the third image;

[0027] Use a given image segmentation algorithm to identify the fourth marking area in the fourth image;

[0028] Extract the third gray value of the third marking area in the third image;

[0029] Extract the fourth gray value of the fourth marking area in the fourth image;

[0030] Determine the standard gray difference between the third gray value and the fourth gray value.

[0031] Optionally, when or after calculating the retroreflective brightness coefficient of the marking at the detected position, the method further includes:

[0032] Correct the retroreflective brightness coefficient of the marking to obtain a distance-corrected retroreflective brightness coefficient, specifically including:

[0033] Obtain the calibrated projection distance and the actual projection distance of the single-wavelength laser projection;

[0034] Determine a distance correction coefficient according to the calibrated projection distance and the actual projection distance;

[0035] According to the distance correction coefficient, correct the retroreflective brightness coefficient of the marking to obtain a distance-corrected retroreflective brightness coefficient.

[0036] Optionally, when or after calculating the retroreflective brightness coefficient of the marking at the detected position, the method further includes:

[0037] Correct the retroreflective brightness coefficient of the marking to obtain an ambient-light-corrected and distance-corrected retroreflective brightness coefficient, specifically including:

[0038] Obtain the ambient light parameters in the marking detection;

[0039] Determine an ambient light correction coefficient according to the ambient light parameters, the marking gray difference, and the standard value of the retroreflective brightness coefficient of the marking at the detected position;

[0040] Obtain the calibrated projection distance and the actual projection distance of the single-wavelength laser projection;

[0041] Determine a distance correction coefficient according to the calibrated projection distance and the actual projection distance;

[0042] According to the ambient light correction coefficient and the distance correction coefficient, correct the retroreflective brightness coefficient of the marking to obtain an ambient-light-corrected and distance-corrected retroreflective brightness coefficient.

[0043] Optionally, before obtaining the first image and the second image, the method further includes:

[0044] Determine the calibration height parameters for the marking visibility observation model, where the calibration height parameters include the calibrated driver observation height and the calibrated vehicle light height;

[0045] Determine the calibration observation distance for the marking visibility observation model;

[0046] Set the calibration projection distance of the light projection for marking detection, and based on the proportional relationship between the calibration observation distance and the calibration projection distance, determine the marking detection height parameter for marking detection from the calibration height parameter, where the marking detection height parameter includes the camera height for image acquisition and the light source height for light projection; or, at least partially set the marking detection height parameter for marking detection, and based on the proportional relationship between the calibration height parameter and at least partially set marking detection height parameter, determine the calibration projection distance of the light projection for marking detection from the calibration observation distance;

[0047] Wherein, the first image and the second image are acquired at the calibration projection distance, camera height, and light projection height.

[0048] In a second aspect, there is provided a method for determining the retroreflective brightness coefficient, including:

[0049] Determine the calibration height parameter for the marking visibility observation model, where the calibration height parameter includes the calibration driver observation height and the calibration vehicle light height;

[0050] Determine the calibration observation distance for the marking visibility observation model;

[0051] Set the calibration projection distance of the light projection for marking detection, and based on the proportional relationship between the calibration observation distance and the calibration projection distance, determine the marking detection height parameter for marking detection from the calibration height parameter, where the marking detection height parameter includes the camera height for image acquisition and the light projection height; or, at least partially set the marking detection height parameter, and based on the proportional relationship between the calibration height parameter and at least partially set marking detection height parameter, determine the calibration projection distance of the light projection for marking detection from the calibration observation distance;

[0052] Obtain the marking detection data corresponding to the detected position in the marking detection, where the image in the marking detection data is acquired at the calibration projection distance, camera height, and light projection height;

[0053] Determine the retroreflective brightness coefficient according to the marking detection data.

[0054] In a third aspect, there is provided a retroreflective brightness coefficient inspection and evaluation device, including:

[0055] An acquisition unit configured to acquire a first image and a second image corresponding to a detected position in a road marking detection, where the first image is an image acquired under the condition of single-wavelength laser projection, and the second image is an image acquired under the condition of no single-wavelength laser projection;

[0056] A first road marking recognition unit configured to identify a first road marking area in the first image by using a given image segmentation algorithm;

[0057] A second road marking recognition unit configured to identify a second road marking area in the second image by using a given image segmentation algorithm;

[0058] A first image processing unit configured to extract a first grayscale value of the first road marking area in the first image;

[0059] A second image processing unit configured to extract a second grayscale value of the second road marking area in the second image;

[0060] A retroreflective brightness coefficient calculation unit configured to calculate the retroreflective brightness coefficient of the road marking at the detected position according to the first grayscale value, the second grayscale value, and a given standard value of the retroreflective brightness coefficient.

[0061] Optionally, the acquisition unit is configured to acquire an ambient light parameter value corresponding to the detected position in the road marking detection; and configured to acquire the actual projection distance of the single-wavelength laser corresponding to the detected position in the road marking detection;

[0062] The retroreflective brightness coefficient evaluation device further includes:

[0063] A third processing unit configured to extract an ambient light correction coefficient and a distance correction coefficient of the road marking at the detected position;

[0064] The retroreflective brightness coefficient calculation unit is configured to calculate the retroreflective brightness coefficient of the road marking at the detected position according to the first grayscale value, the second grayscale value, the ambient light correction coefficient, the distance correction coefficient, and a given standard value of the retroreflective brightness coefficient.

[0065] Optionally, the retroreflective brightness coefficient evaluation device further includes:

[0066] A retroreflective brightness coefficient evaluation unit configured to receive the retroreflective brightness coefficient calculated by the calculation unit, and judge the retroreflective performance of the road marking according to a preset performance index threshold to evaluate whether it meets the specified safety standard.

[0067] In a fourth aspect, there is provided an electronic device, characterized in that it includes: a processor and a memory storing a computer program, and the processor is configured to implement the method of the embodiment of the present application when running the computer program.

[0068] In a fifth aspect, a program product is provided, including a computer program, where when the computer program is executed by a processor, the method of the embodiments of the present application is implemented.

[0069] The retroreflective brightness coefficient determination solution provided by one aspect of the present application not only processes two types of images with or without single-wavelength laser projection, but also accurately segments the marking area by using an image segmentation algorithm for each of these two types of images. By extracting the gray values from the marking area segmented from the marking image and combining them with the ambient light parameters, the actual light projection distance, and the standard value of the retroreflective brightness coefficient of the standard plate, this method can accurately determine the retroreflective brightness coefficient of the object to be measured.

[0070] The retroreflective brightness coefficient determination solution provided by another aspect of the present application can construct a marking detection model based on light projection in proportion to the marking visibility observation model to determine the retroreflective brightness coefficient. While reducing the measurement complexity and the limitations on measurement equipment, this method effectively ensures the accuracy of the detection result, and thus ensures the accuracy of the determined retroreflective brightness coefficient.

[0071] Some of the optional features and other effects of the embodiments of the present application are described below, and some can be understood by reading this article. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Hereinafter, the embodiments of the present application will be described in detail with reference to the drawings. The elements shown are not limited by the scale shown in the drawings, and the same or similar reference numerals in the drawings represent the same or similar elements, where:

[0073] Figure 1 A schematic diagram of a marking detection device according to an embodiment of the present application is shown. The marking detection device can obtain marking detection data for the retroreflective brightness coefficient determination solution according to the embodiment of the present application;

[0074] Figure 2 An example diagram of an image processed by the retroreflective brightness coefficient determination solution according to an embodiment of the present application is shown;

[0075] Figure 3 An example diagram of an image processed by the retroreflective brightness coefficient determination solution according to an embodiment of the present application is shown;

[0076] Figure 4 An exemplary flowchart of the retroreflective brightness coefficient determination method according to an embodiment of the present application is shown;

[0077] Figure 5 An exemplary flowchart of the marking identification processing of the retroreflective brightness coefficient determination method according to an embodiment of the present application is shown;

[0078] Figure 6shows a schematic structural diagram of a marking identification model that can be used to implement Figure 5 the marking identification process;

[0079] Figure 7 shows Figure 6 another schematic structural diagram of the marking identification model shown;

[0080] Figure 8 shows Figure 6 a schematic structural diagram of the feature extraction convolutional module of the marking recognition model shown;

[0081] Figure 9 shows an exemplary flowchart of a retroreflective brightness coefficient determination method according to an embodiment of the present application;

[0082] Figure 10 shows an exemplary flowchart of a retroreflective brightness coefficient determination method according to an embodiment of the present application;

[0083] Figure 11 shows an exemplary flowchart of the correction process of the retroreflective brightness coefficient of the retroreflective brightness coefficient determination method according to an embodiment of the present application;

[0084] Figure 12 shows an exemplary flowchart of a retroreflective brightness coefficient determination method according to an embodiment of the present application;

[0085] Figure 13 shows an exemplary flowchart of a retroreflective brightness coefficient determination method according to an embodiment of the present application;

[0086] Figure 14 shows a schematic diagram of a retroreflective brightness coefficient inspection and evaluation device according to an embodiment of the present application; and

[0087] Figure 15 shows a structural diagram of an electronic device that can be used to implement the retroreflective brightness coefficient determination method according to an embodiment of the present application. Detailed implementation manners

[0088] To make the purpose, technical solutions, and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the specific implementation manners and the accompanying drawings. Here, the schematic implementation manners of the present application and their descriptions are used to explain the present application, but do not limit the present application.

[0089] As used herein, the term "including" and its variations mean open inclusion, i.e., "including but not limited to". Unless otherwise specified, the term "or" means "and / or". The term "based on" means "at least partially based on". The terms "an example embodiment" and "an embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". For ease of understanding this specification, the sequential terms "first", "second", etc. are used herein to distinguish different elements / articles / objects and do not denote the order or importance of different elements / articles / objects. In particular, the method steps denoted by the terms "first", "second", etc. are not used to denote the execution order of the method; when an embodiment includes elements / articles / objects denoted by a later order, the elements / articles / objects denoted by the terms "first", "second", etc. in the earlier order are not necessarily essential technical features of the embodiment.

[0090] In this application, a retroreflective brightness coefficient determination solution capable of accurately determining the retroreflective brightness coefficient of a marking line is provided, specifically relating to a retroreflective brightness coefficient determination method, a retroreflective brightness coefficient determination device, and an electronic device, a program product, and a storage medium that can implement the retroreflective brightness coefficient determination method.

[0091] As described above, the retroreflective brightness coefficient determination solution in one aspect of this application not only processes two types of images with or without single-wavelength laser projection, but also accurately segments the marking line area for each of these two types of images using an image segmentation algorithm. The retroreflective brightness coefficient determination solution in another aspect of this application constructs a marking line detection model based on light projection in proportion to the marking line visibility observation model for determining the retroreflective brightness coefficient. It will be understood that the embodiments of this application are intended to cover any combination solution involving any one or both of the above aspects, which falls within the scope of this application.

[0092] In the retroreflective brightness coefficient determination solution of the embodiments of the present application, optional or preferred features / solutions may also be included. These optional or preferred features / solutions are innovative in themselves and can also be applied to other applications other than determining the retroreflective brightness coefficient. Therefore, those skilled in the art will understand that independent of the retroreflective brightness coefficient determination solution, these optional or preferred features can be claimed in additional or subsequent applications. These optional or preferred features / solutions include, but are not limited to: a marking detection solution for obtaining marking detection data, a marking recognition solution for identifying a marking area, and a correction solution for correcting the retroreflective brightness coefficient. Conversely, those skilled in the art will also understand that the retroreflective brightness coefficient determination solutions of different embodiments of the present application may or may not include (involve or not involve) these optional or preferred features / solutions, and may include (involve) all or part of these optional or preferred features / solutions, which will be further described below.

[0093] As described above, in a preferred embodiment of the present application, the marking detection data for determining the retroreflective brightness coefficient of the marking can be detected by a marking detection device according to the embodiments of the present application.

[0094] Reference Figure 1 , a marking detection device 100 of this embodiment is shown. The marking detection device 100 is configured to detect marking detection data for determining the retroreflective brightness coefficient of the marking. However, as described above, the retroreflective brightness coefficient determination method according to the embodiments of the present application can use other devices or other means to obtain relevant marking detection data, which falls within the scope of the present application.

[0095] Continuing to refer to Figure 1 , the marking detection device 100 may include a camera 110 and a laser source 120. As Figure 1 shown, the camera 110 may be configured to collect an image of the detected position at a downwardly inclined viewing angle. The laser source 120 may be configured to project a single-wavelength laser at a downwardly inclined projection angle onto the detected position. In a preferred embodiment, the single-wavelength laser may be an infrared laser or a green laser. In a preferred example, the wavelength of the single-wavelength laser may be approximately 808 nm. The detection data according to the embodiments of the present application that includes both the projection of the single-wavelength laser and the non-projection image is particularly conducive to detecting or identifying markings, especially markings in images in an outdoor natural light environment.

[0096] In some embodiments of the present application, the marking detection device further includes a filter assembled on the camera, and the wavelength of the filter corresponds to the wavelength of the single-wavelength laser projected by the laser source. This further helps to detect or identify markings in images in an outdoor natural light environment.

[0097] Combined with reference to Figures 1 to 3, the marking line detection device 100 can include a first mode and a second mode. In the first mode, the camera 110 captures images when the laser source 120 projects laser light ( Figure 1 ), and in the second mode, the camera 110 captures images when the laser source 120 does not project laser light (not shown in the figure). Thus, the marking line detection device 100 can detect the marking line detection data corresponding to the detected position, and the marking line detection data corresponding to the detected position includes the first image captured in the first mode (as shown in Figure 2 ) and the second image captured in the second mode (as shown in Figure 3 ).

[0098] Continuing to refer to Figure 1 , the camera 110 can be located above the laser source 120. Although not shown in the figure, the camera 110 and the laser source 120 have substantially the same lateral position in the direction transverse to the projection direction of the laser source 120, that is, the camera 110 is substantially directly above the corresponding laser source 120.

[0099] As described above, a marking line detection model based on light projection can be constructed proportionally using a marking line visibility observation model. Furthermore, the setting angles and heights of the camera 110 and the laser source 120 of the marking line detection device 100 can be determined according to the marking line detection model.

[0100] In a preferred embodiment, the marking line visibility observation model can include a calibration height parameter, and the calibration height parameter includes the calibration driver observation height H d and the calibration vehicle light height H cl . In one embodiment, the calibration driver observation height can represent the height of the eyes of a simulated driver driving a small passenger car. In one example, the calibration driver observation height H d can be 120 cm. In one embodiment, the calibration vehicle light height H cl can represent the height of the front headlights of a small passenger car. In one example, the calibration vehicle light height H cl can be 65 cm.

[0101] In a preferred embodiment, the marking line visibility observation model can include a calibration observation distance D d , which can represent the farthest distance that the eyes of a simulated driver driving a small passenger car can visually observe, and it is assumed that the vehicle lights project to the same position, that is, the vehicle light projection distance D cl is equal to the calibration observation distance. In one example, both the calibration observation distance D d and the vehicle light projection distance D cl are 30 m.

[0102] In a preferred embodiment, the marking line detection model based on light projection may include a calibrated projection distance D s and a marking line detection height parameter. The marking line detection height parameter of the marking line detection model based on light projection may include a camera height H c and a light projection height H of the laser source l .

[0103] In a preferred embodiment of the present application, the calibrated projection distance D s can be preset as needed, and the marking line detection height parameter can be determined by the calibrated height parameter according to the proportional relationship between the preset calibrated observation distance and the calibrated projection distance. Further, according to the proportional relationship between the preset calibrated observation distance and the calibrated projection distance, the camera height can be determined by the driver's observation height, and / or the light projection height can be determined by the calibrated vehicle light height

[0104] In this embodiment, with reference to Figure 1 , the marking line detection height parameter can be determined according to the following formulas (1) and (2):

[0105] H c =H d *D s / D d (1)

[0106] H l =H cl *D s / D d (2)

[0107] In an example, the ratio of the camera height of the camera to the light source height of the laser source is 24 / 13 ± 5%. More preferably, the camera height is 24 cm ± 0.5 cm, and the light source height is 13 cm ± 0.5 cm

[0108] In another preferred embodiment of the present application, the marking detection height parameter can be at least partially preset as needed. For example, the camera height and / or the light source height can be preset, and the calibrated projection distance can be determined from the calibrated observation distance according to the proportional relationship between the preset marking detection height parameter (camera height and / or light source height) and the calibrated height parameter (calibrated driver observation height and / or calibrated vehicle light height). In this preferred embodiment, the light source height can be preset, and the calibrated projection distance can be determined according to the proportional relationship between the marking detection height parameter and the calibrated height parameter; or the camera height can be preset, and the light source height can be determined according to the proportional relationship between the calibrated driver observation height and the calibrated vehicle light height, and then the calibrated projection distance can be determined according to the proportional relationship between the marking detection height parameter and the calibrated height parameter. However, other specific solutions can also be conceived as long as they conform to determining the calibrated projection distance according to at least partially preset marking detection height parameters and the proportional relationship between the marking detection height parameter and the calibrated height parameter.

[0109] In a preferred embodiment, the marking visibility observation model can include an observation angle parameter, and the calibrated angle parameter includes the driver observation angle and the vehicle light projection angle. In one embodiment, the driver observation angle can represent the angle of the farthest position (which is also the vehicle light projection position) visually observed by a simulated driver driving a minibus relative to the road surface. In one embodiment, the vehicle light projection angle can be the angle of the front vehicle lamp to the above-mentioned vehicle light projection position relative to the road surface.

[0110] In a preferred embodiment, the marking detection model based on light projection can further include a marking detection angle parameter, which can include the observation angle of the camera and the projection angle of the laser source.

[0111] In the preferred embodiment of the present application, the observation angle of the camera can also be determined according to the driver observation angle. In one example, the observation angle of the camera is 2.29° ± 0.05°. In the preferred embodiment of the present application, the projection angle of the laser source can also be determined according to the vehicle light projection angle. In one example, the projection angle of the laser source is 1.24° ± 0.05°.

[0112] In the above preferred embodiment, the corresponding relationship between the parameters of the marking detection model based on light projection and the marking visibility observation model is described. It will be understood that the above corresponding relationship between the parameters is a rough corresponding relationship, intended to cover any equivalents and / or fluctuations falling within the application idea. Specifically, the above corresponding relationship clearly covers the fluctuation range within ±5% of the exact corresponding value.

[0113] In the embodiments of the present application, the detection positions of the marking detection device 100 may include various road surfaces, including but not limited to road surfaces with markings, and may also include road surfaces on which marking standard plates are placed. In an exemplary embodiment, the marking standard plate may have any suitable length, such as a length of 50 cm. In the embodiment as Figure 1 shown, when light is projected onto the marking standard plate, the light projection position may be the midpoint of the width W s of the marking standard plate, but the present application is not limited thereto.

[0114] As Figure 1 shown, the marking detection device 100 may further include a distance sensing unit 130 configured to obtain the actual projection distance of the laser projected by the laser source to the detected position. With the light projection distance obtained in real time by means of this distance sensing unit 130, it is particularly beneficial to implement the distance correction process in the embodiments of the present application. In some embodiments, the distance sensing unit 130 may be configured to be synchronized with the light projection of the laser source. For example, the light projection moment t0 of the laser source can be obtained, and the moment t1 when the distance sensing unit 130 receives the reflected light of the projected light can be obtained, and the light projection distance can be determined accordingly. In an example, the distance sensing unit 130 is, for example, a TOF sensor, but the present application is not limited thereto.

[0115] As Figure 1 shown, the marking detection device 100 may further include an ambient light sensing unit 140 configured to obtain ambient light parameters when collecting the first and / or second images. Accordingly, the marking detection data will also include the ambient light parameters.

[0116] In a control example, the following test experiments are carried out on the light source involved therein and its visibility analysis. More specifically, the effects of the marking detection device under the laser light source according to the embodiments of the present application and conventional light sources such as xenon light sources, halogen light sources, and LED light sources can be compared and analyzed through test experiments.

[0117] Specifically, images of markings under different types of light sources and images of markings with the illumination light source turned off are taken with a camera in a dark room and in the natural daylight environment outdoors during the day, and 5 pictures are continuously taken each time. Among them, in this test experiment, except for the different light sources used, the marking detection device adopts similar components, configurations, and parameters of the marking detection model as Figure 1 described in the example (however, this does not mean that the marking detection device using a conventional light source is prior art). Then, the marking images are processed, the gray values of the marking areas in each picture are extracted, and their characteristics are analyzed.

[0118] The research results show that in a dark room, under the light environments irradiated by xenon lamps, halogen lamps, and LED lamps, the marking images captured by the camera and their grayscale values can be clearly identified, and there are differences from the grayscale values of the marking pictures captured with the light source turned off. However, in the outdoor natural light environment, when the xenon lamp, halogen lamp, and LED light sources are turned on, there is no significant difference between the grayscale values of the marking images captured by the camera and the grayscale values of the marking images captured with the conventional light source turned off. After subtracting the two (eliminating the interference of natural light), it is almost impossible to obtain the grayscale value of the marking under the irradiation of only the artificial light source. In contrast, for the marking detection device using the laser light source of the embodiment of the present application, more specifically, when the camera with a filter (only passing 808 nm) captures the infrared laser (wavelength of 808 nm) irradiation, the marking can be well recognized and the grayscale of the marking under the irradiation of only the artificial light source can be extracted in both cases, as Figure 2 shown. The test results of the green laser are consistent with those of the infrared laser, as Figure 3 shown.

[0119] In some embodiments of the present application, the marking detection device includes a plurality of the cameras and a plurality of the laser sources, and each of the cameras is respectively matched with one of the laser sources. The plurality of image acquisition units are spaced apart from each other in a direction transverse to the laser projection direction of the laser source, and each of the cameras and its respective matched laser source have substantially the same lateral position in the direction transverse to the projection direction of the laser source. In some embodiments, the marking detection device can be a vehicle-mounted marking detection device. In this vehicle-mounted marking detection device, for example, cameras and the matched laser sources can be respectively installed on the left front side, right front side, and optionally the middle position of the vehicle.

[0120] Next, a method for determining the retroreflective brightness coefficient according to an embodiment of the present application will be described.

[0121] Referring to Figure 4 , a method for determining the retroreflective brightness coefficient according to an embodiment of the present application is shown, including steps S410 to S460:

[0122] S410: Obtain a first image and a second image corresponding to the detected position in the marking detection.

[0123] Wherein, the first image is an image captured under the condition of single-wavelength laser projection, and the second image is an image captured under the condition of no single-wavelength laser projection.

[0124] In one embodiment, the method for determining the retroreflective brightness coefficient can, for example, use the marking detection data detected by the marking detection device 100 of the above embodiment. However, it can be conceived that in other embodiments, other first and second images related to the marking can be used as long as the first and second images are respectively images obtained under the conditions of light projection and no light projection.

[0125] In some embodiments of the present application, the retroreflective brightness coefficient determination method can be obtained in real time by a vehicle-mounted marking detection device during free driving. In a further embodiment, the retroreflective brightness coefficient determination method can be executed in real time by an electronic device located on the vehicle or by an electronic device of the vehicle itself. In still a further embodiment, the retroreflective brightness coefficient determination method can be executed in real time by a cloud or remote electronic device. When the detection data is obtained by the marking detection device 100 of the above embodiments, the cloud or remote electronic device can be communicatively connected to the marking detection device 100, for example, directly communicatively connected or communicatively connected through the communication unit of the vehicle.

[0126] In some embodiments, the first and second images matching the marking detection data can be obtained in real time at a certain interval frequency. For example, the camera can take 30 images per minute at equal time intervals, and the laser light source is alternately turned on and off when the camera takes pictures, and adjacent images can successively form the first image and the corresponding second image. In a preferred embodiment, the first image can be an image obtained under the projection of an 808 nm infrared laser.

[0127] However, it can be conceived that the retroreflective brightness coefficient determination method of the embodiments of the present application is not limited thereto. For example, the retroreflective brightness coefficient determination method of the embodiments of the present application can be executed non-real-time using stored images.

[0128] S420: Identify the first marking area in the first image using a given image segmentation algorithm.

[0129] S430: Identify the second marking area in the second image using a given image segmentation algorithm.

[0130] In the embodiments of the present application, especially in steps S420 and S430, the image segmentation algorithm is an artificial intelligence image segmentation algorithm based on machine learning or deep learning, especially an end-to-end image segmentation algorithm capable of identifying the marking area in real time. By accurately identifying and calibrating the marking detection area, it facilitates the rapid and accurate determination of the retroreflective brightness coefficient of the marking. By way of explanation and not limitation, compared with the solution of determining the retroreflective brightness coefficient by filtering the entire image to obtain the gray value, the accuracy of extracting the gray value of the marking area after accurately identifying the marking area is higher, and the accuracy of the corresponding determined retroreflective brightness coefficient is also higher.

[0131] In a preferred embodiment, in the embodiments of the present application, especially in steps S420 and S430, the marking identification can be processed using a marking recognition model based on the YOLO image segmentation algorithm. More preferably, it can be processed using a marking recognition model based on the improved YOLOv8n-seg image segmentation algorithm. As Figures 6 to 8As shown, the marking recognition model based on the YOLO image segmentation algorithm includes a backbone network 620, a neck network 630, and multiple segment heads 640 for processing feature maps of different sizes.

[0132] Correspondingly, in a preferred embodiment, in the embodiments of the present application, especially in steps S420 and S430, the marking area recognition includes: an input image 610 - here an image related to the marking (such as the first image or the second image) is input into the marking recognition model based on the YOLO image segmentation algorithm for marking recognition processing, so as to identify the marking area in the image. More specifically, with reference to Figure 5 and Figures 6 - 8 , the input image 610 is sequentially processed by the backbone network 620, the neck network 630, and multiple segment heads 641, 642, 643, 644.

[0133] As Figure 6 and Figure 7 shown, the backbone network 620 can be used to extract multiple feature maps from the input image 610 and output multiple feature maps to the neck network 630 through multiple feature channels. As Figure 6 and Figure 7 shown, the backbone network 620 includes multiple feature extraction convolution modules 621, 622, 623, 624 corresponding to multiple feature channels respectively. As Figure 7 Specifically shown, in addition to the multiple feature extraction convolution modules 621, 622, 623, 624 of the above multiple feature channels, the backbone network 620 may further include additional layers or modules, which will not be elaborated here.

[0134] As Figure 6 and Figure 7 shown, the neck network 630 may include a bottom-up and top-down sampling structure, including multiple necessary neural network layers, such as various layers with sampling functions and the required splicing layers.

[0135] As Figure 6 and Figure 7 shown, after being sampled by the neck network 630, it can correspondingly output the sampled feature maps to the corresponding segment heads 641 to 644. Figure 7In the specifically illustrated embodiments, each segmentation head jointly performs detection and segmentation using a dual loss function, namely, a prediction box loss function (Bbox Loss) and an object classification loss function (ClsLoss).

[0136] In the embodiments of the present application, the (improved) YOLO image segmentation algorithm refers to a YOLO image segmentation algorithm that optimizes lane line recognition using any novel feature of the embodiments of the present application, especially a YOLOv8n-seg image segmentation algorithm that optimizes lane line recognition using any novel feature of the embodiments of the present application. It can be envisioned that the features for lane line identification processing in the embodiments of the present application can be incorporated into or used to improve other end-to-end image segmentation algorithm frameworks, and such novel combinations and improvements may be covered by the present application or subsequent applications.

[0137] In a specific example, a lane line detection device can be configured according to the lane line detection model described above to obtain multiple road images, and they can be divided into a training set, a test set, and a validation set at a certain ratio, such as an 8:1:1 ratio, to verify the recognition results.

[0138] Before or as a preprocessing step in lane line recognition processing, it may also include annotating the images used as the training set. In one example, an image annotation tool (Labelme image label annotation software) can be used to annotate the road lane line images in the training set. To accurately obtain the boundary coordinate points of the road lane lines and the categories of the road lane lines during deep learning, during annotation, a mask with an irregular bounding box is used to mark the targets in the Labelme software, and then the categories of the target road lane lines are created and named, and different categories of targets are marked with masks of different colors.

[0139] In a further embodiment, as Figure 5 shown, using a lane line recognition model based on the YOLO image segmentation algorithm for processing may specifically include steps S510 to S540:

[0140] S510: Use the backbone network to extract features from the input image and output multiple feature maps through multiple feature channels.

[0141] Among them, the multiple feature channels correspond to the multiple segmentation heads.

[0142] S520: Add an attention mechanism to part of the multiple feature channels, thereby applying attention processing to part of the multiple feature maps and not applying attention processing to other parts of the multiple feature maps.

[0143] S530: Use the neck network to perform sampling processing on the multiple feature maps that have undergone partial attention processing, and output the sampled multiple feature maps to the multiple segmentation heads correspondingly;

[0144] S540: Use multiple segmentation heads to process multiple sampled feature maps respectively to segment and extract the marking areas in the multiple feature maps.

[0145] Combined reference Figures 6 to 8 , the multiple segmentation heads may include a large-size segmentation head 641, multiple (such as two) medium-size segmentation heads 642, 643, and a small-size segmentation head 644.

[0146] In this preferred embodiment, a small-size segmentation head, also known as a small-target segmentation head, is added to the marking recognition model based on the improved YOLO image segmentation algorithm.

[0147] Thus, in the above step S510, multiple feature maps are output through multiple feature channels, including: A1: Output the large-size feature map through the first feature channel, A2: Output the multiple medium-size feature maps through the multiple second feature channels respectively, A3: Output the small-size feature map through the third feature channel.

[0148] Furthermore, as Figures 6 to 8 shown, the multiple channels also correspondingly include a first feature channel corresponding to the large-size segmentation head, multiple second feature channels corresponding to the multiple medium-size segmentation heads respectively, and a third feature channel corresponding to the small-size segmentation head.

[0149] Thus, in the above step S540, the corresponding output of the multiple sampled feature maps to the multiple segmentation heads includes: B1: Corresponding output the sampled large-size feature map to the large-size segmentation head, B2: Corresponding output the multiple sampled medium-size feature maps to the multiple medium segmentation heads respectively, B3: Corresponding output the sampled small-size feature map to the small-size segmentation head.

[0150] In the marking recognition model based on the improved YOLO image segmentation algorithm in the embodiments of the present application, after being processed by the backbone network and the neck network, not only large-size feature maps and several medium-size feature maps are output to the corresponding segmentation heads for instance segmentation, but also specifically for the specific scenario of marking recognition, small-size feature maps are output to the corresponding small-size segmentation head (small-target segmentation head) to segment small-target markings. By way of explanation and not limitation, the pixel area scales occupied by road markings captured by the camera at a small angle in the picture vary greatly, and the conventional image segmentation algorithms not specifically modified for marking recognition have weak instance detection capabilities for various small-scale targets. Therefore, it is easy to miss detections for road markings at a relatively long distance from the camera.

[0151] In a preferred embodiment of the present application, the ratio of the large-size feature map corresponding to the large-size segmentation head to the small-size feature map corresponding to the small-size segmentation head is greater than or equal to 8. In one example, an input image with a size of 640*640 can be downsampled by sampling multiples of 20 times (corresponding to the first feature channel of the large-size segmentation head), 40 times, 80 times, and 160 times (corresponding to the third feature channel of the small-size segmentation head) to obtain feature maps with sizes of 32*32, 16*16, 8*8, and 4*4, respectively, which are used for segmentation by multiple feature heads. By means of a segmentation head for a tiny target (as well as the corresponding backbone network and neck network structures, which will be further described below) in the marking recognition model of the improved YOLO image segmentation algorithm, the instance detection ability for small target road markings is effectively improved.

[0152] Combined with reference to Figure 5 and Figures 6 - 8 , in step S530, an attention mechanism, such as a channel attention mechanism (CA), is added to some of the multiple feature channels, so as to apply attention processing to a part of the multiple feature maps and not apply attention processing to other parts of the multiple feature maps. According to the preferred embodiment, the feature map sizes processed by the segmentation heads corresponding to the feature channels to which the attention mechanism is applied are not adjacent. More specifically, combined with reference to Figures 6 to 8 , the above step S520 may correspondingly include: B1: Adding an attention mechanism to the first feature channel and the third channel to apply attention processing to the large-size feature map and the small-size feature map, and not adding an attention mechanism to the second channel to not apply attention processing to the intermediate-size feature map. In other words, only the channel attention mechanism 660 is applied to the first and third feature channels corresponding to the large-size segmentation head and the small-size segmentation head, and the channel attention mechanism is not applied to the second feature channel corresponding to the intermediate-size segmentation head. According to the preferred embodiment, the attention mechanism includes an encoder for embedding the spatial coordinate information of the marking area into the feature and a coordinate attention generation module for generating coordinate attention based on the embedded spatial coordinate information of the marking area.

[0153] The channel attention mechanism is a method of applying weights to the feature maps of each channel to which the attention mechanism is added, so as to improve the importance degree of the channel and the attention information. The applicant has obtained the following surprising discovery through research, that is, adding a partial channel attention mechanism, such as adding the channel attention mechanism non-adjacently, especially applying the channel attention mechanism only to the largest-size and smallest-size feature maps, can avoid various interference factors on the road from interfering with the target information of the road markings to the greatest extent. By way of explanation and not limitation, from bottom to top through the backbone network (such as Figure 6 and Figure 7)Extract deeper features layer by layer. The feature map at the top contains the richest feature information and the vaguest location information, while the feature map at the bottom is the opposite. By applying the channel attention mechanism to the feature channels at both ends, the clearest location information and feature information can be extracted and fused to obtain higher accuracy. In addition, the channel attention mechanism applied in some embodiments of the present application, such as applying the channel attention mechanism non-adjacently, especially only applying the channel attention mechanism to the largest and smallest size feature maps, can also be combined with further preferred features to obtain further beneficial effects for optimizing lane marking recognition.

[0154] Continue to refer to Figures 6 to 8 , as described above, the backbone network 620 includes a plurality of feature extraction convolutional modules 621, 622, 623, 624 corresponding to a plurality of feature channels respectively. In a preferred embodiment of the present application, at least some, preferably all, of the feature extraction convolutional modules corresponding to the feature channels without applying the attention mechanism may include deformable convolution extraction layers, and the feature extraction convolutional modules corresponding to the feature channels with added attention mechanism include fixed convolution extraction layers. As Figure 7 Specifically shown, in the feature extraction convolutional modules 621, 624 corresponding to the first and third feature channels, a C2F (Coarse to Fine) fixed convolution extraction layer may be included; while in the feature extraction convolutional modules 622, 623 corresponding to the second feature channel without adding the attention mechanism, deformable convolution extraction layers 6221 and 6231 may be included. In a preferred embodiment, the deformable convolution extraction layer may include a DCNv3 deformable convolution layer. Specifically refer to Figure 8 , which shows a DCNv3 deformable convolution functional layer in the deformable convolution extraction layer 6221. In the embodiments of the present application, it will be understood that the convolutional extraction layer can be broadly interpreted, which may include functional convolutional (sub) layers with extraction functions, and may also include additional (sub) layers such as batch processing layers, bottleneck layers, splicing layers, etc. as needed. The embodiments of the present application do not limit the different numbers and structures of additional (sub) layers in the convolutional extraction layer.

[0155] When collecting road lane markings, due to operations such as turning and lane changing of the road lane marking collection vehicle, relatively serious shape distortions will occur to the road lane markings, and the angle of the road lane markings in the image will also change unknown. The applicant has obtained the following surprising discovery through research. By making at least some of the feature extraction convolutional modules corresponding to the feature channels without adding the attention mechanism include variable convolution extraction layers, the features of these shape-distorted road lane markings can be effectively and accurately extracted.

[0156] As described above, as Figure 7As shown, each segmentation head uses a dual loss function for joint detection and segmentation, namely the bounding box loss function (Bbox Loss) and the object classification loss function (Cls Loss). In the embodiments of the present application, the bounding box loss function (Bbox Loss) is further improved to optimize the marking line recognition. In a preferred embodiment, before performing the marking line recognition process, the related method may include a training step, specifically including: C1: Inputting a training image containing marked marking lines into a marking line recognition model framework to be trained to obtain the trained marking line recognition model, where the training image is provided with a ground truth related to the marked marking lines, and the ground truth includes the ground truth bounding box of the marked marking lines; C2: The training includes iteratively executing the following steps until a preset training completion condition is reached: C11: Inputting a training sample into the marking line recognition model framework to obtain a predicted value related to the marked marking lines, where the predicted value includes the predicted bounding box of the marked marking lines; C22: Based on a given loss function, calculating the loss value between the predicted value and the ground truth, where the given loss function includes the multi-point distance intersection over union (MPDIOU) loss function, and the calculating the loss value between the predicted value and the ground truth includes: C221: Determining the area intersection over union of the predicted bounding box and the ground truth bounding box; C222: According to a given plurality of predicted bounding box key points, determining the distance between the key points of the predicted bounding box and the corresponding key points of the ground truth bounding box; C223: Determining the multi-point distance intersection over union loss value according to the area intersection over union and the distance; C23: Based on the loss value, reversely updating the parameters of the marking line recognition model framework.

[0157] In this preferred embodiment, the plurality of predicted bounding box key points include diagonal points. The selection of the bounding box diagonal points (the bounding box diagonal line) located on the diagonal line of the bounding box is determined according to the direction and / or shape distortion of the marked marking lines.

[0158] In some known solutions, the bounding box loss function Box_Loss = DFL_Loss + CIOU_Loss is used to measure the overlap degree between the predicted bounding box and the ground truth bounding box. Where DFL_Loss is the depth feature loss function, and the CIOU_Loss function is the complete intersection over union loss function, which uses the distance ratio between the "ground truth bounding box" and the "predicted bounding box" to measure the overlap degree between the predicted bounding box and the ground truth bounding box. However, the applicant has found through research that this loss function cannot well improve the loss value during the marking line recognition (training).

[0159] In response to this, the embodiments of the present application propose to use the multi-point distance intersection over union (MPDIOU) loss function, and preferably, use the bounding box diagonal points as the key points of the multi-point distance intersection over union (MPDIOU) loss function.

[0160] According to a more preferred embodiment, the selection of the diagonal points of the prediction box (box diagonal line) located on the diagonal of the box is determined according to the direction of the marked line and / or the shape distortion. In one example, for a line turning to the right, the upper right and lower left diagonal points of the prediction box can be used as key points. In one example, for a line turning to the left, the upper left and lower right diagonal points of the prediction box can be used as key points. In one example, for a straight line in an image obtained when turning right (and thus causing shape distortion), the upper right and lower left box diagonal points can be used as key points. In one example, for a straight line in an image obtained when turning left (and thus causing shape distortion), the upper left and lower right box diagonal points can be used as key points.

[0161] By way of explanation and not limitation, in the case where the aspect ratios of the prediction box and the ground truth box are the same but the width and height values are different, it is difficult to improve the loss value for line recognition; the embodiment of the present application uses the mean pixel distance intersection over union (MPDIOU) loss function, and uses the box diagonal points as key points, which can accurately segment road markings under the interference of vehicle and pedestrian occlusion, etc.; furthermore, the embodiment of the present application also determines the direction of the box diagonal points (box diagonal line) according to the direction of the marked line and / or the shape distortion, further improving the accurate detection ability of the markings.

[0162] In one embodiment, the accuracy, recall rate, and precision of line detection and segmentation in the image validation set are analyzed. The test results show that the model according to the embodiment of the present application can not only meet the end-to-end real-time line recognition, but also achieve an accuracy rate of 98.4% in the detection category, a recall rate of 94.7%, and a precision of 96.7%; in terms of segmentation, the accuracy rate reaches 98.3%, the recall rate reaches 94.2%, and the precision reaches 96.4%. The model realizes the accurate detection and segmentation of road markings.

[0163] However, it should be understood that other image segmentation algorithms can also be used to obtain new embodiments.

[0164] S440: Extract the first grayscale value of the first marking area in the first image.

[0165] S450: Extract the second grayscale value of the second marking area in the second image.

[0166] In steps S440 and S450, the grayscale value of the marking area can be extracted in various ways, such as but not limited to taking the average, taking the median, etc., and the present application does not limit this. As shown above, the embodiment of the present application extracts the grayscale value of the accurately segmented marking area, rather than extracting the grayscale value from the entire image through threshold screening, improving the accuracy of the calculated retroreflective brightness coefficient.

[0167] S460: Calculate the retroreflective luminance coefficient of the marking line at the detected position according to the first grayscale value, the second grayscale value, and a given standard value of the retroreflective luminance coefficient.

[0168] In some embodiments, as Figure 9 shown, this step S460 may include steps S461 to S464:

[0169] S461: Determine the grayscale difference of the marking line between the first grayscale value and the second grayscale value;

[0170] S462: Obtain the standard value of the retroreflective luminance coefficient corresponding to a given marking standard plate;

[0171] S463: Obtain the standard grayscale difference corresponding to a given marking standard plate;

[0172] In some embodiments, as Figure 10 shown, this step S463 may include steps S4631 to S4636:

[0173] S4631: Obtain the third image and the fourth image corresponding to a given marking standard plate, where the third image is an image collected under the condition of single-wavelength laser projection, and the fourth image is an image collected under the condition of no single-wavelength laser projection;

[0174] S4632: Use a given image segmentation algorithm to identify the third marking area in the third image;

[0175] S4633: Use a given image segmentation algorithm to identify the fourth marking area in the fourth image;

[0176] S4634: Extract the third grayscale value of the third marking area in the third image;

[0177] S4635: Extract the fourth grayscale value of the fourth marking area in the fourth image;

[0178] S4636: Determine the standard grayscale difference between the third grayscale value and the fourth grayscale value.

[0179] In Figure 9 and Figure 10 the embodiments shown, similar or different methods similar to those described in steps S420 and S430 may be used to identify the third and fourth marking areas, and similar or different methods similar to those described in steps S440 and S450 may also be used to extract the third and fourth grayscale values.

[0180] In addition, it will be understood that in this step S463, different methods may also be used to obtain the standard grayscale difference corresponding to the given marking standard plate. For example, the known or stored standard grayscale difference may be obtained by looking up a table.

[0181] S464: Determine the retroreflective brightness coefficient of the marking line at the detected position according to the ratio relationship between the marking line gray-scale difference, the standard gray-scale difference, and the standard value of the retroreflective brightness coefficient.

[0182] In an embodiment of the present application, when calculating or after calculating the retroreflective brightness coefficient of the marking line at the detected position, the method further includes a correction step S470 (not shown): correcting the retroreflective brightness coefficient of the marking line to obtain a corrected retroreflective brightness coefficient.

[0183] More specifically, as Figure 11 shown, the correction may include distance correction. Specifically, the correction step S470 may include:

[0184] S471: Obtain the calibrated projection distance and the actual projection distance of the single-wavelength laser projection.

[0185] S472: Determine a distance correction coefficient according to the calibrated projection distance and the actual projection distance.

[0186] S473: Correct the retroreflective brightness coefficient of the marking line according to the distance correction coefficient to obtain a distance-corrected retroreflective brightness coefficient.

[0187] In some embodiments, determining the distance correction coefficient according to the calibrated projection distance and the actual projection distance includes:

[0188] Determine the distance correction coefficient K according to the following formula (3) D :

[0189] K D = D 2 / D N 2 (3)

[0190] where D is the actual projection distance, and D N is the calibrated projection distance.

[0191] In some embodiments, the correction may include ambient light correction. The correction step S470 may include: correcting the retroreflective brightness coefficient of the marking line to obtain an ambient light-corrected retroreflective brightness coefficient, specifically including: D1: Obtain the ambient light parameters when obtaining the marking line detection data; D2: Determine an ambient light correction coefficient according to the ambient light parameters, the marking line gray-scale difference, and the standard value of the retroreflective brightness coefficient of the marking line at the detected position; D3: Correct the retroreflective brightness coefficient according to the ambient light correction coefficient to obtain an ambient light-corrected retroreflective brightness coefficient.

[0192] As described above, the calibration process is optional, and in some embodiments, the calibration process, such as the calibration processes described in steps S471, S472, S473 and D1, D2, D3 above, can be combined during the calculation of the retroreflective brightness coefficient, or can be processed separately after the calculation of the (original) retroreflective brightness coefficient. In an illustration, the calibration during the calculation of the retroreflective brightness coefficient will be described:

[0193] In this example, for instance, the detection device of the embodiments of the present application can be used to detect the marked line standard plate to be tested (for the sake of convenience of distinction, the detected marked line standard plate is regarded as the marked line for the time being) and determine that the gray values of the marked line areas of the first image and the second image are B1 and B2, then the marked line gray difference B = B1 - B2.

[0194] Retroreflective brightness coefficient without distance / environment calibration: Assume that the standard value of the retroreflective brightness coefficient is RL s = 150 mcd·m-2·lx-1, and its gray difference is the standard B 150 , then the retroreflective brightness coefficient without distance / environment calibration is RL raw = B * RL s / B 150 . Since in this example, the detection is performed using the marked line standard plate, this process can also be conveniently called standard plate calibration, but it will be understood that when processing the real marked line, this process can correspond to determining the (distance - and environment - uncalibrated) retroreflective brightness coefficient of the marked line at the detected position according to steps S461 to S464 above.

[0195] Distance calibration: As described before, K D = D 2 / D N 2 .

[0196] Ambient light calibration: The applicant has found that especially in the natural light environment, there is still some ambient light remaining in the gray difference B. To completely eliminate the influence of the ambient light, the calibration coefficient K E is introduced. Specifically, the correlation relationship between the ambient illuminance E, the gray difference B, and the true value RL* of the retroreflective brightness coefficient can be obtained and analyzed. This correlation relationship can be characterized by the model / algorithm f(E), and the calibration coefficient K E = f(E).

[0197] Thus, in this example, the method for determining the retroreflective brightness coefficient of the marked line based on laser irradiation can determine that the calibrated retroreflective brightness coefficient is RL = K D * K E * RL raw = D 2 / D N 2*f(E)*(B1 - B2)*RL s / B 150 。

[0198] In this example, the calibration coefficient K E = f(E) can be dynamically determined based on the acquired ambient illuminance E. By introducing this calibration coefficient K E the dynamic detection and calibration of the retroreflective brightness coefficient are further realized, and the influence of ambient light is further eliminated advantageously.

[0199] Combined with reference Figure 1 and Figure 5 before acquiring the first image and the second image, the method further includes: S400 (not shown): constructing a marking line detection model based on a proportional relationship. As Figure 12 shown, step S400 may specifically include:

[0200] S401: determining the calibration height parameters for the marking line visibility observation model, where the calibration height parameters include the calibration driver observation height and the calibration vehicle light height.

[0201] S402: determining the calibration observation distance for the marking line visibility observation model.

[0202] S403: setting the calibration projection distance for the light projection used for marking line detection, and based on the proportional relationship between the calibration observation distance and the calibration projection distance, determining the marking line detection height parameters for marking line detection from the calibration height parameters.

[0203] Wherein, the marking line detection height parameters include the camera height for image acquisition and the light source height for light projection.

[0204] As an alternative to step S403, S403' (not shown): at least partially setting the marking line detection height parameters for marking line detection, and based on the proportional relationship between the calibration height parameters and at least partially set marking line detection height parameters, determining the calibration projection distance for the light projection used for marking line detection from the calibration observation distance.

[0205] Wherein, the first image and the second image are acquired at the calibration projection distance, camera height, and light projection height.

[0206] The method for determining the retroreflective brightness coefficient according to one aspect of the present application is described above in conjunction with the accompanying drawings. It can process two types of images with or without single - wavelength laser projection, and accurately segment the marking line area using an image segmentation algorithm for each of these two types of images; and as Figure 12As shown, a preferred embodiment of the retroreflective brightness coefficient determination method for this aspect can further be based on the proportional relationship to construct the processing of the marking detection model. However, it will be understood that on the other hand of the present application, a retroreflective brightness coefficient determination method for constructing the marking detection model can be provided, in which the retroreflective brightness coefficient of the marking can be determined based on or not based on the laser projection image and / or image segmentation features. For example, the retroreflective brightness coefficient determination method for this aspect can determine the retroreflective brightness coefficient based on the ordinary light projection image. Accordingly, as Figure 13 shown, an embodiment of the present application also provides a retroreflective brightness coefficient determination method, which is characterized by including:

[0207] S1310: Determine the calibration height parameters for the marking visibility observation model, where the calibration height parameters include the calibrated driver observation height and the calibrated vehicle light height.

[0208] S1320: Determine the calibrated observation distance for the marking visibility observation model.

[0209] S1330: Set the calibrated projection distance for the light projection used for marking detection, and based on the proportional relationship between the calibrated observation distance and the calibrated projection distance, determine the marking detection height parameters for marking detection from the calibration height parameters.

[0210] The marking detection height parameters include the camera height and the light projection height for image acquisition

[0211] S1330’ (not shown): At least partially set the marking detection height parameters, and based on the proportional relationship between the calibration height parameters and at least partially set marking detection height parameters, determine the calibrated projection distance for the light projection used for marking detection from the calibrated observation distance;

[0212] S1340: Obtain the marking detection data corresponding to the detected position in the marking detection.

[0213] Wherein, the image in the marking detection data is acquired at the calibrated projection distance, the camera height, and the light projection height;

[0214] S1350: Determine the retroreflective brightness coefficient according to the marking detection data.

[0215] In summary, the embodiments of the present application provide the following several innovative invention themes, and any invention theme can be optionally combined or not combined with other innovative themes or their features:

[0216] (1) The embodiments of the present application provide a marking detection device, the marking detection data of which includes two-mode pictures with and without single-wavelength laser projection, so that the data detected by the marking detection device can be particularly beneficial for accurately calculating the retroreflective brightness coefficient of the marking.

[0217] Those skilled in the art will understand that the detected marking data obtained can be not limited to determining the retroreflective brightness coefficient of the marking, or can be additionally used for other purposes of marking detection / recognition, such as autonomous driving, etc. Those skilled in the art will understand that the marking detection device and the retroreflective brightness coefficient determination scheme can be provided or implemented by the same or different entities. For example, the marking detection device is provided in the vehicle, while the retroreflective brightness coefficient determination scheme can be implemented by other electronic devices that belong to or do not belong to the vehicle.

[0218] (2) The embodiments of the present application provide a retroreflective brightness coefficient determination scheme in one aspect, which not only processes two types of images with or without single-wavelength laser projection, but also accurately segments the marking area by using an image segmentation algorithm for each of these two types of images. By extracting the gray values from the marking area segmented from the marking image and combining them with the standard values of the retroreflective brightness coefficient of the standard plate, this method can accurately determine the retroreflective brightness coefficient of the object to be measured.

[0219] (3) The embodiments of the present application provide a retroreflective brightness coefficient determination scheme in another aspect, which can construct a marking detection model based on light projection in proportion to the marking visibility observation model for determining the retroreflective brightness coefficient. While reducing the measurement complexity and the limitations on the measurement equipment, this method effectively ensures the accuracy of the detection result, and further ensures the accuracy of the determined retroreflective brightness coefficient.

[0220] Here, the retroreflective brightness coefficient determination scheme or its features in the first aspect can be combined with the retroreflective brightness coefficient determination scheme in the second aspect.

[0221] (4) The embodiments of the present application provide a marking identification method especially based on artificial intelligence, which can efficiently (in real time, end-to-end) and accurately (especially avoiding occlusion and shape distortion) identify markings. The marking recognition method provided by the embodiments of the present application can be advantageously combined with the retroreflective brightness coefficient determination scheme in the first aspect to further ensure the accurate and rapid determination of the retroreflective brightness coefficient. However, this marking identification method can be not limited to being combined with the scheme for determining the retroreflective brightness coefficient of the marking, and can be additionally used for other purposes of marking detection / recognition, such as autonomous driving, etc.

[0222] (5) The embodiments of the present application provide a method for correcting the retroreflective brightness coefficient of markings based on distance and / or ambient light correction, which solves the problems of strong light interference and dynamic detection angle change, and further ensures the high precision of the retroreflective brightness coefficient. Here, the correction method can cover various variants of only distance correction, only ambient light correction, and both corrections.

[0223] In addition, the calibration method provided by the embodiments of the present application can be advantageously incorporated into the retroreflective brightness coefficient determination scheme and / or the marking line detection device of the embodiments of the present application, further ensuring the accurate determination of the retroreflective brightness coefficient. However, the calibration method provided by the embodiments of the present application can be used to calibrate the retroreflective brightness coefficients obtained by other methods. For example, a known retroreflective brightness coefficient measuring device with ordinary light projection can be modified by adding a distance sensing unit and / or an ambient light sensing unit to determine the actual distance of light projection and / or the ambient light during image acquisition, so as to more accurately calibrate the value of the retroreflective brightness coefficient determined by the known measuring device during the subsequent determination process of the retroreflective brightness coefficient.

[0224] As an example, referring to Figure 14 , a retroreflective brightness coefficient evaluation device 1400 of the embodiments of the present application is also shown, which may include: an acquisition unit 1401, a first marking line recognition unit 1402, a second marking line recognition unit 1403, a first image processing unit 1404, a second image processing unit 1405, and a retroreflective brightness coefficient calculation unit 1407, where:

[0225] The acquisition unit 1401 is configured to acquire a first image and a second image corresponding to the detected position in the marking line detection, where the first image is an image acquired under the condition of single-wavelength laser projection, and the second image is an image acquired under the condition of no single-wavelength laser projection.

[0226] The first marking line recognition unit 1402 is configured to identify the first marking line area in the first image by using a given image segmentation algorithm.

[0227] The second marking line recognition unit 1403 is configured to identify the second marking line area in the second image by using a given image segmentation algorithm.

[0228] The first image processing unit 1404 is configured to extract the first gray value of the first marking line area in the first image.

[0229] The second image processing unit 1405 is configured to extract the second gray value of the second marking line area in the second image.

[0230] The retroreflective brightness coefficient calculation unit 1407 is configured to calculate the retroreflective brightness coefficient of the marking line at the detected position according to the first gray value, the second gray value, and a given standard value of the retroreflective brightness coefficient.

[0231] In some embodiments of the application, the acquisition unit 1401 is further configured to acquire the ambient light parameter value corresponding to the detected position in the marking line detection; and is configured to acquire the actual projection distance of the single-wavelength laser corresponding to the detected position in the marking line detection.

[0232] In this embodiment, referring to Figure 14 , the retroreflective brightness coefficient inspection and evaluation device 1400 may further include a third processing unit 1406, wherein the third processing unit 1406 is configured to extract the ambient light correction coefficient and the distance correction coefficient of the marking line at the detected position.

[0233] In this embodiment, the retroreflective brightness coefficient calculation unit 1407 is further configured to calculate the retroreflective brightness coefficient of the marking line at the detected position according to the first gray value, the second gray value, the ambient light correction coefficient, the distance correction coefficient, and a given standard value of the retroreflective brightness coefficient.

[0234] In some embodiments of the application, referring to Figure 14 , the retroreflective brightness coefficient inspection and evaluation device 1400 may further include a retroreflective brightness coefficient evaluation unit 1408, wherein the retroreflective brightness coefficient evaluation unit 1408 is configured to receive the retroreflective brightness coefficient calculated by the calculation unit, and judge the retroreflective performance of the marking line according to a preset performance index threshold to evaluate whether it meets the specified safety standard.

[0235] Under the teaching of the present application, the features of the method embodiments can be combined into the device embodiments in a non-contradictory manner to obtain new embodiments, and the features of the device embodiments can also be combined into the method embodiments in a non-contradictory manner to obtain new embodiments, which fall within the scope of the present application.

[0236] In the embodiments of the present application, the method and device for determining the retroreflective brightness coefficient may be implemented by a computer integrating relevant functional modules or components. Figure 15 The structural diagram of a computer that can be used to implement the method and device for determining the retroreflective brightness coefficient according to the embodiments of the present application is shown.

[0237] As Figure 15As shown, the computer 1500 includes a processor 1501, which can perform various appropriate operations and processes according to programs and / or data stored in a read-only memory (ROM) 1502 or programs and / or data loaded into a random access memory (RAM) 1503 from a storage section 1508. The processor 1501 may include a central processing unit (CPU), and can be a multi-core processor or may include multiple processors. In some embodiments, the processor 1501 may include a general-purpose main processor and one or more special coprocessors, such as a graphics processing unit (GPU), a neural network processing unit (NPU), a digital signal processor (DSP), and so on. In the RAM 1503, various programs and data required for the operation of the electronic device 1500 are also stored. The processor 1501, the ROM 1502, and the RAM 1503 are connected to each other via a bus 1504. An input / output (I / O) interface 1505 is also connected to the bus 1504.

[0238] The above-mentioned processor and memory are jointly used to execute the program stored in the memory, and when the program is executed by the computer, it can implement the steps or functions of the methods described in the above embodiments.

[0239] The following components are connected to the I / O interface 1505: an input section 1506 including a keyboard, a mouse, a touch screen, etc.; an output section 1507 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 1508 including a hard disk, etc.; and a communication section 1509 including a network interface card such as a LAN card, a modem, etc. The communication section 1509 performs communication processing via a network such as the Internet. A drive 1510 is also connected to the I / O interface 1505 as needed. A removable medium 1511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1510 as needed so that a computer program read from it can be installed into the storage section 1508 as needed. Figure 15 Only some components are schematically shown, and it does not mean that the computer system 1500 only includes Figure 15 the components shown.

[0240] In some embodiments, the computer 1500 refers to a mobile terminal, including a mobile phone, a vehicle-mounted terminal, etc. Taking a mobile phone as an example, the electronic device 1500 further includes device modules such as a display screen with a touch function, an external speaker, a gyroscope, a camera, a 4G / 5G antenna, etc.

[0241] The systems, devices, modules or units illustrated in the above embodiments can be implemented by this computer or its associated components. The computer can be, for example, a mobile terminal, a personal computer, a laptop computer, an in-vehicle human-machine interaction device, a personal digital assistant, a media player, a navigation device, a tablet computer, or a combination thereof.

[0242] Although not shown, in an embodiment of the present application, a program product is provided, including a computer program, wherein when the computer program is executed by a processor, the method described in any embodiment of the present application is implemented.

[0243] Although not shown, in an embodiment of the present application, a storage medium is provided, the storage medium stores a computer program, and the computer program is configured to execute the method described in any embodiment of the present application when being run.

[0244] The storage medium in the embodiments of the present application includes permanent and non-permanent, removable and non-removable articles that can implement information storage by any method or technology. Examples of the storage medium include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0245] The methods, programs, systems, devices, etc. in the embodiments of the present application can be executed or implemented in a single or multiple networked computers, and can also be practiced in a distributed computing environment. In the embodiments of this specification, in these distributed computing environments, tasks can be executed by remote processing devices connected through a communication network.

[0246] Unless explicitly stated, the actions or steps of the methods and programs described according to the embodiments of the present application do not necessarily have to be executed in a specific order and can still achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0247] In this document, multiple embodiments of the present application are described. However, for the sake of brevity, the descriptions of each embodiment are not exhaustive, and the same or similar features or parts between various embodiments may be omitted. In this document, "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean applicable to at least one embodiment or example according to the present application, rather than all embodiments. The above terms do not necessarily refer to the same embodiment or example. Without contradiction, those skilled in the art can combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples.

[0248] The exemplary systems and methods of the present application have been specifically shown and described with reference to the above embodiments, which are only examples of the preferred modes for implementing the systems and methods. Those skilled in the art can understand that various changes can be made to the embodiments of the systems and methods described here when implementing the systems and / or methods without departing from the spirit and scope of the present application defined in the appended claims.

Claims

1. A method for determining a retroreflective brightness coefficient, characterized in that: include: Acquire a first image and a second image corresponding to a detected position in the marking line detection, wherein the first image is an image acquired when a single-wavelength laser is projected, and the second image is an image acquired when the single-wavelength laser is not projected; Identify a first marked line region in the first image using a given image segmentation algorithm; Identifying a second reticle region in the second image using a given image segmentation algorithm; Extracting a first grayscale value of a first marking line area in the first image; Extracting a second grayscale value of a second marking line area in the second image; The retroreflective brightness coefficient of the marking line at the detected position is calculated according to the marking line grayscale difference determined by the first grayscale value and the second grayscale value and a given retroreflective brightness coefficient standard value.

2. The method for determining the retroreflective brightness coefficient according to claim 1, characterized in that: The step of calculating the retroreflective brightness coefficient of the marking line at the detected position according to the grayscale difference of the marking line determined by the first grayscale value and the second grayscale value and a given retroreflective brightness coefficient standard value comprises: Obtaining the standard value of the retroreflective brightness coefficient corresponding to a given marking standard plate; Obtaining a standard grayscale difference corresponding to the given graticule standard plate; The retroreflective brightness coefficient of the marking line at the detected position is determined according to the proportional relationship between the marking line grayscale difference, the standard grayscale difference and the standard value of the retroreflective brightness coefficient.

3. The method for determining the retroreflective brightness coefficient according to claim 2, characterized in that: During or after calculating the retroreflective brightness coefficient of the marking line at the detected position, the method further includes: Correcting the retroreflective brightness coefficient of the marking to obtain the retroreflective brightness coefficient corrected for ambient light includes: Acquiring ambient light parameters in the marking line detection; Determine an ambient light correction coefficient according to the ambient light parameter, the grayscale difference of the marking line, and a standard value of the retroreflective brightness coefficient of the marking line at the detected position; The retroreflection brightness coefficient is corrected according to the ambient light correction coefficient to obtain the retroreflection brightness coefficient corrected for ambient light.

4. The method for determining the retroreflective brightness coefficient according to claim 2, characterized in that: The step of obtaining a standard grayscale difference corresponding to the given reticle standard plate includes: Acquire a third image and a fourth image corresponding to the given reticle standard plate, wherein the third image is an image acquired when a single-wavelength laser is projected, and the fourth image is an image acquired when a single-wavelength laser is not projected; identifying a third marking line region in the third image using a given image segmentation algorithm; identifying a fourth marking line region in the fourth image using a given image segmentation algorithm; Extracting a third grayscale value of a third marking line area in the third image; Extracting a fourth grayscale value of a fourth marking line area in the fourth image; The standard grayscale difference between the third grayscale value and the fourth grayscale value is determined.

5. The method for determining the retroreflective brightness coefficient according to any one of claims 1 to 3, characterized in that: When or after calculating the retroreflective brightness coefficient of the marking line at the detected position, the method further includes: Correcting the retroreflective brightness coefficient of the marking to obtain a distance-corrected retroreflective brightness coefficient includes: Obtaining a calibrated projection distance and an actual projection distance of the single-wavelength laser projection; Determine a distance correction coefficient according to the calibrated projection distance and the actual projection distance; The retroreflective brightness coefficient of the marking line is corrected according to the distance correction coefficient to obtain a distance-corrected retroreflective brightness coefficient.

6. The method for determining the retroreflective brightness coefficient according to claim 1, characterized in that: When or after calculating the retroreflective brightness coefficient of the marking line at the detected position, the method further includes: Correcting the retroreflective brightness coefficient of the marking to obtain an ambient light corrected and distance corrected retroreflective brightness coefficient includes: Acquiring ambient light parameters in the marking line detection; Determine an ambient light correction coefficient according to the ambient light parameter, the grayscale difference of the marking line, and a standard value of the retroreflective brightness coefficient of the marking line at the detected position; Obtaining a calibrated projection distance and an actual projection distance of the single-wavelength laser projection; Determine a distance correction coefficient according to the calibrated projection distance and the actual projection distance; The retroreflective brightness coefficient of the marking line is corrected according to the ambient light correction coefficient and the distance correction coefficient to obtain an ambient light-corrected and distance-corrected retroreflective brightness coefficient.

7. The method for determining the retroreflective brightness coefficient according to any one of claims 1 to 3, characterized in that: Before acquiring the first image and the second image, the method further includes: Determining calibration height parameters for a road marking visibility observation model, wherein the calibration height parameters include a calibration driver observation height and a calibration vehicle light height; Determining a calibrated viewing distance for a reticle visibility viewing model; Setting a calibrated projection distance of light projection for line detection, and determining a line detection height parameter for line detection from the calibrated height parameter based on a proportional relationship between the calibrated observation distance and the calibrated projection distance, wherein the line detection height parameter includes a camera height for capturing an image and a light source height for projecting light; or, at least partially setting a line detection height parameter for line detection, and determining the calibrated projection distance of light projection for line detection from the calibrated observation distance based on a proportional relationship between the calibrated height parameter and the at least partially set line detection height parameter; The first image and the second image are acquired at the calibrated projection distance, camera height and light projection height.

8. A retroreflective brightness coefficient testing and evaluation device, characterized in that: include: An acquisition unit configured to acquire a first image and a second image corresponding to a detected position in the marking line detection, wherein the first image is an image acquired when a single-wavelength laser is projected, and the second image is an image acquired when the single-wavelength laser is not projected; A first marking line recognition unit configured to identify a first marking line region in the first image using a given image segmentation algorithm; a second marking line recognition unit configured to identify a second marking line region in the second image using a given image segmentation algorithm; a first image processing unit configured to extract a first grayscale value of a first reticle area in the first image; a second image processing unit configured to extract a second grayscale value of a second reticle area in the second image; The retroreflection brightness coefficient calculation unit is configured to calculate the retroreflection brightness coefficient of the marking line at the detected position according to the marking line grayscale difference determined by the first grayscale value and the second grayscale value and a given retroreflection brightness coefficient standard value.

9. The retroreflective brightness coefficient testing and evaluation device according to claim 8, characterized in that: The acquisition unit is configured to acquire the ambient light parameter value corresponding to the detected position in the marking line detection; configured to acquire the actual projection distance of the single-wavelength laser corresponding to the detected position in the marking line detection; The retroreflective brightness coefficient testing and evaluation device also includes: a third processing unit configured to extract an ambient light correction coefficient and a distance correction coefficient of the marking line at the detected position; The retroreflection brightness coefficient calculation unit is configured to calculate the retroreflection brightness coefficient of the marking line at the detected position according to the first grayscale value, the second grayscale value, the ambient light correction coefficient, the distance correction coefficient and a given retroreflection brightness coefficient standard value.

10. The retroreflective brightness coefficient testing and evaluation device according to claim 9, characterized in that: The retroreflective brightness coefficient testing and evaluation device also includes: The retroreflective brightness coefficient evaluation unit is configured to receive the retroreflective brightness coefficient calculated by the calculation unit, and judge the retroreflective performance of the marking line according to a preset performance indicator threshold to evaluate whether it meets the prescribed safety standards.

11. An electronic device, characterized in that: include: A processor and a memory storing a computer program, wherein the processor is configured to implement the method according to any one of claims 1 to 7 when running the computer program.

12. A program product comprising a computer program, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images