Retroreflection brightness coefficient determination method and device, electronic equipment and program product

Abstract

The invention discloses a retroreflection brightness coefficient determination method, which comprises the steps that a first image and a second image corresponding to a detected position in marking detection are acquired, the first image is an image acquired under the condition of single-wavelength laser projection, and the second image is an image acquired under the condition of single-wavelength laser non-projection; identifying a first marking area in the first image by using a given image segmentation algorithm; identifying a second marking area in the second image by using a given image segmentation algorithm; extracting a first gray value of a first marking area in the first image; extracting a second gray value of a second marking area in the second image; and according to the first gray value, the second gray value and a given retro-reflection brightness coefficient standard value, calculating the retro-reflection brightness coefficient of the marked line at the detected position. The invention further discloses a retroreflection brightness coefficient detection and evaluation device, electronic equipment and a program product.

Description

Technical Field

[0001] The present application relates to the field of highway traffic, and in particular to a method and device for determining a retroreflection brightness coefficient, an electronic device, and a program product. Background Art

[0002] Road traffic markings are key traffic safety facilities painted on the road surface, which are designed to channel traffic flow, guide vehicle alignment, and ensure traffic safety. These markings have multiple functions of defining lane boundaries, regulating and managing driver driving behavior, guiding driver vision, indicating and warning road conditions ahead, and clarifying the right of way.

[0003] After being rolled over for a long time, the visibility of the road traffic markings decreases, causing the function of the markings to decline. At this time, maintenance is required. An important criterion for judging whether the road markings need maintenance is the retroreflective brightness coefficient of the road traffic markings, which can also be referred to as the retroreflective brightness coefficient.

[0004] In my country, handheld retroreflection measuring instruments are commonly used to detect the retroreflection brightness coefficient of markings. Static inspection requires road closure, which has a great social impact and inestimable costs. Dynamic inspection is fast and has no impact on traffic, but the inspection is easily affected by strong ambient light and dynamic geometric angle changes, which often involves complex settings and harsh operating conditions, making it impossible to accurately determine the retroreflection brightness coefficient of markings.

[0005] The content of this background technology description is only for facilitating understanding of the relevant technology in this field and is not regarded as an admission of the prior art. Summary of the invention

[0006] Therefore, the embodiments of the present application hope to provide a solution for determining the retroreflective brightness coefficient that can at least partially solve the problems mentioned above.

[0007] In a first aspect, a method for determining a retroreflective brightness coefficient is provided, comprising:

[0008] Acquire a first image and a second image corresponding to a detected position in the marking line detection, wherein the first image is an image acquired when a single-wavelength laser is projected, and the second image is an image acquired when the single-wavelength laser is not projected;

[0009] Identify a first marked line region in the first image using a given image segmentation algorithm;

[0010] Identifying a second reticle region in the second image using a given image segmentation algorithm;

[0011] Extracting a first grayscale value of a first marking line area in the first image;

[0012] Extracting a second grayscale value of a second marking line area in the second image;

[0013] The retroreflective brightness coefficient of the marking line at the detected position is calculated according to the first grayscale value, the second grayscale value and a given retroreflective brightness coefficient standard value.

[0014] Optionally, the calculating the retroreflective brightness coefficient of the marking line at the detected position according to the first grayscale value, the second grayscale value and a given retroreflective brightness coefficient standard value includes:

[0015] Determine a grayscale difference between the first grayscale value and the second grayscale value;

[0016] Obtaining the standard value of the retroreflective brightness coefficient corresponding to a given marking standard plate;

[0017] Obtaining a standard grayscale difference corresponding to the given graticule standard plate;

[0018] The retroreflective brightness coefficient of the marking line at the detected position is determined according to the proportional relationship between the marking line grayscale difference, the standard grayscale difference and the standard value of the retroreflective brightness coefficient.

[0019] Optionally, during or after calculating the retroreflective brightness coefficient of the marking line at the detected position, the method further includes:

[0020] Correcting the retroreflective brightness coefficient of the marking to obtain the retroreflective brightness coefficient corrected for ambient light includes:

[0021] Acquiring ambient light parameters in the marking line detection;

[0022] Determine an ambient light correction coefficient according to the ambient light parameter, the grayscale difference of the marking line, and a standard value of the retroreflective brightness coefficient of the marking line at the detected position;

[0023] The retroreflection brightness coefficient is corrected according to the ambient light correction coefficient to obtain the retroreflection brightness coefficient corrected for ambient light.

[0024] Optionally, the obtaining of a standard grayscale difference corresponding to the given reticle standard plate includes:

[0025] Acquire a third image and a fourth image corresponding to the given reticle standard plate, wherein the third image is an image acquired when a single-wavelength laser is projected, and the fourth image is an image acquired when a single-wavelength laser is not projected;

[0026] identifying a third marking line region in the third image using a given image segmentation algorithm;

[0027] identifying a fourth marking line region in the fourth image using a given image segmentation algorithm;

[0028] Extracting a third grayscale value of a third marking line area in the third image;

[0029] Extracting a fourth grayscale value of a fourth marking line area in the fourth image;

[0030] The standard grayscale difference between the third grayscale value and the fourth grayscale value is determined.

[0031] Optionally, when or after calculating the retroreflective brightness coefficient of the marking line at the detected position, the method further includes:

[0032] Correcting the retroreflective brightness coefficient of the marking to obtain a distance-corrected retroreflective brightness coefficient includes:

[0033] Obtaining a calibrated projection distance and an actual projection distance of the single-wavelength laser projection;

[0034] Determine a distance correction coefficient according to the calibrated projection distance and the actual projection distance;

[0035] The retroreflective brightness coefficient of the marking line is corrected according to the distance correction coefficient to obtain a distance-corrected retroreflective brightness coefficient.

[0036] Optionally, when or after calculating the retroreflective brightness coefficient of the marking line at the detected position, the method further includes:

[0037] Correcting the retroreflective brightness coefficient of the marking to obtain an ambient light corrected and distance corrected retroreflective brightness coefficient includes:

[0038] Acquiring ambient light parameters in the marking line detection;

[0039] Determine an ambient light correction coefficient according to the ambient light parameter, the grayscale difference of the marking line, and a standard value of the retroreflective brightness coefficient of the marking line at the detected position;

[0040] Obtaining a calibrated projection distance and an actual projection distance of the single-wavelength laser projection;

[0041] Determine a distance correction coefficient according to the calibrated projection distance and the actual projection distance;

[0042] The retroreflective brightness coefficient of the marking line is corrected according to the ambient light correction coefficient and the distance correction coefficient to obtain an ambient light-corrected and distance-corrected retroreflective brightness coefficient.

[0043] Optionally, before acquiring the first image and the second image, the method further includes:

[0044] Determining calibration height parameters for a road marking visibility observation model, wherein the calibration height parameters include a calibration driver observation height and a calibration vehicle light height;

[0045] Determining a calibrated viewing distance for a reticle visibility viewing model;

[0046] Setting a calibrated projection distance of light projection for line detection, and determining a line detection height parameter for line detection from the calibrated height parameter based on a proportional relationship between the calibrated observation distance and the calibrated projection distance, wherein the line detection height parameter includes a camera height for capturing an image and a light source height for projecting light; or, at least partially setting a line detection height parameter for line detection, and determining the calibrated projection distance of light projection for line detection from the calibrated observation distance based on a proportional relationship between the calibrated height parameter and the at least partially set line detection height parameter;

[0047] The first image and the second image are acquired at the calibrated projection distance, camera height and light projection height.

[0048] In a second aspect, a method for determining a retroreflective brightness coefficient is provided, comprising:

[0049] Determining calibration height parameters for a road marking visibility observation model, wherein the calibration height parameters include a calibration driver observation height and a calibration vehicle light height;

[0050] Determining a calibrated viewing distance for a reticle visibility viewing model;

[0051] Setting a calibrated projection distance of light projection for marking line detection, and determining a marking line detection height parameter for marking line detection from the calibrated height parameter based on a proportional relationship between the calibrated observation distance and the calibrated projection distance, wherein the marking line detection height parameter includes a camera height for collecting images and a light projection height; or, at least partially setting a marking line detection height parameter, and determining the calibrated projection distance of light projection for marking line detection from the calibrated observation distance based on a proportional relationship between the calibrated height parameter and at least partially set marking line detection height parameters;

[0052] Acquire marking line detection data corresponding to the detected position in the marking line detection, wherein an image in the marking line detection data is acquired at the calibrated projection distance, camera height, and light projection height;

[0053] The retroreflection brightness coefficient is determined according to the marking line detection data.

[0054] In a third aspect, a retroreflective brightness coefficient testing and evaluation device is provided, comprising:

[0055] An acquisition unit configured to acquire a first image and a second image corresponding to a detected position in the marking line detection, wherein the first image is an image acquired when a single-wavelength laser is projected, and the second image is an image acquired when the single-wavelength laser is not projected;

[0056] A first marking line recognition unit configured to identify a first marking line region in the first image using a given image segmentation algorithm;

[0057] a second marking line recognition unit configured to identify a second marking line region in the second image using a given image segmentation algorithm;

[0058] a first image processing unit configured to extract a first grayscale value of a first reticle area in the first image;

[0059] a second image processing unit configured to extract a second grayscale value of a second reticle area in the second image;

[0060] The retroreflection brightness coefficient calculation unit is configured to calculate the retroreflection brightness coefficient of the marking line at the detected position according to the first grayscale value, the second grayscale value and a given retroreflection brightness coefficient standard value.

[0061] Optionally, the acquisition unit is configured to acquire an ambient light parameter value corresponding to a detected position in the marking line detection; configured to acquire an actual projection distance of a single-wavelength laser corresponding to a detected position in the marking line detection;

[0062] The retroreflective brightness coefficient testing and evaluation device also includes:

[0063] a third processing unit configured to extract an ambient light correction coefficient and a distance correction coefficient of the marking line at the detected position;

[0064] The retroreflection brightness coefficient calculation unit is configured to calculate the retroreflection brightness coefficient of the marking line at the detected position according to the first grayscale value, the second grayscale value, the ambient light correction coefficient, the distance correction coefficient and a given retroreflection brightness coefficient standard value.

[0065] Optionally, the retroreflective brightness coefficient testing and evaluation device further comprises:

[0066] The retroreflective brightness coefficient evaluation unit is configured to receive the retroreflective brightness coefficient calculated by the calculation unit, and judge the retroreflective performance of the marking line according to a preset performance indicator threshold to evaluate whether it meets the prescribed safety standards.

[0067] In a fourth aspect, an electronic device is provided, characterized in that it includes: a processor and a memory storing a computer program, and the processor is configured to implement the method of the embodiment of the present application when running the computer program.

[0068] In a fifth aspect, a program product is provided, comprising a computer program, wherein the computer program implements the method of the embodiment of the present application when executed by a processor.

[0069] One aspect of the present application provides a solution for determining the retroreflective brightness coefficient, which not only processes two images with or without single-wavelength laser projection, but also uses an image segmentation algorithm to accurately segment the marking area for the two images. By extracting the grayscale value from the segmented marking area in the marking image and combining it with the ambient light parameters, the actual projection distance of the light, and the standard value of the retroreflective brightness coefficient of the standard plate, the method can accurately determine the retroreflective brightness coefficient of the object to be measured.

[0070] On the other hand, the present application provides a solution for determining the retroreflection brightness coefficient, which can construct a marking line detection model based on light projection in proportion to the marking line visibility observation model, for determining the retroreflection brightness coefficient. This method effectively ensures the accuracy of the detection results while reducing the measurement complexity and the limitation on the measurement equipment, thereby ensuring the accuracy of the determined retroreflection brightness coefficient.

[0071] The optional features and other effects of the embodiments of the present application are partially described below, and partially can be understood by reading this document. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Hereinafter, embodiments of the present application will be described in detail in conjunction with the accompanying drawings. The elements shown are not limited to the proportions shown in the drawings and the same or similar reference numerals in the drawings represent the same or similar elements, wherein:

[0073] Figure 1 A schematic diagram of a marking line detection device according to an embodiment of the present application is shown, and the marking line detection device can obtain marking line detection data for use in a retroreflective brightness coefficient determination scheme according to an embodiment of the present application;

[0074] Figure 2 An example diagram showing an image processed by the retroreflective brightness coefficient determination scheme according to an embodiment of the present application;

[0075] Figure 3 An example diagram showing an image processed by the retroreflective brightness coefficient determination scheme according to an embodiment of the present application;

[0076] Figure 4 An exemplary flow chart of a method for determining a retroreflective brightness coefficient according to an embodiment of the present application is shown;

[0077] Figure 5 An exemplary flow chart of marking line identification processing of the method for determining retroreflective brightness coefficient according to an embodiment of the present application is shown;

[0078] Figure 6shows that it can be used to implement Figure 5 A schematic structural diagram of a line marking model for line marking processing;

[0079] Figure 7 Shows Figure 6 Another schematic structural diagram of the marking line identification model shown;

[0080] Figure 8 Shows Figure 6 Schematic diagram of the structure of the feature extraction convolution module of the marking line recognition model shown;

[0081] Fig. 9 An exemplary flow chart of a method for determining a retroreflective brightness coefficient according to an embodiment of the present application is shown;

[0082] Fig.10 An exemplary flow chart of a method for determining a retroreflective brightness coefficient according to an embodiment of the present application is shown;

[0083] Fig.11 An exemplary flow chart showing a correction process of a retroreflection brightness coefficient of a method for determining a retroreflection brightness coefficient according to an embodiment of the present application;

[0084] Fig.12 An exemplary flow chart of a method for determining a retroreflective brightness coefficient according to an embodiment of the present application is shown;

[0085] Fig.13 An exemplary flow chart of a method for determining a retroreflective brightness coefficient according to an embodiment of the present application is shown;

[0086] Fig.14 A schematic diagram showing a retroreflective brightness coefficient testing and evaluation device according to an embodiment of the present application; and

[0087] Fig.15 A structural diagram of an electronic device that can be used to implement the method for determining the retroreflective brightness coefficient according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0088] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with specific implementation methods and drawings. Here, the illustrative implementation methods and descriptions of the present application are used to explain the present application, but are not intended to limit the present application.

[0089] The term "including" and its variations used in this document represent open inclusion, that is, "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "based at least in part on". The terms "an example embodiment" and "an embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one other embodiment". In order to facilitate the understanding of this specification, the sequential terms "first", "second", etc. are used in this document to distinguish different elements / items / objects, and do not indicate the order or importance of different elements / items / objects. In particular, the method steps represented by the terms "first", "second", etc. are not used to indicate the order of method execution; when an embodiment contains elements / items / objects represented in a later order, the elements / items / objects represented by the terms "first", "second", etc. in the previous order are not necessarily essential technical features of the embodiment.

[0090] The present application provides a scheme for determining the retroreflective brightness coefficient that can accurately determine the retroreflective brightness coefficient of a marking line, specifically involving a method for determining the retroreflective brightness coefficient, an apparatus for determining the retroreflective brightness coefficient, and electronic equipment, program products, and storage media that can implement the method for determining the retroreflective brightness coefficient.

[0091] As mentioned above, the scheme for determining the retroreflective brightness coefficient in one aspect of the present application not only processes two images with or without single-wavelength laser projection, but also accurately segments the marking area using an image segmentation algorithm for the two images. The scheme for determining the retroreflective brightness coefficient in another aspect of the present application constructs a marking detection model based on light projection in proportion to the marking visibility observation model to determine the retroreflective brightness coefficient. It will be understood that the embodiments of the present application are intended to cover any combination of the above-mentioned aspects or both, which falls within the scope of the present application.

[0092] In the scheme for determining the retroreflective brightness coefficient of the embodiment of the present application, optional or preferred features / schemes may also be included, which are innovative in themselves and can also be applied to other applications besides determining the retroreflective brightness coefficient. Therefore, those skilled in the art will understand that, independent of the scheme for determining the retroreflective brightness coefficient, these optional or preferred features may be claimed in additional or subsequent applications. These optional or preferred features / schemes include, but are not limited to: a marking detection scheme for acquiring marking detection data, a marking recognition scheme for identifying marking areas, and a correction scheme for correcting the retroreflective brightness coefficient. Conversely, those skilled in the art will also understand that the scheme for determining the retroreflective brightness coefficient of different embodiments of the present application may include or not include (involve or not involve) these optional or preferred features / schemes, and may include (involve) all or part of these optional or preferred features / schemes, which will be further described below.

[0093] As mentioned above, in a preferred embodiment of the present application, the marking line detection data used to determine the marking line retroreflection brightness coefficient can be detected and obtained by a marking line detection device according to an embodiment of the present application.

[0094] refer to Figure 1 , shows a marking line detection device 100 of this embodiment, which is configured to detect marking line detection data for determining the retroreflective brightness coefficient of the marking line. However, as mentioned above, the method for determining the retroreflective brightness coefficient according to the embodiment of the present application can use other devices or other means to obtain relevant marking line detection data, which falls within the scope of the present application.

[0095] Continue to refer Figure 1 , the marking line detection device 100 may include a camera 110 and a laser source 120. Figure 1 As shown, the camera 110 can be configured to capture an image of the detected position at a downwardly inclined observation angle. The laser source 120 can be configured to project a single-wavelength laser to the detected position at a downwardly inclined projection angle. In a preferred embodiment, the single-wavelength laser can be an infrared laser or a green laser. In a preferred example, the wavelength of the single-wavelength laser can be approximately 808nm. The detection data that includes both single-wavelength laser projection and non-projected images according to the embodiments of the present application is particularly conducive to detecting or identifying markings, especially markings in images under outdoor natural light environments.

[0096] In some embodiments of the present application, the marking detection device further includes a filter mounted on the camera, the wavelength of the filter corresponding to the wavelength of the single-wavelength laser projected by the laser source, which further helps to detect or identify markings in images under outdoor natural light environments.

[0097] Combined with reference Figures 1 to 3, the marking line detection device 100 may include a first mode and a second mode. In the first mode, the camera 110 projects a laser beam from the laser source 120 ( Figure 1 ) to capture images, and in the second mode, the camera 110 captures images when the laser source 120 does not project laser light (not shown in the figure). Thus, the marking line detection device 100 can detect and obtain marking line detection data corresponding to the detected position, and the marking line detection data corresponding to the detected position includes the first image (such as the first image) captured in the first mode. Figure 2 ) and a second image acquired in the second mode (as shown Figure 3 shown).

[0098] Continue to refer Figure 1 , the camera 110 may be located above the laser source 120. Although not shown in the figure, the camera 110 and the laser source 120 have approximately the same lateral position in a direction transverse to the projection direction of the laser source 120, that is, the camera 110 is approximately located directly above the corresponding laser source 120.

[0099] As mentioned above, the marking line visibility observation model can be used to construct a marking line detection model based on light projection in proportion. Furthermore, the setting angle and height of the camera 110 and the laser source 120 of the marking line detection device 100 can be determined according to the marking line detection model.

[0100] In a preferred embodiment, the line visibility observation model may include a calibration height parameter, the calibration height parameter including a calibration driver observation height H d and calibrate the vehicle light height H cl In one embodiment, the calibrated driver viewing height may represent the height of the eyes of a simulated driver driving a small passenger car. d It can be 120cm. In one embodiment, the vehicle light height H is calibrated cl The height of the headlights of a small passenger car can be characterized. In one example, the vehicle headlight height H is calibrated cl It can be 65cm.

[0101] In a preferred embodiment, the line visibility observation model may include a calibration observation distance D d , which can represent the maximum distance of binocular visual observation of a simulated driver driving a small passenger car, and assuming that the vehicle lights are projected at the same position, that is, the vehicle light projection distance D cl = equal to the calibrated observation distance. In one example, the calibrated observation distance D d And the vehicle light projection distance D cl Both are 30m.

[0102] In a preferred embodiment, the light projection based reticle detection model may include a calibration projection distance D s and the line detection height parameter. The line detection height parameter of the line detection model based on light projection may include the camera height H c and the light projection height H of the laser source l .

[0103] In a preferred embodiment of the present application, the projection distance D is calibrated s It can be preset as needed, and the marking line detection height parameter can be determined by the calibration height parameter according to the ratio between the preset calibration observation distance and the calibration projection distance. Further, the camera height can be determined by the driver's observation height according to the preset ratio between the calibration observation distance and the calibration projection distance, and / or the light projection height can be determined by the calibration vehicle light height.

[0104] In this embodiment, in conjunction with reference Figure 1 , the line detection height parameter can be determined according to the following formulas (1) and (2):

[0105] H c =H d *D s / D d (1)

[0106] H l =H cl *D s / D d (2)

[0107] In one example, the ratio of the camera height of the camera to the light source height of the laser source is 24 / 13±5%. More preferably, the camera height is 24cm±0.5cm and the light source height is 13cm±0.5cm.

[0108] In another preferred embodiment of the present application, the marking detection height parameter can be at least partially pre-set as needed, for example, the camera height and / or the light source height can be pre-set, and the calibrated projection distance can be determined from the calibrated observation distance according to the proportional relationship between the pre-set marking detection height parameter (camera height and / or light source height) and the calibration height parameter (calibrated driver observation height and / or calibrated vehicle light height). In this preferred embodiment, the light source height can be pre-set, and the calibrated projection distance can be determined according to the proportional relationship between the marking detection height parameter and the calibration height parameter; the camera height can also be pre-set, and the light source height can be determined according to the proportional relationship between the calibrated driver observation height and the calibrated vehicle light height, and then the calibrated projection distance can be determined according to the proportional relationship between the marking detection height parameter and the calibration height parameter. However, other specific schemes can also be thought of, as long as they comply with the determination of the calibrated projection distance according to the proportional relationship between the marking detection height parameter and the calibration height parameter based on the at least partially pre-set marking detection height parameter.

[0109] In a preferred embodiment, the line visibility observation model may include observation angle parameters, and the calibration angle parameters include the driver observation angle and the vehicle light projection angle. In one embodiment, the driver observation angle may represent the angle of the farthest position (also the vehicle light projection position) of the simulated driver driving a small passenger car relative to the road surface. In one embodiment, the vehicle light projection angle may be the angle from the headlight to the above-mentioned vehicle light projection position relative to the road surface.

[0110] In a preferred embodiment, the light projection-based marking line detection model may further include marking line detection angle parameters, which may include an observation angle of a camera and a projection angle of a laser source.

[0111] In a preferred embodiment of the present application, the observation angle of the camera can also be determined according to the driver's observation angle. In one example, the observation angle of the camera is 2.29°±0.05°. In a preferred embodiment of the present application, the projection angle of the laser source can also be determined according to the vehicle light projection angle. In one example, the projection angle of the laser source is 1.24°±0.05°.

[0112] In the above preferred embodiment, the correspondence between the parameters of the reticle detection model based on light projection and the reticle visibility observation model is described. It will be understood that the correspondence between the above parameters is a rough correspondence, and is intended to cover any equivalence and / or fluctuations that fall within the application concept. Specifically, the above correspondence explicitly covers the fluctuation range within ±5% of the exact corresponding value.

[0113] In the embodiment of the present application, the detection position of the road marking detection device 100 may include various road surfaces, including but not limited to road surfaces with road markings, and may also include road surfaces with road marking standard plates placed. In an exemplary embodiment, the road marking standard plate may have any suitable length, such as a length of 50 cm. Figure 1 In the embodiment shown, when the light is projected onto the reticle standard plate, the light projection position may be the width W of the reticle standard plate. s The midpoint of , but the application is not limited to this.

[0114] like Figure 1 As shown, the marking detection device 100 may also include a distance sensing unit 130, and the distance sensing unit 130 is configured to obtain the actual projection distance of the laser projected by the laser source to the detected position. The light projection distance obtained in real time by the distance sensing unit 130 can be particularly beneficial to realize the distance correction processing of the embodiment of the present application. In some embodiments, the distance sensing unit 130 can be configured synchronously with the light projection of the laser source, for example, the laser source light projection time t0 can be obtained, and the time t1 when the distance sensing unit 130 receives the reflected light of the projection light can be obtained, and the light projection distance can be determined accordingly. In one example, the distance sensing unit 130 is, for example, a TOF sensor, but the present application is not limited to this.

[0115] like Figure 1 As shown, the marking line detection device 100 may further include an ambient light sensing unit 140, which is configured to obtain ambient light parameters when collecting the first and / or second images. Accordingly, the marking line detection data will also include the ambient light parameters.

[0116] In a comparative example, the following test experiment is conducted on the light source involved and its visibility analysis. More specifically, the test experiment can be used to compare and analyze the effect of the marking detection device under the illumination of the laser light source according to the embodiment of the present application and conventional light sources such as xenon light source, halogen light source, LED light source, etc.

[0117] Specifically, the camera was used to take images of the marking lines illuminated by different types of light sources and images of the marking lines with the light source turned off in a dark room and outdoors during the day. Five images were taken each time. Figure 1 Similar components, configurations and parameters of the marking detection model of the example (however, this does not mean that the marking detection device using a conventional light source is prior art). Then, the marking image is processed, the grayscale value of the marking area in each image is extracted, and its characteristics are analyzed.

[0118] The research results show that in a dark room, under the lighting environment of xenon lamps, halogen lamps, and LED lamps, the marking line images taken by the camera and their grayscale values ​​can be clearly identified, and there are differences in the grayscale values ​​of the marking line images taken with the light source turned off. However, in an outdoor natural light environment, there is no significant difference in the grayscale values ​​of the marking line images taken by the camera when the xenon lamps, halogen lamps, and LED lamps are turned on and the grayscale values ​​of the marking line images taken when the conventional light source is turned off. After subtracting the two (removing natural light interference), it is almost impossible to obtain the grayscale value of the marking line under artificial light alone. In contrast, the marking line detection device using the laser light source of the embodiment of the present application, more specifically, when a camera with a filter (only passes 808nm) is used to shoot an infrared laser (wavelength of 808nm), in both cases, it can well identify the marking line and extract the grayscale of the marking line under artificial light alone, such as Figure 2 The test results of green laser and infrared laser are consistent, as shown in Figure 3 shown.

[0119] In some embodiments of the present application, the line marking detection device includes a plurality of the cameras and a plurality of the laser sources, and each of the cameras is matched with one of the laser sources. The plurality of image acquisition units are spaced apart from each other in a direction transverse to the laser projection direction of the laser source, and each of the cameras and the laser source matched with each other have approximately the same lateral position in a direction transverse to the projection direction of the laser source. In some embodiments, the line marking detection device may be a vehicle-mounted line marking detection device. In the vehicle-mounted line marking detection device, for example, cameras and matching laser sources may be installed respectively on the left front side and the right front side of the vehicle and an optional middle position.

[0120] Next, a method for determining a retroreflective brightness coefficient according to an embodiment of the present application will be described.

[0121] refer to Figure 4 , shows a method for determining a retroreflective brightness coefficient according to an embodiment of the present application, including steps S410 to S460:

[0122] S410: Acquire a first image and a second image corresponding to the detected position in the marking line detection.

[0123] The first image is an image captured when a single-wavelength laser is projected, and the second image is an image captured when a single-wavelength laser is not projected.

[0124] In one embodiment, the method for determining the retroreflective brightness coefficient may use, for example, the marking line detection data obtained by the marking line detection device 100 of the above embodiment. However, it is conceivable that, in other embodiments, other first and second images related to the marking line may be used, as long as the first and second images are images obtained under light projection and light non-projection conditions, respectively.

[0125] In some embodiments of the present application, the method for determining the retroreflective brightness coefficient can be acquired in real time during free driving using a vehicle-mounted road marking detection device. In a further embodiment, the method for determining the retroreflective brightness coefficient can be executed in real time by an electronic device located on the vehicle or by an electronic device of the vehicle itself. In another further embodiment, the method for determining the retroreflective brightness coefficient can be executed in real time by a cloud-based or remote electronic device. When the detection data is acquired by the road marking detection device 100 of the above-mentioned embodiment, the cloud-based or remote electronic device can, for example, be communicatively connected to the road marking detection device 100, such as directly communicating or communicating via a communication unit of the vehicle.

[0126] In some embodiments, the first and second images of the marking detection data can be obtained in real time at a certain interval frequency. For example, the camera can take 30 images per minute at equal time intervals, and the laser light source is alternately turned on and off when the camera takes pictures, and adjacent images can sequentially constitute the first image and the corresponding second image. In a preferred embodiment, the first image can be an image obtained under the projection of an infrared laser at 808nm.

[0127] However, it is conceivable that the method for determining the retroreflective brightness coefficient of the embodiment of the present application is not limited thereto. For example, the method for determining the retroreflective brightness coefficient of the embodiment of the present application can be performed in non-real time using stored images.

[0128] S420: Identify a first marked line region in the first image using a given image segmentation algorithm.

[0129] S430: Identify a second marking line region in the second image using a given image segmentation algorithm.

[0130] In the embodiment of the present application, especially in steps S420 and S430, the image segmentation algorithm is an artificial intelligence image segmentation algorithm based on machine learning or deep learning, especially an end-to-end image segmentation algorithm capable of identifying the marking line area in real time. By accurately identifying and calibrating the marking line detection area, it is possible to quickly and accurately determine the marking line retroreflection brightness coefficient. As an explanation but not limitation, compared to the scheme of determining the retroreflection brightness coefficient by filtering the entire image to obtain the grayscale value, the grayscale value of the marking line area extracted after accurately identifying the marking line area is more accurate, and the accuracy of the retroreflection brightness coefficient determined accordingly is also higher.

[0131] In a preferred embodiment, the marking line identification described in the embodiment of the present application, especially in steps S420 and S430, can be processed using a marking line recognition model based on the YOLO image segmentation algorithm, and more preferably, can be processed using a marking line recognition model based on an improved YOLOv8n-seg image segmentation algorithm. Figures 6 to 8As shown, the marking line recognition model based on the YOLO image segmentation algorithm includes a backbone network (Backbone network) 620, a neck network (Neck network) 630 and multiple segmentation heads (Segment Head) 640 for processing feature maps of different sizes.

[0132] Accordingly, in a preferred embodiment, in the embodiment of the present application, especially in steps S420 and S430, the marking area recognition includes: the input image 610, which is an image related to the marking (such as the first image or the second image) is input into the marking recognition model based on the YOLO image segmentation algorithm for marking recognition processing, so as to identify the marking area in the image. More specifically, in conjunction with reference Figure 5 and Figure 6-8 The input image 610 is processed in sequence by a backbone network 620, a neck network 630 and multiple segmentation heads 641, 642, 643, and 644.

[0133] like Figure 6 and Figure 7 As shown, the backbone network 620 can be used to extract multiple feature maps from the input image 610, and output the multiple feature maps to the neck network 630 through multiple feature channels. Figure 6 and Figure 7 As shown, the backbone network 620 includes a plurality of feature extraction convolution modules 621, 622, 623, and 624 corresponding to a plurality of feature channels. Figure 7 Specifically, in addition to the multiple feature extraction convolution modules 621, 622, 623, and 624 of the above-mentioned multiple feature channels, the backbone network 620 may also include additional layers or modules, which are not described in detail here.

[0134] like Figure 6 and Figure 7 As shown, the neck network 630 may include a bottom-up and top-down sampling structure, including a plurality of necessary neural network layers, such as various layers with sampling functions and required concatenation layers.

[0135] like Figure 6 and Figure 7 As shown, after the sampling process, the neck network 630 can output the sampled feature map to the corresponding segmentation heads 641 to 644. Figure 7In the specific illustrated embodiment, each segmentation head uses a dual loss function to jointly perform detection and segmentation, namely, a prediction box loss function (Bbox Loss) and a target classification loss function (ClsLoss).

[0136] In the embodiments of the present application, the (improved) YOLO-based image segmentation algorithm refers to a YOLO image segmentation algorithm that uses any novel feature of the embodiments of the present application to optimize line marking recognition, and in particular, refers to a YOLOv8n-seg image segmentation algorithm that uses any novel feature of the embodiments of the present application to optimize line marking recognition. It is conceivable that the features of the embodiments of the present application for line marking processing can be combined with or used to improve other end-to-end image segmentation algorithm frameworks, and this application or subsequent applications may cover such novel combinations and improvements.

[0137] In a specific example, a road marking detection device can be configured according to the road marking detection model described above to obtain multiple road images, and the recognition results can be verified by dividing them into a training set, a test set, and a validation set in a certain ratio, such as 8:1:1.

[0138] Before the road marking recognition process or as a preprocessing step, the image used as the training set may also be annotated. In one example, an image annotation tool (Labelme image labeling software) may be used to annotate the road marking images in the training set. In order to accurately obtain the boundary coordinate points and categories of road markings during deep learning, an irregular bounding box mask is used in the Labelme software to mark the target during annotation, and then the category of the target road marking is created and named, and different color masks are used to mark targets of different categories.

[0139] In a further embodiment, Figure 5 As shown, the processing using the marking line recognition model based on the YOLO image segmentation algorithm may specifically include steps S510 to S540:

[0140] S510: Extract features from the input image using the backbone network, and output multiple feature maps through multiple feature channels.

[0141] The multiple feature channels correspond to the multiple segmentation heads.

[0142] S520: Add an attention mechanism to the multiple feature channel parts, so as to apply attention processing to the parts of the multiple feature maps, and do not apply attention processing to other parts of the multiple feature maps.

[0143] S530: using the neck network to perform sampling processing on the multiple feature maps processed by the partial attention, and outputting the multiple feature maps processed by the sampling processing to the multiple segmentation heads accordingly;

[0144] S540: using a plurality of segmentation heads to process the plurality of feature maps that have been sampled, respectively, so as to segment and extract the marked line areas in the plurality of feature maps.

[0145] Combined with reference Figures 6 to 8 The plurality of segmentation heads may include a large-size segmentation head 641 , a plurality (eg, two) of intermediate-size segmentation heads 642 , 643 , and a small-size segmentation head 644 .

[0146] In this preferred embodiment, a small-size segmentation head is added to the marking line recognition model based on the improved YOLO image segmentation algorithm, which can also be called a small target segmentation head.

[0147] Therefore, in the above step S510, multiple feature maps are output through multiple feature channels, including: A1: outputting the large-size feature map through the first feature channel, A2: outputting the multiple intermediate-size feature maps respectively through the multiple second feature channels, and A3: outputting the small-size feature map through the third feature channel.

[0148] Furthermore, if Figures 6 to 8 As shown, the plurality of channels also include a first characteristic channel corresponding to a large-size segmentation head, a plurality of second characteristic channels corresponding to a plurality of intermediate-size segmentation heads, and a third characteristic channel corresponding to a small-size segmentation head.

[0149] Therefore, in the above step S540, the multiple feature maps that have been sampled and processed are outputted to the multiple segmentation heads accordingly, including: B1: the large-size feature map that has been sampled and processed is outputted to the large-size segmentation head accordingly, B2: the multiple intermediate-size feature maps that have been sampled and processed are outputted to the multiple intermediate segmentation heads respectively accordingly, and B3: the small-size feature map that has been sampled and processed is outputted to the small-size segmentation head accordingly.

[0150] In the road marking recognition model based on the improved YOLO image segmentation algorithm in the embodiment of the present application, after being processed by the backbone network (Backbone network) and the neck network (Neck network), not only a large-scale feature map and several intermediate-scale feature maps are output to the corresponding segmentation head for instance segmentation, but also a small-scale feature map is output to the corresponding small-scale segmentation head (small target segmentation head) to segment small target markings, especially for the specific scene of road marking recognition. As an explanation but not a limitation, the scale of the pixel area occupied by the road markings taken by the camera at a small angle in the picture is relatively different. The conventional image segmentation algorithm that has not been specifically modified for road marking recognition has weak instance detection capabilities for various small-scale targets, so it is easy to miss the detection of road markings that are far away from the camera.

[0151] In a preferred embodiment of the present application, the ratio of the large-size feature map corresponding to the large-size segmentation head to the small-size feature map corresponding to the small-size segmentation head is greater than or equal to 8. In one example, the input image size of 640*640 can be downsampled by 20 times (corresponding to the first feature channel of the large-size segmentation head), 40 times, 80 times and 160 times (corresponding to the third feature channel of the small-size segmentation head) to obtain feature maps of sizes 32*32, 16*16, 8*8 and 4*4, which are respectively used for segmentation by multiple feature heads. By using a segmentation head for a small target in the line recognition model of the improved YOLO image segmentation algorithm (and the corresponding backbone network and neck network structure, as further described below), the instance detection capability of small target road markings is effectively improved.

[0152] Combined with reference Figure 5 and Figure 6-8 In step S530, an attention mechanism, such as a channel attention mechanism (CA), is added to the plurality of feature channel parts, so that attention processing is applied to a part of the plurality of feature maps, and attention processing is not applied to other parts of the plurality of feature maps. According to a preferred embodiment, the feature map sizes processed by the segmentation head corresponding to the feature channels to which the attention mechanism is applied are not adjacent. More specifically, in combination with reference Figures 6 to 8 , the above step S520 may correspondingly include: B1: adding an attention mechanism to the first feature channel and the third channel so as to apply attention processing to the large-size feature map and the small-size feature map, and not applying the attention mechanism to the second channel so as not to apply attention processing to the intermediate-size feature map. In other words, the channel attention mechanism 660 is applied only to the first and third feature channels corresponding to the large-size segmentation head and the small-size segmentation head, while the channel attention mechanism is not applied to the second feature channel corresponding to the intermediate-size segmentation head. According to a preferred embodiment, the attention mechanism includes an encoder for embedding the spatial coordinate information of the marking area into the feature and a coordinate attention generation module for generating coordinate attention based on the embedded spatial coordinate information of the marking area.

[0153] The channel attention mechanism is a method of increasing the importance of the channel and the attention information by applying weights to the feature maps of each channel to which the attention mechanism is added. The applicant has made the following surprising discovery through research, that is, adding partial channel attention mechanisms, such as adding channel attention mechanisms non-adjacently, and especially applying channel attention mechanisms only to the feature maps of the largest and smallest sizes, can minimize the interference of various road interference factors on the target information of the road markings. As an explanation but not limitation, through the backbone network bottom-up (such as Figure 6 and Figure 7) Extract deeper features layer by layer. The feature map at the top contains the richest feature information and the most ambiguous position information, while the feature map at the bottom is the opposite. By applying the channel attention mechanism to the feature channels at both ends, the clearest position information and feature information can be extracted and fused to obtain higher accuracy. In addition, the partially applied channel attention mechanism according to the embodiment of the present application, such as applying the channel attention mechanism non-adjacently, especially applying the channel attention mechanism only to the maximum size and minimum size feature maps, can also be combined with further preferred features to obtain further beneficial effects of optimizing marking recognition.

[0154] Continue to refer Figures 6 to 8 As mentioned above, the backbone network 620 includes a plurality of feature extraction convolution modules 621, 622, 623, and 624 corresponding to a plurality of feature channels. In a preferred embodiment of the present application, at least some, preferably all, of the feature extraction convolution modules corresponding to the feature channels to which the attention mechanism is not applied may include a deformable convolution extraction layer, and the feature extraction convolution modules corresponding to the feature channels to which the attention mechanism is added may include a fixed convolution extraction layer. Figure 7 Specifically, in the feature extraction convolution modules 621 and 624 corresponding to the first and third feature channels, a C2F (Coarse to Fine) fixed convolution extraction layer may be included; and in the feature extraction convolution modules 622 and 623 corresponding to the second feature channel to which the attention mechanism is not added, deformable convolution extraction layers 6221 and 6231 may be included. In a preferred embodiment, the deformable convolution extraction layer may include a DCNv3 deformable convolution layer. For specific reference Figure 8 , showing that the deformable convolution extraction layer 6221 has a DCNv3 deformable convolution functional layer. In the embodiment of the present application, it will be understood that the convolution extraction layer can be interpreted broadly, which may include a functional convolution (sub) layer with an extraction function, and may also include additional (sub) layers such as a batch layer, a bottleneck layer, and a splicing layer as needed. The embodiment of the present application does not limit the different numbers and structures of additional (sub) layers in the convolution extraction layer.

[0155] When collecting road markings, the road markings may be severely distorted in shape due to the turning and lane changing operations of the road marking collection vehicle, and the angle of the road markings in the image may also change unknownly. The applicant has made the following surprising discovery through research: by making the feature extraction convolution module corresponding to at least some feature channels without adding the attention mechanism include a variable convolution extraction layer, these distorted road marking features can be effectively and accurately extracted.

[0156] As mentioned above, if Figure 7As shown, each segmentation head uses a dual loss function to jointly perform detection and segmentation, namely, a box loss function (Bbox Loss) and a target classification loss function (Cls Loss). In an embodiment of the present application, the box loss function (Bbox Loss) is further improved to optimize line recognition. In a preferred embodiment, before performing the line recognition process, the relevant method may include a training step, specifically including: C1: inputting a training image containing annotated lines into a line recognition model framework to be trained for training to obtain the trained line recognition model, wherein the training image carries a true value related to the annotated lines, and the true value includes the true box of the annotated lines; C2: the training includes iteratively executing the following steps until the preset training completion condition is reached: C11: inputting a training sample into the line recognition model framework to obtain a predicted value related to the annotated lines, and the predicted value includes a predicted box of the annotated lines. ; C22: Based on a given loss function, calculate the loss value between the predicted value and the true value, wherein the given loss function includes a key point intersection-over-union (MPDIOU) loss function, and the calculation of the loss value between the predicted value and the true value includes: C221: determine the area intersection-over-union ratio of the predicted box and the true box, C222: determine the distance between the box key points of the predicted box and the corresponding box key points of the true box based on a given plurality of prediction box key points, C223: determine the key point intersection-over-union loss value based on the area intersection-over-union ratio and the distance; C23: reversely update the parameters of the marking line recognition model framework based on the loss value.

[0157] In the preferred embodiment, the plurality of predicted box key points include diagonal points. The selection of box diagonal points (box diagonal lines) located on the box diagonal lines is determined according to the orientation and / or shape distortion of the annotated marking lines.

[0158] In some known solutions, the border loss function Box_Loss = DFL_Loss + CIOU_Loss is used to measure the overlap between the predicted box and the real box. DFL_Loss is a deep feature loss function, and CIOU_Loss is a complete intersection-over-union loss function, which uses the distance ratio between the "real box" and the "predicted box" to measure the overlap between the predicted box and the real box. However, after research, the applicant found that this loss function cannot improve the loss value well during line recognition (training).

[0159] In this regard, an embodiment of the present application proposes to use a key point intersection-over-union (MPDIOU) loss function, and preferably, the diagonal points of the box are used as key points of the key point intersection-over-union (MPDIOU) loss function.

[0160] According to a more preferred embodiment, the selection of predicted box diagonal points (box diagonal points) located on the box diagonal is determined according to the direction and / or shape distortion of the marked markings. In one example, for markings turning to the right. The top right and bottom left predicted box diagonal points can be used as key points. In one example, for markings turning to the left, the top left and bottom right predicted box diagonal points can be used as key points. In one example, for straight markings in an image acquired when turning right (and thus causing shape distortion), the top right and bottom left box diagonal points can be used as key points. In one example, for straight markings in an image acquired when turning left (and thus causing shape distortion), the top left and bottom right box diagonal points can be used as key points. In one example, for straight markings in an image acquired when turning left (and thus causing shape distortion), the top left and bottom right box diagonal points can be used as key points.

[0161] By way of explanation and not limitation, for road marking recognition, when the predicted box and the real box have the same aspect ratio but different width and height values, it is difficult to improve the loss value; the embodiment of the present application utilizes the key point intersection-over-union (MPDIOU) loss function, and uses the diagonal points of the box as key points, which can accurately segment road markings caused by interference such as occlusion by vehicles and pedestrians; further, the embodiment of the present application also determines the direction of the diagonal points of the selection box (box diagonal) according to the orientation and / or shape distortion of the marked markings, thereby further improving the ability to accurately detect road markings.

[0162] In one embodiment, the accuracy, recall rate and precision of road marking detection and segmentation in the image verification set are analyzed. The test results show that the model according to the embodiment of the present application can not only meet the end-to-end real-time road marking recognition, but also achieves an accuracy rate of 98.4%, a recall rate of 94.7% and a precision of 96.7% in the detection category; the accuracy rate in segmentation reaches 98.3%, the recall rate reaches 94.2% and the precision reaches 96.4%. The model realizes accurate detection and segmentation of road markings.

[0163] However, it should be understood that other image segmentation algorithms may also be used to obtain new embodiments.

[0164] S440: Extracting a first grayscale value of a first marking line area in the first image.

[0165] S450: Extracting a second grayscale value of a second marking line area in the second image.

[0166] In steps S440 and S450, the grayscale value of the marking area can be extracted in a variety of ways, such as but not limited to averaging, median, etc., and the present application does not limit this. As shown above, the embodiment of the present application improves the accuracy of the calculated retroreflection brightness coefficient by extracting the grayscale value of the accurately segmented marking area instead of extracting the grayscale value from the entire image through threshold screening.

[0167] S460: Calculate the retroreflective brightness coefficient of the marking line at the detected position according to the first grayscale value, the second grayscale value and a given retroreflective brightness coefficient standard value.

[0168] In some embodiments, Fig. 9 As shown, step S460 may include steps S461 to S464:

[0169] S461: Determine a grayscale difference between a first grayscale value and a second grayscale value;

[0170] S462: Obtaining a standard value of the retroreflective brightness coefficient corresponding to a given marking standard plate;

[0171] S463: Obtaining a standard grayscale difference corresponding to a given graticule standard plate;

[0172] In some embodiments, Fig.10 As shown, step S463 may include steps S4631 to S4636:

[0173] S4631: Acquire a third image and a fourth image corresponding to a given reticle standard plate, wherein the third image is an image acquired when a single-wavelength laser is projected, and the fourth image is an image acquired when a single-wavelength laser is not projected;

[0174] S4632: Identify a third marking line region in the third image using a given image segmentation algorithm;

[0175] S4633: Identify a fourth marking line region in the fourth image using a given image segmentation algorithm;

[0176] S4634: extracting a third grayscale value of a third marking line area in the third image;

[0177] S4635: extracting a fourth grayscale value of a fourth reticle area in the fourth image;

[0178] S4636: Determine a standard grayscale difference between the third grayscale value and the fourth grayscale value.

[0179] exist Fig. 9 and Fig.10 In the illustrated embodiment, similar or different methods as described in steps S420 and S430 may be used to identify the third and fourth marking line areas, and similar or different methods as described in steps S440 and S450 may be used to extract the third and fourth grayscale values.

[0180] In addition, it will be understood that in step S463, different methods can be used to obtain the standard grayscale difference corresponding to the given reticle standard plate. For example, a known or stored standard grayscale difference can be obtained by looking up a table.

[0181] S464: Determine the retroreflective brightness coefficient of the marking line at the detected position according to the proportional relationship between the marking line grayscale difference, the standard grayscale difference and the standard value of the retroreflective brightness coefficient.

[0182] In the embodiment of the present application, when or after calculating the retroreflective brightness coefficient of the marking line at the detected position, the method further includes a correction step S470 (not shown): correcting the retroreflective brightness coefficient of the marking line to obtain a corrected retroreflective brightness coefficient.

[0183] More specifically, if Fig.11 As shown, the correction may include distance correction, and specifically the correction step S470 may include:

[0184] S471: Obtain the calibrated projection distance and actual projection distance of single-wavelength laser projection.

[0185] S472: Determine a distance correction coefficient according to the calibrated projection distance and the actual projection distance.

[0186] S473: Correcting the retroreflective brightness coefficient of the marking line according to the distance correction coefficient to obtain a distance-corrected retroreflective brightness coefficient.

[0187] In some embodiments, determining a distance correction coefficient according to the calibrated projection distance and the actual projection distance includes:

[0188] Determine the distance correction coefficient K according to the following formula (3): D :

[0189] K D = D 2 / D N 2 (3)

[0190] Where D is the actual projection distance, D N To calibrate the projection distance.

[0191] In some embodiments, the correction may include ambient light correction, and the correction step S470 may include: correcting the retroreflection brightness coefficient of the marking to obtain a retroreflection brightness coefficient corrected for ambient light, specifically including: D1: obtaining ambient light parameters when obtaining the marking detection data; D2: determining an ambient light correction coefficient based on the ambient light parameters, the marking grayscale difference and the standard value of the retroreflection brightness coefficient of the marking at the detected position; D3: correcting the retroreflection brightness coefficient based on the ambient light correction coefficient to obtain a retroreflection brightness coefficient corrected for ambient light.

[0192] As described above, the correction process is optional, and in some embodiments, the correction process, such as the correction process described in steps S471, S472, S473 and D1, D2, D3, can be combined with the calculation of the retroreflection brightness coefficient, or can be processed separately after the calculation of the (original) retroreflection brightness coefficient. In one exemplary embodiment, the correction process combined with the calculation of the retroreflection brightness coefficient will be described:

[0193] In this example, for example, the detection device of the embodiment of the present application can be used to detect the tested marking standard plate (for the convenience of distinction, the tested marking standard plate is temporarily regarded as the marking line) and determine that the grayscale values ​​of the marking area of ​​the first image and the second image are B1 and B2, then the marking grayscale difference B = B1-B2.

[0194] Retroreflection brightness coefficient without distance / environment correction: Assuming the standard value of retroreflection brightness coefficient is RL s =150mcd.m-2.lx-1, the grayscale difference is standard B 150 , then the retroreflection brightness coefficient without distance / environment correction is RL raw =B*RL s / B 150 Since in this example, the detection is performed using a reticle standard plate, the process can also be conveniently referred to as standard plate calibration, but it will be understood that when the process is performed on a real reticle, the process may correspond to determining the (uncorrected for distance and environment) retroreflective brightness coefficient of the reticle at the detected position according to the above steps S461 to S464.

[0195] Distance correction: As before K D =D 2 / D N 2 .

[0196] Ambient light correction: The applicant found that the grayscale difference B still has some ambient light, especially in natural light environment. In order to completely eliminate the influence of ambient light, the correction coefficient K is introduced. E Specifically, the correlation between the ambient light illuminance E, the grayscale difference B, and the true value RL* of the retroreflection brightness coefficient can be obtained and analyzed. The correlation can be characterized by the model / algorithm f(E) and the correction coefficient K can be determined. E =f(E).

[0197] Therefore, in this example, the method for determining the retroreflective brightness coefficient of the marking line based on laser irradiation can determine the corrected retroreflective brightness coefficient as RL=K D *K E *RL raw =D 2 / D N 2*f(E)*(B1-B2)*RL s / B 150 .

[0198] In this example, the correction factor K E =f(E) can be dynamically determined based on the acquired ambient light illumination E, and the correction coefficient K E The introduction of further realizes the dynamic detection and correction of the retroreflection brightness coefficient, which is beneficial to further eliminate the influence of ambient light.

[0199] Combined with reference Figure 1 and Figure 5 Before acquiring the first image and the second image, the method further includes: S400 (not shown): constructing a marking line detection model based on a proportional relationship, such as Fig.12 As shown, step S400 may specifically include:

[0200] S401: Determine a calibration height parameter for a road marking visibility observation model, where the calibration height parameter includes a calibration driver observation height and a calibration vehicle light height.

[0201] S402: Determine a calibrated observation distance for a marking line visibility observation model.

[0202] S403: setting a calibrated projection distance of light projection for marking line detection, and determining a marking line detection height parameter for marking line detection from a calibrated height parameter based on a proportional relationship between the calibrated observation distance and the calibrated projection distance.

[0203] The marking line detection height parameters include a camera height for collecting images and a light source height for projecting light.

[0204] As an alternative to step S403, S403' (not shown): at least partially set a line detection height parameter for line detection, and based on the proportional relationship between the calibrated height parameter and the at least partially set line detection height parameter, determine the calibrated projection distance of light projection for line detection from the calibrated observation distance.

[0205] The first image and the second image are acquired at the calibrated projection distance, camera height and light projection height.

[0206] The above text describes a method for determining the retroreflective brightness coefficient according to one aspect of the present application in conjunction with the accompanying drawings, which can process two images with or without single-wavelength laser projection, and use an image segmentation algorithm to accurately segment the marking area for the two images respectively; and Fig.12As shown, the preferred embodiment of the method for determining the retroreflective brightness coefficient of this aspect can further construct a process of constructing a marking line detection model based on a proportional relationship. However, it will be understood that in another aspect of the present application, a method for determining the retroreflective brightness coefficient of a marking line detection model can be provided, in which the retroreflective brightness coefficient of the marking line can be determined based on or not based on the laser projection image and / or image segmentation features. For example, the method for determining the retroreflective brightness coefficient of this aspect can determine the retroreflective brightness coefficient based on an ordinary light projection image. Accordingly, as Fig.13 As shown, the embodiment of the present application also provides a method for determining a retroreflective brightness coefficient, which is characterized by comprising:

[0207] S1310: Determine a calibration height parameter for a road marking visibility observation model, where the calibration height parameter includes a calibration driver observation height and a calibration vehicle light height.

[0208] S1320: Determine a calibrated observation distance for the marking line visibility observation model.

[0209] S1330: setting a calibrated projection distance of light projection for marking line detection, and determining a marking line detection height parameter for marking line detection from a calibrated height parameter based on a proportional relationship between the calibrated observation distance and the calibrated projection distance.

[0210] The marking detection height parameters include the camera height and light projection height used to collect images.

[0211] S1330′ (not shown): at least partially setting a mark line detection height parameter, and based on a proportional relationship between the calibrated height parameter and the at least partially set mark line detection height parameter, determining a calibrated projection distance of light projection for mark line detection from the calibrated observation distance;

[0212] S1340: Acquire marking line detection data corresponding to the detected position in the marking line detection.

[0213] The image in the marking detection data is acquired at the calibrated projection distance, camera height and light projection height;

[0214] S1350: Determine the retroreflection brightness coefficient according to the marking line detection data.

[0215] In summary, the embodiments of the present application provide the following several innovative inventive themes, any of which may be combined with or not combined with other innovative themes or their features:

[0216] (1) An embodiment of the present application provides a marking detection device, whose marking detection data includes two mode images with and without single-wavelength laser projection, so that the data detected by the marking detection device can be particularly helpful for accurately calculating the retro-reflective brightness coefficient of the marking.

[0217] Those skilled in the art will appreciate that the line detection data obtained by the line detection data may not be limited to determining the line retroreflection brightness coefficient, or may be additionally used for other purposes of line detection / identification, such as autonomous driving, etc. Those skilled in the art will appreciate that the line detection device and the retroreflection brightness coefficient determination scheme may be provided or implemented by the same or different entities, for example, the line detection device may be provided in the vehicle, and the retroreflection brightness coefficient determination scheme may be implemented by other electronic devices that may or may not belong to the vehicle.

[0218] (2) The embodiment of the present application provides a method for determining the retroreflective brightness coefficient, which not only processes two images with or without single-wavelength laser projection, but also uses an image segmentation algorithm to accurately segment the marking area for the two images. By extracting the grayscale value from the segmented marking area in the marking image and combining it with the standard value of the retroreflective brightness coefficient of the standard plate, the method can accurately determine the retroreflective brightness coefficient of the object to be measured.

[0219] (3) The embodiment of the present application provides another aspect of the retroreflection brightness coefficient determination scheme, which can construct a marking line detection model based on light projection in proportion to the marking line visibility observation model, for determining the retroreflection brightness coefficient. This method effectively ensures the accuracy of the detection results while reducing the measurement complexity and the restrictions on the measurement equipment, thereby ensuring the accuracy of the determined retroreflection brightness coefficient.

[0220] Here, the retroreflective brightness coefficient determination scheme or its features of the first aspect may be combined with the retroreflective brightness coefficient determination scheme of the second aspect.

[0221] (4) The embodiments of the present application provide a method for marking road markings, especially one based on artificial intelligence, which can efficiently (in real time, end-to-end) and accurately (especially avoiding occlusion and shape distortion) mark road markings. The method for identifying road markings provided in the embodiments of the present application can be advantageously combined with the scheme for determining the retroreflective brightness coefficient of the first aspect to further ensure the accurate and rapid determination of the retroreflective brightness coefficient. However, the method for identifying road markings may not be limited to the scheme for determining the retroreflective brightness coefficient of road markings, and may be additionally used for other purposes of detecting / identifying road markings, such as autonomous driving.

[0222] (5) The embodiment of the present application provides a method for correcting the retroreflective brightness coefficient of a marking line based on distance and / or ambient light correction, which solves the problems of strong light interference and dynamic detection angle change, and further ensures the high accuracy of the retroreflective brightness coefficient. Here, the correction method can cover multiple variations of only distance correction, only ambient light correction, and both corrections.

[0223] In addition, the correction method provided in the embodiment of the present application can be advantageously combined with the retroreflection brightness coefficient determination scheme and / or the marking detection device of the embodiment of the present application to further ensure the accurate determination of the retroreflection brightness coefficient. However, the correction method provided in the embodiment of the present application can be used to correct the retroreflection brightness coefficient obtained by other methods. For example, a known retroreflection brightness coefficient measurement device for ordinary light projection can be modified, and a distance sensing unit and / or an ambient light sensing unit can be added to determine the actual distance of light projection and / or the ambient light when collecting images, so as to more accurately correct the retroreflection brightness coefficient value determined by the known measuring device in the subsequent retroreflection brightness coefficient determination process.

[0224] As an example, refer to Fig.14 , also shows a retroreflection brightness coefficient evaluation device 1400 of an embodiment of the present application, which may include: an acquisition unit 1401, a first marking line recognition unit 1402, a second marking line recognition unit 1403, a first image processing unit 1404, a second image processing unit 1405 and a retroreflection brightness coefficient calculation unit 1407, wherein:

[0225] The acquisition unit 1401 is configured to acquire a first image and a second image corresponding to the detected position in the marking line detection, wherein the first image is an image captured when a single-wavelength laser is projected, and the second image is an image captured when a single-wavelength laser is not projected.

[0226] The first marking line identification unit 1402 is configured to identify a first marking line region in the first image using a given image segmentation algorithm.

[0227] The second marking line identification unit 1403 is configured to identify the second marking line region in the second image using a given image segmentation algorithm.

[0228] The first image processing unit 1404 is configured to extract a first grayscale value of a first reticle area in the first image.

[0229] The second image processing unit 1405 is configured to extract a second grayscale value of a second reticle area in the second image.

[0230] The retroreflection brightness coefficient calculation unit 1407 is configured to calculate the retroreflection brightness coefficient of the marking line at the detected position according to the first grayscale value, the second grayscale value and a given retroreflection brightness coefficient standard value.

[0231] In some embodiments of the application, the acquisition unit 1401 is further configured to acquire the ambient light parameter value corresponding to the detected position in the marking line detection; and configured to acquire the actual projection distance of the single-wavelength laser corresponding to the detected position in the marking line detection.

[0232] In this embodiment, reference Fig.14 The retroreflective brightness coefficient testing and evaluation device 1400 may further include a third processing unit 1406, wherein the third processing unit 1406 is configured to extract an ambient light correction coefficient and a distance correction coefficient of the marking line at the detected position.

[0233] In this embodiment, the retroreflection brightness coefficient calculation unit 1407 is further configured to calculate the retroreflection brightness coefficient of the marking line at the detected position based on the first grayscale value, the second grayscale value, the ambient light correction coefficient, the distance correction coefficient and a given retroreflection brightness coefficient standard value.

[0234] In some embodiments of the application, reference Fig.14 The retroreflective brightness coefficient inspection and evaluation device 1400 may also include a retroreflective brightness coefficient evaluation unit 1408, wherein the retroreflective brightness coefficient evaluation unit 1408 is configured to receive the retroreflective brightness coefficient calculated by the calculation unit, and judge the retroreflective performance of the marking according to a preset performance indicator threshold to evaluate whether it meets the prescribed safety standards.

[0235] Under the teachings of the present application, features of a method embodiment may be combined with an apparatus embodiment in a non-contradictory manner to obtain a new embodiment, and features of an apparatus embodiment may also be combined with a method embodiment in a non-contradictory manner to obtain a new embodiment, which falls within the scope of the present application.

[0236] In the embodiment of the present application, the method and device for determining the retroreflective brightness coefficient can be implemented by a computer integrating relevant functional modules or components. Fig.15 A structural diagram of a computer that can be used to implement the method and device for determining the retroreflective brightness coefficient of an embodiment of the present application is shown.

[0237] like Fig.15As shown, the computer 1500 includes a processor 1501, which can perform various appropriate operations and processes according to the programs and / or data stored in the read-only memory (ROM) 1502 or the programs and / or data loaded from the storage part 1508 to the random access memory (RAM) 1503. Processor 1501 may include a central processing unit (CPU), and may be a multi-core processor, or may include multiple processors. In some embodiments, processor 1501 may include a general-purpose main processor and one or more special coprocessors, such as a graphics processing unit (GPU), a neural network processor (NPU), a digital signal processor (DSP), etc. In RAM 1503, various programs and data required for the operation of electronic device 1500 are also stored. Processor 1501, ROM 1502 and RAM 1503 are connected to each other via bus 1504. Input / output (I / O) interface 1505 is also connected to bus 1504.

[0238] The processor and the memory are used together to execute the program stored in the memory. When the program is executed by the computer, the steps or functions of the methods described in the above embodiments can be implemented.

[0239] The following components are connected to the I / O interface 1505: an input section 1506 including a keyboard, a mouse, a touch screen, etc.; an output section 1507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1508 including a hard disk, etc.; and a communication section 1509 including a network interface card such as a LAN card, a modem, etc. The communication section 1509 performs communication processing via a network such as the Internet. A drive 1510 is also connected to the I / O interface 1505 as needed. A removable medium 1511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1510 as needed, so that a computer program read therefrom is installed into the storage section 1508 as needed. Fig.15 Only some components are schematically shown in the figure, which does not mean that the computer system 1500 only includes Fig.15 Components shown.

[0240] In some embodiments, the computer 1500 refers to a mobile terminal, including a mobile phone, a vehicle-mounted terminal, etc. Taking a mobile phone as an example, the electronic device 1500 also includes a display screen with a touch function, an external speaker, a gyroscope, a camera, a 4G / 5G antenna and other device modules.

[0241] The systems, devices, modules or units described in the above embodiments may be implemented by the computer or its associated components. The computer may be, for example, a mobile terminal, a personal computer, a notebook computer, a vehicle-mounted human-computer interaction device, a personal digital assistant, a media player, a navigation device, a tablet computer, or a combination thereof.

[0242] Although not shown, in an embodiment of the present application, a program product is provided, including a computer program, wherein when the computer program is executed by a processor, the method described in any one of the embodiments of the present application is implemented.

[0243] Although not shown, in an embodiment of the present application, a storage medium is provided, the storage medium storing a computer program, the computer program being configured to execute any method of the embodiments of the present application when executed.

[0244] The storage media in the embodiments of the present application include permanent and non-permanent, removable and non-removable items that can be used to store information by any method or technology. Examples of storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0245] The methods, programs, systems, devices, etc. of the embodiments of the present application may be executed or implemented in a single or multiple networked computers, or may be practiced in a distributed computing environment. In the embodiments of the present specification, in these distributed computing environments, tasks may be performed by remote processing devices connected via a communication network.

[0246] Unless explicitly stated, the actions or steps of the methods, programs, and embodiments of the present application do not have to be performed in a specific order and can still achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0247] In this article, multiple embodiments of the present application are described, but for the sake of simplicity, the description of each embodiment is not exhaustive, and the same or similar features or parts between the various embodiments may be omitted. In this article, "one embodiment", "some embodiments", "example", "specific example", or "some examples" are intended to be applicable to at least one embodiment or example according to the present application, rather than all embodiments. The above terms do not necessarily mean to refer to the same embodiment or example. In the absence of mutual contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples.

[0248] The exemplary system and method of the present application have been specifically shown and described with reference to the above embodiments, which are only examples of the preferred modes for implementing the present system and method. It will be appreciated by those skilled in the art that various changes may be made to the embodiments of the system and method described herein when implementing the present system and / or method without departing from the spirit and scope of the present application as defined in the appended claims.

Claims

1. A method for determining a retroreflective brightness coefficient, characterized in that: include: Acquire a first image and a second image corresponding to a detected position in the marking line detection, wherein the first image is an image acquired when a single-wavelength laser is projected, and the second image is an image acquired when the single-wavelength laser is not projected; Identify a first marked line region in the first image using a given image segmentation algorithm; Identifying a second reticle region in the second image using a given image segmentation algorithm; Extracting a first grayscale value of a first marking line area in the first image; Extracting a second grayscale value of a second marking line area in the second image; The retroreflective brightness coefficient of the marking line at the detected position is calculated according to the first grayscale value, the second grayscale value and a given retroreflective brightness coefficient standard value.

2. The method for determining the retroreflective brightness coefficient according to claim 1, characterized in that: The step of calculating the retroreflective brightness coefficient of the marking line at the detected position according to the first grayscale value, the second grayscale value and a given retroreflective brightness coefficient standard value comprises: Determine a grayscale difference between the first grayscale value and the second grayscale value; Obtaining the standard value of the retroreflective brightness coefficient corresponding to a given marking standard plate; Obtaining a standard grayscale difference corresponding to the given graticule standard plate; The retroreflective brightness coefficient of the marking line at the detected position is determined according to the proportional relationship between the marking line grayscale difference, the standard grayscale difference and the standard value of the retroreflective brightness coefficient.

3. The method for determining the retroreflective brightness coefficient according to claim 2, characterized in that: During or after calculating the retroreflective brightness coefficient of the marking line at the detected position, the method further includes: Correcting the retroreflective brightness coefficient of the marking to obtain the retroreflective brightness coefficient corrected for ambient light includes: Acquiring ambient light parameters in the marking line detection; Determine an ambient light correction coefficient according to the ambient light parameter, the grayscale difference of the marking line, and a standard value of the retroreflective brightness coefficient of the marking line at the detected position; The retroreflection brightness coefficient is corrected according to the ambient light correction coefficient to obtain the retroreflection brightness coefficient corrected for ambient light.

4. The method for determining the retroreflective brightness coefficient according to claim 2, characterized in that: The step of obtaining a standard grayscale difference corresponding to the given reticle standard plate includes: Acquire a third image and a fourth image corresponding to the given reticle standard plate, wherein the third image is an image acquired when a single-wavelength laser is projected, and the fourth image is an image acquired when a single-wavelength laser is not projected; identifying a third marking line region in the third image using a given image segmentation algorithm; identifying a fourth marking line region in the fourth image using a given image segmentation algorithm; Extracting a third grayscale value of a third marking line area in the third image; Extracting a fourth grayscale value of a fourth marking line area in the fourth image; The standard grayscale difference between the third grayscale value and the fourth grayscale value is determined.

5. The method for determining the retroreflective brightness coefficient according to any one of claims 1 to 3, characterized in that: When or after calculating the retroreflective brightness coefficient of the marking line at the detected position, the method further includes: Correcting the retroreflective brightness coefficient of the marking to obtain a distance-corrected retroreflective brightness coefficient includes: Obtaining a calibrated projection distance and an actual projection distance of the single-wavelength laser projection; Determine a distance correction coefficient according to the calibrated projection distance and the actual projection distance; The retroreflective brightness coefficient of the marking line is corrected according to the distance correction coefficient to obtain a distance-corrected retroreflective brightness coefficient.

6. The method for determining the retroreflective brightness coefficient according to claim 1, characterized in that: When or after calculating the retroreflective brightness coefficient of the marking line at the detected position, the method further includes: Correcting the retroreflective brightness coefficient of the marking to obtain an ambient light corrected and distance corrected retroreflective brightness coefficient includes: Acquiring ambient light parameters in the marking line detection; Determine an ambient light correction coefficient according to the ambient light parameter, the grayscale difference of the marking line, and a standard value of the retroreflective brightness coefficient of the marking line at the detected position; Obtaining a calibrated projection distance and an actual projection distance of the single-wavelength laser projection; Determine a distance correction coefficient according to the calibrated projection distance and the actual projection distance; The retroreflective brightness coefficient of the marking line is corrected according to the ambient light correction coefficient and the distance correction coefficient to obtain an ambient light-corrected and distance-corrected retroreflective brightness coefficient.

7. The method for determining the retroreflective brightness coefficient according to any one of claims 1 to 3, characterized in that: Before acquiring the first image and the second image, the method further includes: Determining calibration height parameters for a road marking visibility observation model, wherein the calibration height parameters include a calibration driver observation height and a calibration vehicle light height; Determining a calibrated viewing distance for a reticle visibility viewing model; Setting a calibrated projection distance of light projection for line detection, and determining a line detection height parameter for line detection from the calibrated height parameter based on a proportional relationship between the calibrated observation distance and the calibrated projection distance, wherein the line detection height parameter includes a camera height for capturing an image and a light source height for projecting light; or, at least partially setting a line detection height parameter for line detection, and determining the calibrated projection distance of light projection for line detection from the calibrated observation distance based on a proportional relationship between the calibrated height parameter and the at least partially set line detection height parameter; The first image and the second image are acquired at the calibrated projection distance, camera height and light projection height.

8. A method for determining a retroreflective brightness coefficient, characterized in that: include: Determining calibration height parameters for a road marking visibility observation model, wherein the calibration height parameters include a calibration driver observation height and a calibration vehicle light height; Determining a calibrated viewing distance for a reticle visibility viewing model; Setting a calibrated projection distance of light projection for marking line detection, and determining a marking line detection height parameter for marking line detection from the calibrated height parameter based on a proportional relationship between the calibrated observation distance and the calibrated projection distance, wherein the marking line detection height parameter includes a camera height for collecting images and a light projection height; or, at least partially setting a marking line detection height parameter, and determining the calibrated projection distance of light projection for marking line detection from the calibrated observation distance based on a proportional relationship between the calibrated height parameter and at least partially set marking line detection height parameters; Acquire marking line detection data corresponding to the detected position in the marking line detection, wherein an image in the marking line detection data is acquired at the calibrated projection distance, camera height, and light projection height; The retroreflection brightness coefficient is determined according to the marking line detection data.

9. A retroreflective brightness coefficient testing and evaluation device, characterized in that: include: An acquisition unit configured to acquire a first image and a second image corresponding to a detected position in the marking line detection, wherein the first image is an image acquired when a single-wavelength laser is projected, and the second image is an image acquired when the single-wavelength laser is not projected; A first marking line recognition unit configured to identify a first marking line region in the first image using a given image segmentation algorithm; a second marking line recognition unit configured to identify a second marking line region in the second image using a given image segmentation algorithm; a first image processing unit configured to extract a first grayscale value of a first reticle area in the first image; a second image processing unit configured to extract a second grayscale value of a second reticle area in the second image; The retroreflection brightness coefficient calculation unit is configured to calculate the retroreflection brightness coefficient of the marking line at the detected position according to the first grayscale value, the second grayscale value and a given retroreflection brightness coefficient standard value.

10. The retroreflective brightness coefficient testing and evaluation device according to claim 9, characterized in that: The acquisition unit is configured to acquire the ambient light parameter value corresponding to the detected position in the marking line detection; configured to acquire the actual projection distance of the single-wavelength laser corresponding to the detected position in the marking line detection; The retroreflective brightness coefficient testing and evaluation device also includes: a third processing unit configured to extract an ambient light correction coefficient and a distance correction coefficient of the marking line at the detected position; The retroreflection brightness coefficient calculation unit is configured to calculate the retroreflection brightness coefficient of the marking line at the detected position according to the first grayscale value, the second grayscale value, the ambient light correction coefficient, the distance correction coefficient and a given retroreflection brightness coefficient standard value.

11. The retroreflective brightness coefficient testing and evaluation device according to claim 10, characterized in that: The retroreflective brightness coefficient testing and evaluation device also includes: The retroreflective brightness coefficient evaluation unit is configured to receive the retroreflective brightness coefficient calculated by the calculation unit, and judge the retroreflective performance of the marking line according to a preset performance indicator threshold to evaluate whether it meets the prescribed safety standards.

12. An electronic device, characterized in that: include: A processor and a memory storing a computer program, wherein the processor is configured to implement the method according to any one of claims 1 to 8 when running the computer program.

13. A program product comprising a computer program, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images