Retroreflection brightness coefficient determination method and device, electronic equipment and program product

Abstract

The invention discloses a retroreflection brightness coefficient determination method and electronic equipment. The retroreflection brightness coefficient determination method comprises the following steps: determining a calibration height parameter for a marking visibility observation model; determining a calibration observation distance for the marking visibility observation model; setting a calibration projection distance of light projection for marking detection, and determining marking detection height parameters for marking detection by the calibration height parameters based on a proportional relation between the calibration observation distance and the calibration projection distance, the marking detection height parameters including a camera height for acquiring an image and a light projection height; or, at least part of the marked line detection height parameters are set, and based on the proportional relation between the calibrated height parameters and the at least part of the set marked line detection height parameters, the calibrated projection distance of light projection used for marked line detection is determined through the calibrated observation distance; acquiring marked line detection data corresponding to the detected position in marked line detection; and determining a retroreflection brightness coefficient according to the marking detection data.

Description

[0001] This application is a divisional application based on the patent application with the application date of September 30, 2024, application number 202411375640.0, and invention name "Method and device for determining retroreflection coefficient, electronic device and program product". Technical Field

[0002] The present application relates to the field of highway transportation, and in particular to a method and device for determining a retroreflective brightness coefficient, an electronic device, and a program product. Background Art

[0003] Road markings are key traffic safety features painted on the road, designed to channel traffic flow, guide vehicle alignment, and ensure traffic safety. These markings have multiple functions: defining lane boundaries, regulating and managing driving behavior, guiding drivers' sight lines, indicating and warning of road conditions ahead, and clarifying right-of-way.

[0004] After long-term wear and tear, road markings lose visibility and their functionality. Maintenance is necessary. A key criterion for determining whether road markings require maintenance is their retroreflective luminance coefficient, also known as the retroreflective luminance coefficient.

[0005] In my country, handheld retroreflectometers are commonly used to measure the retroreflective brightness coefficient of road markings. Static evaluation requires road closures, resulting in significant social impact and inestimable costs. Dynamic evaluation is fast and has no impact on traffic, but the test is susceptible to interference from strong ambient light and dynamic geometric angle changes, often requiring complex setup and demanding operating conditions, making it difficult to accurately determine the retroreflective brightness coefficient of road markings.

[0006] The content of this background technology description is only for facilitating understanding of the relevant technology in this field and is not regarded as an admission of the prior art. Summary of the Invention

[0007] Therefore, the embodiments of the present application hope to provide a solution for determining the retroreflective brightness coefficient that can at least partially solve the problems mentioned above.

[0008] In a first aspect, a method for determining a retroreflective brightness coefficient is provided, comprising: Acquire a first image and a second image corresponding to a detected position in the marking line detection, wherein the first image is an image captured when a single-wavelength laser is projected, and the second image is an image captured when the single-wavelength laser is not projected; Identifying a first marked line region in the first image using a given image segmentation algorithm; Identifying a second reticle region in the second image using a given image segmentation algorithm; Extracting a first grayscale value of a first marking line area in the first image; extracting a second grayscale value of a second reticle area in the second image; The retroreflective brightness coefficient of the marking line at the detected position is calculated according to the first grayscale value, the second grayscale value and a given standard value of the retroreflective brightness coefficient.

[0009] Optionally, calculating the retroreflective brightness coefficient of the marking line at the detected position according to the first grayscale value, the second grayscale value and a given retroreflective brightness coefficient standard value includes: Determine a grayscale difference between the first grayscale value and the second grayscale value; Obtaining the standard value of the retroreflective brightness coefficient corresponding to a given marking standard plate; Obtaining a standard grayscale difference corresponding to the given reticle standard plate; The retroreflective brightness coefficient of the marking line at the detected position is determined according to the proportional relationship among the marking line grayscale difference, the standard grayscale difference and the standard value of the retroreflective brightness coefficient.

[0010] Optionally, during or after calculating the retroreflective brightness coefficient of the marking line at the detected position, the method further includes: Correcting the retroreflective luminance coefficient of the reticle to obtain an ambient light corrected retroreflective luminance coefficient includes: Acquiring ambient light parameters during the line marking detection; Determining an ambient light correction coefficient according to the ambient light parameter, the grayscale difference of the marking line, and a standard value of the retroreflective brightness coefficient of the marking line at the detected position; The retroreflection brightness coefficient is corrected according to the ambient light correction coefficient to obtain an ambient light corrected retroreflection brightness coefficient.

[0011] Optionally, obtaining a standard grayscale difference corresponding to the given reticle standard plate includes: Acquiring a third image and a fourth image corresponding to the given reticle standard plate, wherein the third image is an image captured when a single-wavelength laser is projected, and the fourth image is an image captured when the single-wavelength laser is not projected; identifying a third marked line region in the third image using a given image segmentation algorithm; identifying a fourth marked line region in the fourth image using a given image segmentation algorithm; Extracting a third grayscale value of a third reticle area in the third image; Extracting a fourth grayscale value of a fourth reticle area in the fourth image; The standard grayscale difference between the third grayscale value and the fourth grayscale value is determined.

[0012] Optionally, when or after calculating the retroreflective brightness coefficient of the marking line at the detected position, the method further includes: Correcting the retroreflective brightness coefficient of the marking to obtain a distance-corrected retroreflective brightness coefficient includes: Obtaining a calibrated projection distance and an actual projection distance of the single-wavelength laser projection; Determining a distance correction coefficient according to the calibrated projection distance and the actual projection distance; The retroreflective brightness coefficient of the marking line is corrected according to the distance correction coefficient to obtain a distance-corrected retroreflective brightness coefficient.

[0013] Optionally, when or after calculating the retroreflective brightness coefficient of the marking line at the detected position, the method further includes: Correcting the retroreflective luminance coefficient of the reticle to obtain an ambient light corrected and distance corrected retroreflective luminance coefficient includes: Acquiring ambient light parameters during the line marking detection; Determining an ambient light correction coefficient according to the ambient light parameter, the grayscale difference of the marking line, and a standard value of the retroreflective brightness coefficient of the marking line at the detected position; Obtaining a calibrated projection distance and an actual projection distance of the single-wavelength laser projection; Determining a distance correction coefficient according to the calibrated projection distance and the actual projection distance; The retroreflective brightness coefficient of the marking line is corrected according to the ambient light correction coefficient and the distance correction coefficient to obtain an ambient light-corrected and distance-corrected retroreflective brightness coefficient.

[0014] Optionally, before acquiring the first image and the second image, the method further includes: Determining calibration height parameters for a road marking visibility observation model, the calibration height parameters including a calibration driver observation height and a calibration vehicle light height; Determining a calibrated viewing distance for a reticle visibility viewing model; Setting a calibrated projection distance for light projection for line detection, and determining a line detection height parameter for line detection from the calibrated height parameter based on a proportional relationship between the calibrated observation distance and the calibrated projection distance, wherein the line detection height parameter includes a height of a camera for capturing an image and a height of a light source for projecting light; or, at least partially setting the line detection height parameter for line detection, and determining the calibrated projection distance for light projection for line detection from the calibrated observation distance based on a proportional relationship between the calibrated height parameter and the at least partially set line detection height parameter; The first image and the second image are acquired at the calibrated projection distance, camera height, and light projection height.

[0015] In a second aspect, a method for determining a retroreflective brightness coefficient is provided, comprising: Determining calibration height parameters for a road marking visibility observation model, the calibration height parameters including a calibration driver observation height and a calibration vehicle light height; Determining a calibrated viewing distance for a reticle visibility viewing model; Setting a calibrated projection distance of light projection for line detection, and determining a line detection height parameter for line detection from the calibrated height parameter based on a proportional relationship between the calibrated observation distance and the calibrated projection distance, wherein the line detection height parameter includes a camera height for capturing an image and a light projection height; or at least partially setting the line detection height parameter, and determining the calibrated projection distance of light projection for line detection from the calibrated observation distance based on a proportional relationship between the calibrated height parameter and at least a partially set line detection height parameter; Acquiring marking detection data corresponding to a detected position in the marking detection, wherein an image in the marking detection data is acquired at the calibrated projection distance, camera height, and light projection height; The retroreflection brightness coefficient is determined according to the marking line detection data.

[0016] In a third aspect, a retroreflective brightness coefficient testing and evaluation device is provided, comprising: an acquisition unit configured to acquire a first image and a second image corresponding to a detected position in the marking line detection, wherein the first image is an image acquired when a single-wavelength laser is projected, and the second image is an image acquired when the single-wavelength laser is not projected; a first marking line recognition unit configured to identify a first marking line region in the first image using a given image segmentation algorithm; a second marking line recognition unit configured to identify a second marking line region in the second image using a given image segmentation algorithm; a first image processing unit configured to extract a first grayscale value of a first reticle area in the first image; a second image processing unit configured to extract a second grayscale value of a second reticle area in the second image; The retroreflective brightness coefficient calculation unit is configured to calculate the retroreflective brightness coefficient of the marking line at the detected position according to the first grayscale value, the second grayscale value and a given retroreflective brightness coefficient standard value.

[0017] Optionally, the acquisition unit is configured to acquire an ambient light parameter value corresponding to a detected position in the marking line detection; configured to acquire an actual projection distance of a single-wavelength laser corresponding to a detected position in the marking line detection; The retroreflective brightness coefficient testing and evaluation device further comprises: a third processing unit configured to extract an ambient light correction coefficient and a distance correction coefficient of the marking line at the detected position; The retroreflection brightness coefficient calculation unit is configured to calculate the retroreflection brightness coefficient of the marking line at the detected position according to the first grayscale value, the second grayscale value, the ambient light correction coefficient, the distance correction coefficient and a given retroreflection brightness coefficient standard value.

[0018] Optionally, the retroreflective brightness coefficient testing and evaluation device further includes: The retroreflective brightness coefficient evaluation unit is configured to receive the retroreflective brightness coefficient calculated by the calculation unit and judge the retroreflective performance of the marking according to a preset performance index threshold to evaluate whether it meets the prescribed safety standards.

[0019] In a fourth aspect, an electronic device is provided, characterized in that it includes: a processor and a memory storing a computer program, and the processor is configured to implement the method of the embodiment of the present application when running the computer program.

[0020] In a fifth aspect, a program product is provided, comprising a computer program, wherein the computer program implements the method of the embodiment of the present application when executed by a processor.

[0021] One aspect of this application provides a method for determining the retroreflective brightness coefficient. This method not only processes two types of images, one with and one without single-wavelength laser projection, but also accurately segments the reticle area using an image segmentation algorithm for each type of image. By extracting grayscale values ​​from the segmented reticle area in the reticle image and combining this with ambient light parameters, the actual projection distance, and the standard value of the retroreflective brightness coefficient of a standard plate, this method can accurately determine the retroreflective brightness coefficient of the object being measured.

[0022] Another aspect of this application is a method for determining the retroreflective brightness coefficient, which enables the construction of a light projection-based marking detection model proportional to the marking visibility observation model for determining the retroreflective brightness coefficient. This method effectively ensures the accuracy of the detection results while reducing measurement complexity and equipment limitations, thereby ensuring the accuracy of the determined retroreflective brightness coefficient.

[0023] The optional features and other effects of the embodiments of the present application are partially described below, and partially can be understood by reading this document. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Hereinafter, embodiments of the present application will be described in detail with reference to the accompanying drawings. The elements shown are not limited to the scale shown in the drawings, and the same or similar reference numerals in the drawings represent the same or similar elements, wherein: Figure 1 A schematic diagram of a marking detection device according to an embodiment of the present application is shown, wherein the marking detection device can obtain marking detection data for use in a retroreflective brightness coefficient determination scheme according to an embodiment of the present application; Figure 2 An example diagram showing an image processed by the retroreflective brightness coefficient determination solution according to an embodiment of the present application; Figure 3 An example diagram showing an image processed by the retroreflective brightness coefficient determination solution according to an embodiment of the present application; Figure 4 An exemplary flow chart of a method for determining a retroreflective brightness coefficient according to an embodiment of the present application is shown; Figure 5 An exemplary flow chart of a marking process of a retroreflective brightness coefficient determination method according to an embodiment of the present application is shown; Figure 6 shows that it can be used to implement Figure 5 A schematic structural diagram of a line marking model for line marking processing; Figure 7 Shown Figure 6 Another schematic structural diagram of the marking model shown; Figure 8 Shown Figure 6 The schematic structure diagram of the feature extraction convolution module of the line recognition model shown; Figure 9 An exemplary flow chart of a method for determining a retroreflective brightness coefficient according to an embodiment of the present application is shown; Figure 10 An exemplary flow chart of a method for determining a retroreflective brightness coefficient according to an embodiment of the present application is shown; Figure 11 An exemplary flow chart showing a correction process of a retroreflective brightness coefficient of a method for determining a retroreflective brightness coefficient according to an embodiment of the present application is shown; Figure 12 An exemplary flow chart of a method for determining a retroreflective brightness coefficient according to an embodiment of the present application is shown; Figure 13 An exemplary flow chart of a method for determining a retroreflective brightness coefficient according to an embodiment of the present application is shown; Figure 14 A schematic diagram showing a retroreflective brightness coefficient evaluation device according to an embodiment of the present application is shown; and Figure 15The structure of an electronic device that can be used to implement the method for determining the retroreflective brightness coefficient according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solutions and advantages of this application more clear, the present application is further described in detail below in conjunction with specific implementation methods and drawings. Here, the illustrative implementation methods and descriptions of this application are used to explain this application, but are not intended to limit this application.

[0026] As used herein, the term "including" and its variations denote open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" denotes "and / or". The term "based on" denotes "based at least in part on". The terms "an example embodiment" and "an embodiment" denote "at least one example embodiment". The term "another embodiment" denotes "at least one other embodiment". To facilitate understanding of this specification, the sequential terms "first", "second", etc. are used herein to distinguish different elements / items / objects and do not denote the order or importance of different elements / items / objects. In particular, method steps expressed with the terms "first", "second", etc. are not intended to indicate the order in which the methods are executed; when an embodiment contains elements / items / objects expressed in a later order, the elements / items / objects expressed in the earlier order with the terms "first", "second", etc. are not necessarily essential technical features of the embodiment.

[0027] The present application provides a retroreflective brightness coefficient determination scheme that can accurately determine the retroreflective brightness coefficient of a marking line, specifically involving a retroreflective brightness coefficient determination method, a retroreflective brightness coefficient determination device, and electronic equipment, program products, and storage media that can implement the retroreflective brightness coefficient determination method.

[0028] As previously described, in one aspect of the present application, the scheme for determining the retroreflective brightness coefficient not only processes two images, one with or without single-wavelength laser projection, but also employs an image segmentation algorithm to accurately segment the marking area for each of these two images. In another aspect of the present application, the scheme for determining the retroreflective brightness coefficient constructs a marking detection model based on light projection in proportion to the marking visibility observation model to determine the retroreflective brightness coefficient. It will be understood that the embodiments of the present application are intended to encompass any combination of schemes involving any one or both of the aforementioned aspects, which fall within the scope of the present application.

[0029] The scheme for determining the retroreflective brightness coefficient in the embodiment of the present application may also include optional or preferred features / schemes, which are innovative in themselves and can also be applied to other applications besides determining the retroreflective brightness coefficient. Therefore, those skilled in the art will understand that, independent of the scheme for determining the retroreflective brightness coefficient, these optional or preferred features may be claimed in additional or subsequent applications. These optional or preferred features / schemes include, but are not limited to: a marking detection scheme for obtaining marking detection data, a marking recognition scheme for identifying marking areas, and a correction scheme for correcting the retroreflective brightness coefficient. Conversely, those skilled in the art will also understand that the scheme for determining the retroreflective brightness coefficient in different embodiments of the present application may include or not include (involve or not involve) these optional or preferred features / schemes, and may include (involve) all or part of these optional or preferred features / schemes, which will be further explained below.

[0030] As mentioned above, in a preferred embodiment of the present application, the marking line detection data used to determine the marking line retroreflection brightness coefficient can be detected and obtained by the marking line detection device according to the embodiment of the present application.

[0031] refer to Figure 1 , shows a marking detection device 100 of this embodiment, which is configured to detect marking detection data used to determine the retroreflective brightness coefficient of the marking. However, as previously mentioned, the method for determining the retroreflective brightness coefficient according to the embodiment of the present application may use other devices or other means to obtain relevant marking detection data, which falls within the scope of the present application.

[0032] Continue to refer Figure 1 , the marking line detection device 100 may include a camera 110 and a laser source 120. Figure 1 As shown, the camera 110 can be configured to capture an image of the inspected location at a downwardly tilted observation angle. The laser source 120 can be configured to project a single-wavelength laser toward the inspected location at a downwardly tilted projection angle. In a preferred embodiment, the single-wavelength laser can be an infrared laser or a green laser. In a preferred example, the wavelength of the single-wavelength laser can be approximately 808 nm. The detection data according to the embodiments of the present application that includes both single-wavelength laser projection and non-projected images is particularly advantageous for detecting or identifying markings, especially markings in images taken outdoors under natural light.

[0033] In some embodiments of the present application, the marking detection device further includes a filter mounted on the camera, wherein the wavelength of the filter corresponds to the wavelength of the single-wavelength laser projected by the laser source. This further facilitates the detection or recognition of markings in images under outdoor natural light conditions.

[0034] Combined with reference Figures 1 to 3, the marking line detection device 100 may include a first mode and a second mode. In the first mode, the camera 110 projects a laser beam from the laser source 120 ( Figure 1 ) captures images. In the second mode, the camera 110 captures images when the laser source 120 does not project laser light (not shown in the figure). As a result, the marking line detection device 100 can detect and obtain marking line detection data corresponding to the detected position. The marking line detection data corresponding to the detected position includes the first image captured in the first mode (such as Figure 2 ) and a second image acquired in the second mode (as shown Figure 3 shown).

[0035] Continue to refer Figure 1 , the camera 110 may be located above the laser source 120. Although not shown in the figure, the camera 110 and the laser source 120 have approximately the same lateral position in a direction transverse to the projection direction of the laser source 120, that is, the camera 110 is approximately located directly above the corresponding laser source 120.

[0036] As mentioned above, the line visibility observation model can be used to construct a line detection model based on light projection in proportion. Furthermore, the setting angle and height of the camera 110 and the laser source 120 of the line detection device 100 can be determined based on the line detection model.

[0037] In a preferred embodiment, the road marking visibility observation model may include a calibration height parameter, the calibration height parameter including the calibration driver observation height H d and calibrate vehicle headlight height H cl In one embodiment, the calibrated driver viewing height may represent the height of the eyes of a simulated driver driving a small passenger car. H d In one embodiment, the vehicle light height is calibrated H cl The height of the headlights of a small passenger car can be characterized. In one example, the vehicle headlight height is calibrated H cl It can be 65cm.

[0038] In a preferred embodiment, the reticle visibility observation model may include a calibration observation distance D d , which can represent the maximum distance of binocular visual observation of a simulated driver driving a small passenger car, and assuming that the vehicle lights are projected at the same position, that is, the vehicle light projection distance D cl Equal to the calibration observation distance. In one example, the calibration observation distance Dd and vehicle light projection distance D cl Both are 30m.

[0039] In a preferred embodiment, the light projection based line detection model may include a calibration projection distance D s And the line detection height parameter, the line detection height parameter of the light projection based line detection model can include the camera height H c and the light projection height of the laser source H l .

[0040] In a preferred embodiment of the present application, the projection distance is calibrated D s The height parameter for line detection can be pre-set as needed, and the line detection height parameter can be determined from the calibrated height parameter based on the ratio between the pre-set calibrated observation distance and the calibrated projection distance. Furthermore, the camera height can be determined from the driver's observation height, and / or the light projection height can be determined from the calibrated vehicle light height, based on the pre-set ratio between the calibrated observation distance and the calibrated projection distance.

[0041] In this embodiment, reference is made to Figure 1 , the line detection height parameters can be determined according to the following formulas (1) and (2): H c =H d *D s / D d (1) H l =H cl *D s / D d (2) In one example, the ratio of the camera height to the light source height of the laser source is 24 / 13±5%. More preferably, the camera height is 24 cm±0.5 cm, and the light source height is 13 cm±0.5 cm.

[0042] In another preferred embodiment of the present application, the line detection height parameter can be at least partially pre-set as needed. For example, the camera height and / or light source height can be pre-set, and the calibrated projection distance can be determined from the calibrated observation distance based on the proportional relationship between the pre-set line detection height parameter (camera height and / or light source height) and the calibration height parameter (calibrated driver observation height and / or calibrated vehicle light height). In this preferred embodiment, the light source height can be pre-set, and the calibrated projection distance can be determined based on the proportional relationship between the line detection height parameter and the calibration height parameter; the camera height can also be pre-set, and the light source height can be determined based on the proportional relationship between the calibrated driver observation height and the calibrated vehicle light height, and then the calibrated projection distance can be determined based on the proportional relationship between the line detection height parameter and the calibration height parameter. However, other specific solutions are also conceivable, as long as they comply with the method of determining the calibrated projection distance based on the proportional relationship between the line detection height parameter and the calibration height parameter according to the at least partially pre-set line detection height parameter.

[0043] In a preferred embodiment, the road marking visibility observation model may include observation angle parameters. Calibrated angle parameters include the driver observation angle and the vehicle light projection angle. In one embodiment, the driver observation angle may represent the angle of the farthest point of visual observation of a simulated driver of a small passenger car (also the vehicle light projection position) relative to the road surface. In one embodiment, the vehicle light projection angle may be the angle between the headlights and the aforementioned vehicle light projection position relative to the road surface.

[0044] In a preferred embodiment, the light projection-based marking line detection model may further include marking line detection angle parameters, which may include an observation angle of a camera and a projection angle of a laser source.

[0045] In a preferred embodiment of the present application, the camera's observation angle can also be determined based on the driver's observation angle. In one example, the camera's observation angle is 2.29°±0.05°. In a preferred embodiment of the present application, the laser source's projection angle can also be determined based on the vehicle's light projection angle. In one example, the laser source's projection angle is 1.24°±0.05°.

[0046] In the preferred embodiment described above, the corresponding relationship between the parameters of the light projection-based reticle detection model and the reticle visibility observation model is described. It will be understood that the corresponding relationship described above is a rough correspondence, intended to encompass any equivalence and / or fluctuations that fall within the scope of the application. Specifically, the corresponding relationship explicitly encompasses fluctuations within ±5% of the exact corresponding value.

[0047] In the embodiment of the present application, the detection position of the road marking detection device 100 may include various road surfaces, including but not limited to road surfaces with markings, and may also include road surfaces with road marking standard plates placed. In an exemplary embodiment, the road marking standard plate may have any suitable length, for example, a length of 50 cm. Figure 1 In the embodiment shown, when light is projected onto the reticle standard plate, the light projection position may be the width of the reticle standard plate. W s The midpoint of , but the application is not limited thereto.

[0048] like Figure 1 As shown, the marking detection device 100 may further include a distance sensing unit 130, which is configured to obtain the actual projection distance of the laser light projected by the laser source to the detected position. The light projection distance obtained in real time by the distance sensing unit 130 can be particularly beneficial for implementing the distance correction processing of the embodiment of the present application. In some embodiments, the distance sensing unit 130 can be configured synchronously with the light projection of the laser source, for example, it can obtain the actual projection distance of the laser light projected by the laser source. t 0 , and obtain the time when the distance sensing unit 130 receives the reflected light of the projected light t 1 , the light projection distance can be determined accordingly. In one example, the distance sensing unit 130 is, for example, a TOF sensor, but the present application is not limited thereto.

[0049] like Figure 1 As shown, the marking detection device 100 may further include an ambient light sensing unit 140, which is configured to obtain ambient light parameters when capturing the first and / or second images. Accordingly, the marking detection data will also include the ambient light parameters.

[0050] In a comparative example, the following test experiments were conducted on the light source and its visibility analysis. More specifically, the test experiments can be used to compare and analyze the effects of the marking detection device under the illumination of the laser light source according to the embodiment of the application and conventional light sources such as xenon light source, halogen light source, and LED light source.

[0051] Specifically, the camera was used to take images of the marking lines illuminated by different types of light sources and images of the marking lines with the light source turned off, respectively, in a dark room and outdoors during the day. Five images were taken continuously each time. In this test, in addition to the different light sources used, the marking detection device used Figure 1 The marking detection model of the example has similar components, configurations, and parameters (however, this does not mean that the marking detection device using a conventional light source is prior art). Then, the marking image is processed, the grayscale value of the marking area in each image is extracted, and its characteristics are analyzed.

[0052] The research results show that in a dark room, under the lighting environment of xenon lamps, halogen lamps, and LED lamps, the marking line images and their grayscale values ​​taken by the camera can be clearly identified, and there are differences in the grayscale values ​​of the marking line images taken with the light source turned off. However, in an outdoor natural light environment, there is no significant difference in the grayscale values ​​of the marking line images taken by the camera when the xenon lamps, halogen lamps, and LED lamps are turned on and when the conventional light source is turned off. After subtracting the two (removing the interference of natural light), it is almost impossible to obtain the grayscale value of the marking line under the illumination of artificial light sources alone. In contrast, the marking line detection device using the laser light source of the embodiment of the present application, more specifically, when a camera with a filter (only passes 808nm) is used to shoot infrared laser (wavelength of 808nm), in both cases, it can well identify the marking line and extract the grayscale of the marking line under the illumination of artificial light sources, such as Figure 2 The test results of green laser and infrared laser are consistent, as shown in Figure 3 shown.

[0053] In some embodiments of the present application, the road marking detection device includes a plurality of cameras and a plurality of laser sources, each camera being matched with a respective laser source. The plurality of image acquisition units are spaced apart from each other in a direction transverse to the laser projection direction of the laser source, and each camera and its respective matched laser source have approximately the same transverse position transverse to the projection direction of the laser source. In some embodiments, the road marking detection device may be a vehicle-mounted road marking detection device. In this vehicle-mounted road marking detection device, for example, cameras and matching laser sources may be mounted on the left front side and right front side of the vehicle, as well as an optional intermediate position.

[0054] Next, a method for determining a retroreflective brightness coefficient according to an embodiment of the present application will be described.

[0055] refer to Figure 4 , shows a method for determining a retroreflective brightness coefficient according to an embodiment of the present application, including steps S410 to S460: S410: Acquire a first image and a second image corresponding to a detected position in the marking line detection.

[0056] The first image is an image captured when a single-wavelength laser is projected, and the second image is an image captured when a single-wavelength laser is not projected.

[0057] In one embodiment, the method for determining the retroreflective brightness coefficient may use, for example, the marking detection data obtained by the marking detection apparatus 100 of the above-described embodiment. However, it is contemplated that in other embodiments, other first and second images related to the marking may be used, as long as the first and second images are images obtained under light projection and non-light projection conditions, respectively.

[0058] In some embodiments of the present application, the method for determining the retroreflective brightness coefficient can be obtained in real time during free driving using a vehicle-mounted road marking detection device. In a further embodiment, the method for determining the retroreflective brightness coefficient can be executed in real time by an electronic device located on the vehicle or by an electronic device of the vehicle itself. In another further embodiment, the method for determining the retroreflective brightness coefficient can be executed in real time by a cloud-based or remote electronic device. When the detection data is acquired by the road marking detection device 100 of the above embodiment, the cloud-based or remote electronic device can, for example, be communicatively connected to the road marking detection device 100, such as directly communicating or relaying the communication connection through the vehicle's communication unit.

[0059] In some embodiments, the matching first and second images of the marking detection data can be acquired in real time at a predetermined interval. For example, a camera can capture 30 images per minute at a constant interval, and the laser light source can be alternately turned on and off during the camera capture. Adjacent images can sequentially constitute the first image and the corresponding second image. In a preferred embodiment, the first image can be acquired using an 808nm infrared laser.

[0060] However, it is conceivable that the method for determining the retroreflective brightness coefficient of the embodiment of the present application is not limited thereto. For example, the method for determining the retroreflective brightness coefficient of the embodiment of the present application can be performed in non-real time using stored images.

[0061] S420: Identify a first marked line region in the first image using a given image segmentation algorithm.

[0062] S430: Identify a second marking line region in the second image using a given image segmentation algorithm.

[0063] In the embodiments of the present application, and particularly in steps S420 and S430, the image segmentation algorithm is an artificial intelligence image segmentation algorithm based on machine learning or deep learning, and in particular, an end-to-end image segmentation algorithm capable of identifying marking areas in real time. By precisely identifying and demarcating the marking detection area, the marking retroreflective brightness coefficient can be quickly and accurately determined. By way of explanation and not limitation, compared to determining the retroreflective brightness coefficient by filtering the entire image to obtain grayscale values, extracting the grayscale values ​​of the marking area after precisely identifying the marking area is more accurate, and the correspondingly determined retroreflective brightness coefficient is also more accurate.

[0064] In a preferred embodiment, the marking line identification in the embodiment of the present application, especially in steps S420 and S430, can be processed using a marking line recognition model based on the YOLO image segmentation algorithm, and more preferably, can be processed using a marking line recognition model based on the improved YOLOv8n-seg image segmentation algorithm. Figures 6 to 8As shown, the line marking recognition model based on the YOLO image segmentation algorithm includes a backbone network 620, a neck network 630, and multiple segmentation heads 640 for processing feature maps of different sizes.

[0065] Accordingly, in a preferred embodiment, in the embodiment of the present application, especially in steps S420 and S430, the marking area recognition includes: inputting an image 610, which is an image related to the marking (such as the first image or the second image) into a marking recognition model based on the YOLO image segmentation algorithm for marking recognition processing, thereby identifying the marking area in the image. More specifically, in conjunction with reference to Figure 5 and Figure 6-8 The input image 610 is processed in sequence by the backbone network 620, the neck network 630 and multiple segmentation heads 641, 642, 643 and 644.

[0066] like Figure 6 and Figure 7 As shown, the backbone network 620 can be used to extract multiple feature maps from the input image 610 and output the multiple feature maps to the neck network 630 through multiple feature channels. Figure 6 and Figure 7 As shown, the backbone network 620 includes multiple feature extraction convolution modules 621, 622, 623, and 624 corresponding to multiple feature channels. Figure 7 Specifically, in addition to the multiple feature extraction convolution modules 621, 622, 623, and 624 of the multiple feature channels, the backbone network 620 may further include additional layers or modules, which are not described in detail here.

[0067] like Figure 6 and Figure 7 As shown, the neck network 630 may include a bottom-up and top-down sampling structure, including a plurality of necessary neural network layers, such as various layers with sampling functions and required concatenation layers.

[0068] like Figure 6 and Figure 7 As shown, after the neck network 630 is sampled, it can output the sampled feature map to the corresponding segmentation heads 641 to 644. Figure 7In the embodiment specifically shown, each segmentation head uses a dual loss function to jointly perform detection and segmentation, namely, a prediction box loss function (Bbox Loss) and an object classification loss function (ClsLoss).

[0069] In the embodiments of this application, the (improved) YOLO-based image segmentation algorithm refers to a YOLO image segmentation algorithm that uses any of the novel features of the embodiments of this application to optimize line marking recognition, and in particular, refers to a YOLOv8n-seg image segmentation algorithm that uses any of the novel features of the embodiments of this application to optimize line marking recognition. It is conceivable that the features used for line marking processing in the embodiments of this application can be incorporated into or used to improve other end-to-end image segmentation algorithm frameworks, and this application or subsequent applications may cover such novel combinations and improvements.

[0070] In a specific example, a road marking detection device can be configured according to the road marking detection model described above to obtain multiple road images, and the recognition results can be verified by dividing them into training set, test set and validation set in a certain ratio, such as 8:1:1.

[0071] Before road marking recognition, or as a preprocessing step, the training set images can also be annotated. In one example, an image annotation tool (Labelme image labeling software) can be used to annotate road marking images in the training set. To accurately obtain the boundary coordinates and categories of road markings during deep learning, Labelme uses irregular bounding box masks to mark the targets. The target road marking categories are then created and named, and different color masks are used to mark different categories of targets.

[0072] In a further embodiment, Figure 5 As shown, the processing using the marking line recognition model based on the YOLO image segmentation algorithm may specifically include steps S510 to S540: S510: Using the backbone network to extract features from the input image, and output multiple feature maps through multiple feature channels.

[0073] The multiple feature channels correspond to the multiple segmentation heads.

[0074] S520: Add an attention mechanism to the multiple feature channel parts, thereby applying attention processing to parts of the multiple feature maps, and not applying attention processing to other parts of the multiple feature maps.

[0075] S530: Using the neck network to sample the multiple feature maps processed by the partial attention, and output the multiple feature maps processed by the sampling process to multiple segmentation heads; S540: Utilize multiple segmentation heads to process the multiple feature maps that have been sampled, respectively, to segment and extract the marked line areas in the multiple feature maps.

[0076] Combined with reference Figures 6 to 8 The plurality of segmentation heads may include a large-size segmentation head 641 , a plurality (eg, two) of intermediate-size segmentation heads 642 , 643 , and a small-size segmentation head 644 .

[0077] In this preferred embodiment, a small-size segmentation head, which may also be called a small target segmentation head, is added to the line recognition model based on the improved YOLO image segmentation algorithm.

[0078] Therefore, in the above step S510, multiple feature maps are output through multiple feature channels, including: A1: outputting the large-size feature map through the first feature channel, A2: outputting the multiple intermediate-size feature maps respectively through the multiple second feature channels, and A3: outputting the small-size feature map through the third feature channel.

[0079] Furthermore, if Figures 6 to 8 As shown, the multiple channels also include a first characteristic channel corresponding to the large-size segmentation head, a plurality of second characteristic channels corresponding to the multiple intermediate-size segmentation heads, and a third characteristic channel corresponding to the small-size segmentation head.

[0080] Therefore, in the above step S540, the multiple feature maps that have been sampled and processed are output to the multiple segmentation heads, including: B1: the large-size feature map that has been sampled and processed is output to the large-size segmentation head, B2: the multiple intermediate-size feature maps that have been sampled and processed are output to the multiple intermediate segmentation heads respectively, and B3: the small-size feature map that has been sampled and processed is output to the small-size segmentation head.

[0081] In the road marking recognition model based on the improved YOLO image segmentation algorithm of the present application embodiment, after processing by the backbone network (Backbone network) and the neck network, not only a large-scale feature map and several intermediate-scale feature maps are output to the corresponding segmentation head for instance segmentation, but also, specifically for the specific scenario of road marking recognition, a small-scale feature map is output to the corresponding small-scale segmentation head (small object segmentation head) to segment small target road markings. By way of explanation and not limitation, the pixel areas occupied by road markings in images captured by cameras at low angles vary greatly in scale. Conventional image segmentation algorithms that have not been specifically modified for road marking recognition have weak instance detection capabilities for various small-scale targets, and therefore are prone to missing road markings that are far from the camera.

[0082] In a preferred embodiment of the present application, the ratio of the large-scale feature map corresponding to the large-scale segmentation head to the small-scale feature map corresponding to the small-scale segmentation head is greater than or equal to 8. In one example, a 640*640 input image can be downsampled by 20 times (corresponding to the first feature channel of the large-scale segmentation head), 40 times, 80 times, and 160 times (corresponding to the third feature channel of the small-scale segmentation head) to obtain feature maps of sizes 32*32, 16*16, 8*8, and 4*4, respectively, which are used for segmentation by multiple feature heads. By using a small target segmentation head (and the corresponding backbone network and neck network structure, as further described below) in the road marking recognition model of the improved YOLO image segmentation algorithm, the instance detection capability of small target road markings is effectively improved.

[0083] Combined with reference Figure 5 and Figure 6-8 In step S530, an attention mechanism, such as a channel attention mechanism (CA), is added to the plurality of feature channel portions, thereby applying attention processing to a portion of the plurality of feature maps, and not applying attention processing to other portions of the plurality of feature maps. According to a preferred embodiment, the feature maps processed by the segmentation head corresponding to the feature channels to which the attention mechanism is applied are not adjacent in size. More specifically, in combination with reference Figures 6 to 8 , the above-mentioned step S520 may correspondingly include: B1: adding an attention mechanism to the first feature channel and the third channel so as to apply attention processing to the large-size feature map and the small-size feature map, and not applying the attention mechanism to the second channel so as not to apply attention processing to the intermediate-size feature map. In other words, the channel attention mechanism 660 is applied only to the first and third feature channels corresponding to the large-size segmentation head and the small-size segmentation head, while the channel attention mechanism is not applied to the second feature channel corresponding to the intermediate-size segmentation head. According to a preferred embodiment, the attention mechanism includes an encoder for embedding the spatial coordinate information of the marking area into the feature and a coordinate attention generation module for generating coordinate attention based on the embedded spatial coordinate information of the marking area.

[0084] The channel attention mechanism is a method of increasing the importance of the channel and the information of interest by applying weights to the feature maps of each channel to which the attention mechanism is added. The applicant has made the following surprising discovery through research, namely, adding partial channel attention mechanisms, such as adding channel attention mechanisms non-adjacently, and especially applying channel attention mechanisms only to the feature maps of the largest and smallest sizes, can minimize the interference of various road interference factors on the target information of the road markings. As an explanation but not limitation, through the backbone network bottom-up (such as Figure 6 and Figure 7) Extract deeper features layer by layer. The feature map at the top contains the richest feature information and the most ambiguous position information, while the feature map at the bottom is the opposite. By applying the channel attention mechanism to the feature channels at both ends, the clearest position information and feature information can be extracted and fused to achieve higher accuracy. In addition, the partially applied channel attention mechanism according to the embodiments of the present application, such as the non-adjacent application of the channel attention mechanism, especially the application of the channel attention mechanism only to the maximum and minimum size feature maps, can also be combined with further preferred features to achieve further beneficial effects of optimizing line marking recognition.

[0085] Continue to refer Figures 6 to 8 As mentioned above, the backbone network 620 includes multiple feature extraction convolution modules 621, 622, 623, and 624 corresponding to multiple feature channels. In a preferred embodiment of the present application, at least some, preferably all, of the feature extraction convolution modules corresponding to feature channels to which the attention mechanism is not applied may include deformable convolution extraction layers, and the feature extraction convolution modules corresponding to feature channels to which the attention mechanism is added may include fixed convolution extraction layers. Figure 7 Specifically, the feature extraction convolution modules 621 and 624 corresponding to the first and third feature channels may include C2F (Coarse to Fine) fixed convolution extraction layers; and the feature extraction convolution modules 622 and 623 corresponding to the second feature channel without the addition of the attention mechanism may include deformable convolution extraction layers 6221 and 6231. In a preferred embodiment, the deformable convolution extraction layer may include a DCNv3 deformable convolution layer. For details, see Figure 8 , showing a DCNv3 deformable convolution functional layer in the deformable convolution extraction layer 6221. In this embodiment of the present application, it will be understood that the convolution extraction layer can be broadly interpreted to include a functional convolution (sub)layer with extraction functionality, and may also include additional (sub)layers such as batch layers, bottleneck layers, and splicing layers as needed. This embodiment of the present application does not limit the number and structure of additional (sub)layers in the convolution extraction layer.

[0086] When capturing road markings, the road markings can be severely distorted due to maneuvers like turns and lane changes by the vehicle collecting them. Furthermore, the angle of the road markings in the image can change in unknown ways. Through research, the applicant has made the surprising discovery that by including variable convolutional extraction layers in the feature extraction convolution modules corresponding to at least some feature channels without an attention mechanism, these distorted road marking features can be effectively and accurately extracted.

[0087] As mentioned above, if Figure 7As shown, each segmentation head uses a dual loss function to jointly perform detection and segmentation, namely the box loss function (Bbox Loss) and the target classification loss function (Cls Loss). In an embodiment of the present application, the box loss function (Bbox Loss) is further improved to optimize line recognition. In a preferred embodiment, before performing the line recognition process, the relevant method may include a training step, specifically including: C1: inputting the training image containing the marked lines into the line recognition model framework to be trained for training to obtain the trained line recognition model, wherein the training image carries the true value related to the marked lines, and the true value includes the true box of the marked lines; C2: the training includes iteratively executing the following steps until the preset training completion condition is reached: C11: inputting the training sample into the line recognition model framework to obtain the predicted value related to the marked lines, and the predicted value includes the predicted box of the marked lines. ; C22: Based on a given loss function, calculate the loss value between the predicted value and the true value, wherein the given loss function includes a key point intersection-over-union (MPDIOU) loss function, and the calculation of the loss value between the predicted value and the true value includes: C221: determining the area intersection-over-union ratio of the predicted box and the true box, C222: determining the distance between the box key point of the predicted box and the corresponding box key point of the true box based on a given plurality of predicted box key points, C223: determining the key point intersection-over-union loss value based on the area intersection-over-union ratio and the distance; C23: reversely updating the parameters of the line marking recognition model framework based on the loss value.

[0088] In the preferred embodiment, the plurality of predicted box key points include diagonal points. The selection of the box diagonal points (box diagonal lines) located on the box diagonal lines is determined based on the orientation and / or shape distortion of the marked lines.

[0089] In some known solutions, the bounding box loss function Box_Loss = DFL_Loss + CIOU_Loss measures the overlap between the predicted box and the ground-truth box. DFL_Loss is a deep feature loss function, and CIOU_Loss is a complete intersection-over-union loss function, which uses the ratio of the distance between the "ground-truth box" and the "predicted box" to measure the overlap between the predicted and ground-truth boxes. However, after research, the applicant found that this loss function does not effectively improve the loss value during line marking recognition (training).

[0090] In this regard, an embodiment of the present application proposes using a key point intersection-over-union (MPDIOU) loss function, and preferably, using the diagonal points of a box as key points of the MPDIOU loss function.

[0091] According to a more preferred embodiment, the selection of predicted box diagonals (box diagonals) located on the box diagonals is determined based on the orientation and / or shape distortion of the annotated marking. In one example, for a marking that turns right, the top right and bottom left predicted box diagonals can be used as key points. In one example, for a marking that turns left, the top left and bottom right predicted box diagonals can be used as key points. In one example, for a straight marking in an image acquired when turning right (and thus causing shape distortion), the top right and bottom left box diagonals can be used as key points. In one example, for a straight marking in an image acquired when turning left (and thus causing shape distortion), the top left and bottom right box diagonals can be used as key points. In one example, for a straight marking in an image acquired when turning left (and thus causing shape distortion), the top left and bottom right box diagonals can be used as key points.

[0092] By way of explanation and not limitation, for road marking recognition, when the predicted box and the real box have the same aspect ratio but different width and height values, it is difficult to improve the loss value; the embodiment of the present application utilizes the key point intersection-over-union (MPDIOU) loss function, and uses the diagonal points of the box as key points, which can accurately segment road markings caused by interference such as occlusion by vehicles and pedestrians; further, the embodiment of the present application also determines the direction of the diagonal points of the selection box (the diagonal line of the box) based on the orientation and / or shape distortion of the marked markings, thereby further improving the ability to accurately detect road markings.

[0093] In one embodiment, the accuracy, recall, and precision of road marking detection and segmentation in an image validation set were analyzed. Experimental results show that the model according to the present embodiment not only meets the requirements for end-to-end real-time road marking recognition, but also achieves 98.4% accuracy, 94.7% recall, and 96.7% precision in detection categories; and 98.3% accuracy, 94.2% recall, and 96.4% precision in segmentation. This model achieves accurate detection and segmentation of road markings.

[0094] However, it should be understood that other image segmentation algorithms may also be used to obtain new embodiments.

[0095] S440: Extracting a first grayscale value of a first marking line area in the first image.

[0096] S450: Extracting a second grayscale value of a second reticle area in the second image.

[0097] In steps S440 and S450, the grayscale values ​​of the marking area can be extracted by various methods, such as, but not limited to, averaging or median calculation, which are not limited in this application. As previously described, the embodiment of the present application improves the accuracy of the calculated retroreflective brightness coefficient by extracting the grayscale values ​​of the accurately segmented marking area, rather than extracting grayscale values ​​from the entire image through threshold screening.

[0098] S460: Calculating the retroreflective brightness coefficient of the marking line at the detected position according to the first grayscale value, the second grayscale value and a given retroreflective brightness coefficient standard value.

[0099] In some embodiments, as Figure 9 As shown, step S460 may include steps S461 to S464: S461: Determine a grayscale difference between a first grayscale value and a second grayscale value; S462: Obtaining a standard value of the retroreflective brightness coefficient corresponding to a given marking standard plate; S463: Obtaining the standard grayscale difference corresponding to the given reticle standard plate; In some embodiments, as Figure 10 As shown, step S463 may include steps S4631 to S4636: S4631: Acquire a third image and a fourth image corresponding to a given reticle standard plate, wherein the third image is an image acquired when a single-wavelength laser is projected, and the fourth image is an image acquired when the single-wavelength laser is not projected; S4632: Identify a third marking line region in the third image using a given image segmentation algorithm; S4633: Identify a fourth marking line region in the fourth image using a given image segmentation algorithm; S4634: Extracting a third grayscale value of a third reticle area in the third image; S4635: Extracting a fourth grayscale value of a fourth reticle area in the fourth image; S4636: Determine a standard grayscale difference between the third grayscale value and the fourth grayscale value.

[0100] exist Figure 9 and Figure 10 In the illustrated embodiment, similar or different methods as described in steps S420 and S430 may be used to identify the third and fourth marking areas, and similar or different methods as described in steps S440 and S450 may be used to extract the third and fourth grayscale values.

[0101] In addition, it will be understood that in step S463, different methods can be used to obtain the standard grayscale difference corresponding to the given reticle standard plate. For example, a known or stored standard grayscale difference can be obtained by looking up a table.

[0102] S464: Determine the retroreflective brightness coefficient of the marking line at the detected position according to the proportional relationship among the marking line grayscale difference, the standard grayscale difference and the standard value of the retroreflective brightness coefficient.

[0103] In the embodiment of the present application, when or after calculating the retroreflective brightness coefficient of the marking line at the detected position, the method further includes a correction step S470 (not shown): correcting the retroreflective brightness coefficient of the marking line to obtain a corrected retroreflective brightness coefficient.

[0104] More specifically, if Figure 11 As shown, the correction may include distance correction. Specifically, the correction step S470 may include: S471: Obtain the calibrated projection distance and actual projection distance of single-wavelength laser projection.

[0105] S472: Determine a distance correction coefficient based on the calibrated projection distance and the actual projection distance.

[0106] S473: Correcting the retroreflective brightness coefficient of the marking line according to the distance correction coefficient to obtain a distance-corrected retroreflective brightness coefficient.

[0107] In some embodiments, determining a distance correction coefficient according to the calibrated projection distance and the actual projection distance includes: Determine the distance correction coefficient according to the following formula (3): K D : K D = D 2 / D N 2 (3) in, D is the actual projection distance, D N To calibrate the projection distance.

[0108] In some embodiments, the correction may include ambient light correction, and the correction step S470 may include: correcting the retroreflection brightness coefficient of the marking to obtain a retroreflection brightness coefficient corrected for ambient light, specifically including: D1: obtaining the ambient light parameters when obtaining the marking detection data; D2: determining the ambient light correction coefficient based on the ambient light parameters, the marking grayscale difference and the standard value of the retroreflection brightness coefficient of the marking at the detected position; D3: correcting the retroreflection brightness coefficient based on the ambient light correction coefficient to obtain a retroreflection brightness coefficient corrected for ambient light.

[0109] As described above, the correction process is optional, and in some embodiments, the correction process, such as the correction process described in steps S471, S472, S473 and D1, D2, and D3, can be combined with the calculation of the retroreflection brightness coefficient, or can be performed separately after the calculation of the (original) retroreflection brightness coefficient. In one exemplary embodiment, the correction process combined with the calculation of the retroreflection brightness coefficient will be described as follows: In this example, the detection device of the embodiment of the present application can be used to detect the reticle standard plate to be tested (for the sake of distinction, the reticle standard plate to be tested is temporarily regarded as the reticle) and determine the grayscale values ​​of the reticle area of ​​the first image and the second image B1 and B2, then the grayscale difference of the reticle is B=B 1 -B 2 .

[0110] Retroreflection brightness coefficient without distance / environment correction: Assuming the standard value of retroreflection brightness coefficient is RL s =150mcd.m-2.lx-1, its grayscale difference is standard B 150 , then the retroreflection brightness coefficient without distance / environment correction is RL raw = B * RL s / B 150 Since the detection is performed using a reticle standard plate in this example, this process can also be conveniently referred to as standard plate calibration. However, it will be understood that when the process is performed on a real reticle, the process may correspond to determining the retroreflective brightness coefficient (not corrected for distance and environment) of the reticle at the detected position according to the above steps S461 to S464.

[0111] Distance correction: As described above K D = D 2 / D N 2 .

[0112] Ambient light correction: The applicant found that the grayscale difference B still has some ambient light, especially in natural light environment. In order to completely eliminate the influence of ambient light, a correction coefficient is introduced. K E Specifically, the correlation between the ambient light intensity E, the grayscale difference B, and the true value RL* of the retroreflective brightness coefficient can be obtained and analyzed. The correlation can be obtained by the model / algorithm. f(E) Characterize and determine the correction factor K E =f(E) .

[0113] Thus, in this example, the method for determining the retroreflective brightness coefficient of the marking line based on laser irradiation can determine the corrected retroreflective brightness coefficient as RL=K D *K E *RL raw = D 2 / D N 2 * f(E)*( B 1 -B 2 )*RL s / B 150 .

[0114] In this example, the correction factor K E =f(E) It can be dynamically determined based on the acquired ambient light illuminance E, through the correction coefficient K E The introduction of further realizes the dynamic detection and correction of the retroreflection brightness coefficient, which further eliminates the influence of ambient light.

[0115] Combined with reference Figure 1 and Figure 5 Before acquiring the first image and the second image, the method further includes: S400 (not shown): constructing a marking line detection model based on the proportional relationship, such as Figure 12 As shown, step S400 may specifically include: S401: Determine calibration height parameters for a road marking visibility observation model, where the calibration height parameters include a calibration driver observation height and a calibration vehicle light height.

[0116] S402: Determine a calibrated observation distance for a marking line visibility observation model.

[0117] S403: setting a calibrated projection distance of light projection for marking line detection, and determining a marking line detection height parameter for marking line detection from a calibrated height parameter based on a proportional relationship between the calibrated observation distance and the calibrated projection distance.

[0118] The marking line detection height parameters include the camera height for collecting images and the light source height for projecting light.

[0119] As an alternative to step S403, S403' (not shown): at least partially set the marking detection height parameter for marking detection, and based on the proportional relationship between the calibrated height parameter and the at least partially set marking detection height parameter, determine the calibrated projection distance of the light projection for marking detection from the calibrated observation distance.

[0120] The first image and the second image are acquired at the calibrated projection distance, camera height, and light projection height.

[0121] The above describes a method for determining the retroreflective brightness coefficient according to one aspect of the present application in conjunction with the accompanying drawings. The method can process two images, one with or without single-wavelength laser projection, and accurately segment the marking area using an image segmentation algorithm for each of the two images; and Figure 12 As shown, the preferred embodiment of the method for determining the retroreflective brightness coefficient of this aspect can further construct a marking line detection model based on a proportional relationship. However, it will be understood that another aspect of the present application can provide a method for determining the retroreflective brightness coefficient for constructing a marking line detection model, in which the retroreflective brightness coefficient of the marking line can be determined based on or without the laser projection image and / or image segmentation features. For example, the method for determining the retroreflective brightness coefficient of this aspect can determine the retroreflective brightness coefficient based on an ordinary light projection image. Accordingly, as Figure 13 As shown, the embodiment of the present application also provides a method for determining a retroreflective brightness coefficient, comprising: S1310: Determine calibration height parameters for a road marking visibility observation model, where the calibration height parameters include a calibration driver observation height and a calibration vehicle light height.

[0122] S1320: Determine a calibrated observation distance for the marking visibility observation model.

[0123] S1330: Setting a calibrated projection distance of light projection for marking line detection, and determining a marking line detection height parameter for marking line detection from a calibrated height parameter based on a proportional relationship between the calibrated observation distance and the calibrated projection distance.

[0124] The line detection height parameters include the camera height and light projection height used to collect images. S1330′ (not shown): at least partially setting a reticle detection height parameter, and determining a calibrated projection distance of light projection for reticle detection from the calibrated observation distance based on a proportional relationship between the calibrated height parameter and the at least partially set reticle detection height parameter; S1340: Acquire marking line detection data corresponding to the detected position during marking line detection.

[0125] The image in the marking detection data is acquired at the calibrated projection distance, camera height, and light projection height; S1350: Determine the retroreflection brightness coefficient according to the marking line detection data.

[0126] In summary, the embodiments of the present application provide the following several innovative themes, any of which may be combined or not combined with other innovative themes or features thereof: (1) The embodiment of the present application provides a marking detection device, wherein the marking detection data includes two mode images with and without single-wavelength laser projection, so that the data obtained by the marking detection device can be particularly helpful for accurately measuring the retroreflective brightness coefficient of the marking.

[0127] Those skilled in the art will appreciate that the road marking detection data obtained may not be limited to determining the retroreflective brightness coefficient of road markings, but may also be used for other purposes such as road marking detection / identification, such as autonomous driving. Those skilled in the art will appreciate that the road marking detection device and the retroreflective brightness coefficient determination scheme may be provided or implemented by the same or different entities. For example, the road marking detection device may be provided on a vehicle, while the retroreflective brightness coefficient determination scheme may be implemented by other electronic devices that may or may not be part of the vehicle.

[0128] (2) The present invention provides a method for determining the retroreflective brightness coefficient. The method not only processes two images, one with or without single-wavelength laser projection, but also accurately segments the marking area using an image segmentation algorithm for each of the two images. By extracting the grayscale value from the segmented marking area in the marking image and combining it with the standard value of the retroreflective brightness coefficient of the standard plate, the method can accurately determine the retroreflective brightness coefficient of the object to be measured.

[0129] (3) The present invention provides another method for determining the retroreflective brightness coefficient. This method can construct a light projection-based marking detection model proportional to the marking visibility observation model to determine the retroreflective brightness coefficient. This method effectively ensures the accuracy of the detection results while reducing measurement complexity and the limitations on the measurement equipment, thereby ensuring the accuracy of the determined retroreflective brightness coefficient.

[0130] Here, the retroreflective brightness coefficient determination scheme or its features of the first aspect may be combined with the retroreflective brightness coefficient determination scheme of the second aspect.

[0131] (4) The embodiments of the present application provide a method for marking road markings, particularly one based on artificial intelligence, which is capable of efficiently (in real time, end-to-end) and accurately (especially avoiding occlusion and shape distortion) marking road markings. The method for identifying road markings provided in the embodiments of the present application can be advantageously combined with the scheme for determining the retroreflective brightness coefficient of the first aspect to further ensure accurate and rapid determination of the retroreflective brightness coefficient. However, the method for identifying road markings is not limited to being combined with the scheme for determining the retroreflective brightness coefficient of road markings and can be additionally used for other purposes of detecting / identifying road markings, such as autonomous driving.

[0132] (5) The present invention provides a method for correcting the retroreflective brightness coefficient of a marking based on distance and / or ambient light correction, addressing the issues of strong light interference and dynamic detection angle changes, and further ensuring high accuracy of the retroreflective brightness coefficient. The correction method may include variations involving distance correction only, ambient light correction only, or both.

[0133] In addition, the correction method provided in the embodiment of the present application can be advantageously combined with the retroreflection brightness coefficient determination scheme and / or the marking detection device of the embodiment of the present application to further ensure the accurate determination of the retroreflection brightness coefficient. However, the correction method provided in the embodiment of the present application can be used to correct the retroreflection brightness coefficient obtained by other methods. For example, a known retroreflection brightness coefficient measurement device for ordinary light projection can be modified to add a distance sensing unit and / or an ambient light sensing unit to determine the actual distance of light projection and / or the ambient light when the image is collected, so as to more accurately correct the retroreflection brightness coefficient value determined by the known measuring device in the subsequent retroreflection brightness coefficient determination process.

[0134] As an example, refer to Figure 14 , also shows a retroreflective brightness coefficient evaluation device 1400 according to an embodiment of the present application, which may include: an acquisition unit 1401, a first marking line recognition unit 1402, a second marking line recognition unit 1403, a first image processing unit 1404, a second image processing unit 1405, and a retroreflective brightness coefficient calculation unit 1407, wherein: The acquisition unit 1401 is configured to acquire a first image and a second image corresponding to the detected position in the marking line detection, wherein the first image is an image captured when a single wavelength laser is projected, and the second image is an image captured when a single wavelength laser is not projected.

[0135] The first marking line identification unit 1402 is configured to identify a first marking line region in the first image using a given image segmentation algorithm.

[0136] The second marking line identification unit 1403 is configured to identify a second marking line region in the second image using a given image segmentation algorithm.

[0137] The first image processing unit 1404 is configured to extract a first grayscale value of a first reticle area in the first image.

[0138] The second image processing unit 1405 is configured to extract a second grayscale value of a second reticle area in the second image.

[0139] The retroreflection brightness coefficient calculation unit 1407 is configured to calculate the retroreflection brightness coefficient of the marking line at the detected position according to the first grayscale value, the second grayscale value and a given retroreflection brightness coefficient standard value.

[0140] In some embodiments of the application, the acquisition unit 1401 is further configured to acquire the ambient light parameter value corresponding to the detected position in the marking detection; and configured to acquire the actual projection distance of the single-wavelength laser corresponding to the detected position in the marking detection.

[0141] In this embodiment, reference Figure 14 The retroreflective brightness coefficient evaluation device 1400 may further include a third processing unit 1406, wherein the third processing unit 1406 is configured to extract the ambient light correction coefficient and the distance correction coefficient of the marking line at the detected position.

[0142] In this embodiment, the retroreflective brightness coefficient calculation unit 1407 is further configured to calculate the retroreflective brightness coefficient of the marking line at the detected position based on the first grayscale value, the second grayscale value, the ambient light correction coefficient, the distance correction coefficient and the given retroreflective brightness coefficient standard value.

[0143] In some embodiments of the application, reference Figure 14 The retroreflective brightness coefficient inspection and evaluation device 1400 may also include a retroreflective brightness coefficient evaluation unit 1408, wherein the retroreflective brightness coefficient evaluation unit 1408 is configured to receive the retroreflective brightness coefficient calculated by the calculation unit, and judge the retroreflective performance of the marking according to a preset performance indicator threshold to evaluate whether it meets the prescribed safety standards.

[0144] Under the teachings of this application, the features of the method embodiments can be combined with the device embodiments in a non-contradictory manner to obtain new embodiments, and the features of the device embodiments can also be combined with the method embodiments in a non-contradictory manner to obtain new embodiments, which falls within the scope of this application.

[0145] In the embodiment of the present application, the method and apparatus for determining the retroreflective brightness coefficient may be implemented by a computer integrating relevant functional modules or components. Figure 15 The structure diagram of a computer that can be used to implement the method and device for determining the retroreflective brightness coefficient of the embodiment of the present application is shown.

[0146] like Figure 15 As shown, computer 1500 includes a processor 1501, which can perform various appropriate operations and processes based on programs and / or data stored in read-only memory (ROM) 1502 or programs and / or data loaded from storage 1508 into random access memory (RAM) 1503. Processor 1501 may include a central processing unit (CPU), and may be a multi-core processor or include multiple processors. In some embodiments, processor 1501 may include a general-purpose main processor and one or more specialized coprocessors, such as a graphics processing unit (GPU), a neural network processing unit (NPU), a digital signal processor (DSP), etc. RAM 1503 also stores various programs and data required for the operation of electronic device 1500. Processor 1501, ROM 1502, and RAM 1503 are interconnected via a bus 1504. An input / output (I / O) interface 1505 is also connected to bus 1504.

[0147] The processor and the memory are used together to execute the program stored in the memory. When the program is executed by the computer, the steps or functions of the methods described in the above embodiments can be implemented.

[0148] The following components are connected to the I / O interface 1505: an input section 1506 including a keyboard, mouse, touch screen, etc.; an output section 1507 including devices such as a cathode ray tube (CRT), liquid crystal display (LCD), and speakers; a storage section 1508 including a hard disk; and a communication section 1509 including a network interface card such as a LAN card or modem. The communication section 1509 performs communication processing via a network such as the Internet. A drive 1510 is also connected to the I / O interface 1505 as needed. Removable media 1511, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in the drive 1510 as needed, so that computer programs read from the media can be installed in the storage section 1508 as needed. Figure 15 Only some components are shown schematically, and it does not mean that the computer system 1500 only includes Figure 15 Components shown.

[0149] In some embodiments, the computer 1500 refers to a mobile terminal, including a mobile phone, a vehicle-mounted terminal, etc. Taking a mobile phone as an example, the electronic device 1500 also includes a display screen with a touch function, an external speaker, a gyroscope, a camera, a 4G / 5G antenna and other device modules.

[0150] The systems, devices, modules, or units described in the above embodiments may be implemented by the computer or its associated components. The computer may be, for example, a mobile terminal, a personal computer, a laptop computer, an in-vehicle human-computer interaction device, a personal digital assistant, a media player, a navigation device, a tablet computer, or a combination thereof.

[0151] Although not shown, in an embodiment of the present application, a program product is provided, including a computer program, wherein when the computer program is executed by a processor, the method described in any one of the embodiments of the present application is implemented.

[0152] Although not shown, in an embodiment of the present application, a storage medium is provided, wherein the storage medium stores a computer program, and the computer program is configured to execute any method of the embodiment of the present application when executed.

[0153] Storage media in embodiments of the present application include permanent and non-permanent, removable and non-removable items that can be used to store information using any method or technology. Examples of storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0154] The methods, programs, systems, and apparatuses of the embodiments of the present application may be executed or implemented in a single or multiple networked computers, or may be practiced in a distributed computing environment. In the embodiments of this specification, in these distributed computing environments, tasks may be performed by remote processing devices connected via a communication network.

[0155] Unless explicitly stated, the actions or steps of the methods, procedures, and embodiments of the present application do not have to be performed in a specific order and can still achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0156] In this document, multiple embodiments of the present application are described, but for the sake of brevity, the description of each embodiment is not exhaustive, and the same or similar features or parts between the various embodiments may be omitted. In this document, "one embodiment", "some embodiments", "example", "specific example", or "some examples" are intended to apply to at least one embodiment or example according to the present application, rather than all embodiments. The above terms do not necessarily mean to refer to the same embodiment or example. Those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they contradict each other.

[0157] While the exemplary systems and methods of the present application have been specifically shown and described with reference to the above-described embodiments, these are merely examples of preferred modes for implementing the present systems and methods. Those skilled in the art will appreciate that various changes may be made to the embodiments of the systems and methods described herein when implementing the present systems and / or methods without departing from the spirit and scope of the present application as defined in the appended claims.

Claims

1. A method for determining a retroreflective brightness coefficient, characterized in that: include: Determining calibration height parameters for a line visibility observation model; Determining a calibrated viewing distance for a reticle visibility viewing model; Setting a calibrated projection distance of light projection for line detection, and determining a line detection height parameter for line detection from the calibrated height parameter based on a proportional relationship between the calibrated observation distance and the calibrated projection distance; or at least partially setting a line detection height parameter, and determining the calibrated projection distance of light projection for line detection from the calibrated observation distance based on a proportional relationship between the calibrated height parameter and at least partially set line detection height parameters, wherein the line detection height parameter includes a camera height for capturing an image and a light projection height; Acquiring marking detection data corresponding to a detected position in the marking detection, wherein an image in the marking detection data is acquired at the calibrated projection distance, camera height, and light projection height; The retroreflection brightness coefficient is determined according to the marking line detection data.

2. The method for determining the retroreflective brightness coefficient according to claim 1, wherein: The calibrated height parameters include a calibrated driver's viewing height and a calibrated vehicle light height.

3. The method for determining the retroreflective brightness coefficient according to claim 1, wherein: The marking line detection data includes a first image with single-wavelength laser projection and a second image without single-wavelength laser projection.

4. The method for determining the retroreflective brightness coefficient according to any one of claims 1 to 3, characterized in that: The obtaining of the marking detection data corresponding to the detected position in the marking detection includes: Acquire a first image and a second image corresponding to a detected position in the marking line detection, wherein the first image is an image captured when a single-wavelength laser is projected, and the second image is an image captured when the single-wavelength laser is not projected; The determining of the retroreflection brightness coefficient according to the marking line detection data includes: Identifying a first marked line region in the first image using a given image segmentation algorithm; Identifying a second reticle region in the second image using a given image segmentation algorithm; Extracting a first grayscale value of a first marking line area in the first image; extracting a second grayscale value of a second reticle area in the second image; The retroreflective brightness coefficient of the marking line at the detected position is calculated according to the first grayscale value, the second grayscale value and a given standard value of the retroreflective brightness coefficient.

5. The method for determining the retroreflective brightness coefficient according to claim 4, wherein: Calculating the retroreflective brightness coefficient of the marking line at the detected position according to the first grayscale value, the second grayscale value and a given retroreflective brightness coefficient standard value includes: Determine a grayscale difference between the first grayscale value and the second grayscale value; Obtaining the standard value of the retroreflective brightness coefficient corresponding to a given marking standard plate; Obtaining a standard grayscale difference corresponding to the given reticle standard plate; The retroreflective brightness coefficient of the marking line at the detected position is determined according to the proportional relationship among the marking line grayscale difference, the standard grayscale difference and the standard value of the retroreflective brightness coefficient.

6. The method for determining the retroreflective brightness coefficient according to claim 4, wherein: During or after calculating the retroreflective brightness coefficient of the marking line at the detected position, the method further includes: Correcting the retroreflective luminance coefficient of the reticle to obtain an ambient light corrected retroreflective luminance coefficient includes: Acquiring ambient light parameters during the line marking detection; Determining an ambient light correction coefficient according to the ambient light parameter, the grayscale difference of the marking line, and a standard value of the retroreflective brightness coefficient of the marking line at the detected position; The retroreflection brightness coefficient is corrected according to the ambient light correction coefficient to obtain an ambient light corrected retroreflection brightness coefficient.

7. The method for determining the retroreflective brightness coefficient according to claim 4, wherein: The obtaining of the standard grayscale difference corresponding to the given reticle standard plate includes: Acquiring a third image and a fourth image corresponding to the given reticle standard plate, wherein the third image is an image captured when a single-wavelength laser is projected, and the fourth image is an image captured when the single-wavelength laser is not projected; identifying a third marked line region in the third image using a given image segmentation algorithm; identifying a fourth marked line region in the fourth image using a given image segmentation algorithm; Extracting a third grayscale value of a third reticle area in the third image; Extracting a fourth grayscale value of a fourth reticle area in the fourth image; The standard grayscale difference between the third grayscale value and the fourth grayscale value is determined.

8. The method for determining the retroreflective brightness coefficient according to claim 4, wherein: When or after calculating the retroreflective brightness coefficient of the marking line at the detected position, the method further includes: Correcting the retroreflective brightness coefficient of the marking to obtain a distance-corrected retroreflective brightness coefficient includes: Obtaining a calibrated projection distance and an actual projection distance of the single-wavelength laser projection; Determining a distance correction coefficient according to the calibrated projection distance and the actual projection distance; The retroreflective brightness coefficient of the marking line is corrected according to the distance correction coefficient to obtain a distance-corrected retroreflective brightness coefficient.

9. The method for determining the retroreflective brightness coefficient according to claim 4, wherein: During or after determining the retroreflective brightness coefficient of the marking, the method further comprises: Correcting the retroreflective luminance coefficient of the reticle to obtain an ambient light corrected and distance corrected retroreflective luminance coefficient includes: Acquiring ambient light parameters during the line marking detection; Determining an ambient light correction coefficient according to the ambient light parameter, the grayscale difference of the marking line, and a standard value of the retroreflective brightness coefficient of the marking line at the detected position; Obtaining a calibrated projection distance and an actual projection distance of the single-wavelength laser projection; Determining a distance correction coefficient according to the calibrated projection distance and the actual projection distance; The retroreflective brightness coefficient of the marking line is corrected according to the ambient light correction coefficient and the distance correction coefficient to obtain an ambient light-corrected and distance-corrected retroreflective brightness coefficient.

10. An electronic device, characterized in that: include: A processor and a memory storing a computer program, wherein the processor is configured to implement the method according to any one of claims 1 to 9 when running the computer program.

Images