Method and device for determining road markings

Abstract

The present invention discloses a method and device for determining road markings. A processor in the device is used to determine a plurality of single-frame images obtained by sampling a video stream including road markings shot by a video capture device; for each pixel point at the same position in each single-frame image, each pixel point is grouped; the difference between the pixel value of each pixel point in each pixel group and the determined reference pixel value is within a preset range; the pixel value of each pixel point in the target pixel group is used to determine the target pixel value at the position corresponding to the target pixel group; the target pixel group is the pixel group with the largest number of pixels included in each pixel group; the pixel points corresponding to each target pixel value obtained at each position are combined into a target frame image; and the wear state of the road markings included in the target frame image is determined according to the target frame image. A clear and complete image including the road markings is obtained, and the wear state of the road markings is accurately determined according to the image.

Description

Technical Field

[0001] The present invention relates to the technical field of public transportation, and in particular to a method and device for determining road markings. Background Art

[0002] The markings on urban roads mainly include directional markings, prohibition markings and warning markings. Among them, directional markings can indicate the lane, driving direction, road edge, sidewalk, etc.; prohibition markings can announce special regulations such as compliance, prohibition, and restriction of road traffic; warning markings prompt road users to understand special situations on the road, increase vigilance, and prepare preventive or contingency measures. However, problems such as unclear and contaminated road markings may cause traffic accidents and threaten the safety of pedestrians, and also affect the success rate of illegal capture. Therefore, in order to understand the use of road markings, it is first necessary to obtain clear, complete and unobstructed road markings.

[0003] In the related technologies, three methods are usually used to obtain clear and complete road markings: manual on-site inspection, manual video inspection, and intelligent video inspection. Among them, manual on-site inspection is the most primitive solution, which has a long cycle, difficult management, laborious and poor effect. Both manual video inspection and intelligent video inspection are based on video inspection, but due to interference from moving objects such as passing vehicles and personnel, the analysis efficiency and judgment accuracy are directly affected. The reason is that online inspection personnel or intelligent inspection programs cannot obtain complete and unobstructed pictures of intersection facilities. If all facilities are to be checked, they need to be checked area by area in the gaps between people or vehicles, which is time-consuming and labor-intensive, easy to miss, and cannot provide a picture to prove the current status of the inspection. Summary of the invention

[0004] In an exemplary embodiment of the present invention, a method and device for determining road markings are provided to obtain a clear, complete, accurate and unobstructed image including road markings, thereby accurately determining the wear state of the road markings, and the operation is simple and quick.

[0005] According to a first aspect of an exemplary embodiment, there is provided a method for determining a road marking, the method comprising:

[0006] Determine a plurality of single-frame images obtained by sampling a video stream including road markings captured by a video capture device; wherein each single-frame image has the same resolution;

[0007] For each pixel point at the same position in each single-frame image, the pixel points are grouped to obtain each pixel group; wherein the difference between the pixel value of each pixel point in each pixel group and the determined reference pixel value is within a preset range;

[0008] Determine the target pixel value at the position corresponding to the target pixel group by using the pixel value of each pixel point in the target pixel group; wherein the target pixel group is the pixel group including the largest number of pixel points among the pixel groups;

[0009] The pixel points corresponding to each target pixel value obtained at each position form a target frame image;

[0010] The wear state of the road marking included in the target frame image is determined according to the target frame image.

[0011] In the above embodiment, the video stream captured by the video capture device includes the static characteristic object of the road marking, and it can be known from the actual situation that it also includes other dynamic characteristic objects. Therefore, firstly, according to the principle of the same single frame resolution, a plurality of single frame images obtained by sampling the video stream are determined. Since the resolution of each single frame image is the same, that is, the number of pixels in each single frame image is the same, in this way, for each pixel at the same position in each single frame image, each pixel is grouped to ensure that the difference between the pixel value of each pixel in each pixel group and the determined reference pixel value is within a preset range, that is, the pixel value of the pixel in each pixel group is relatively close. At this time, the pixel group with the largest number of pixels in each pixel group is determined as the target pixel group, so that the difference point in the blocked image (for example, a part of the vehicle body appears in the image, and the pixel point corresponding to the vehicle body is the difference point) can be discarded, and then the pixel value of each pixel in the target pixel group is used to determine the target pixel value at the position corresponding to the target pixel group, so as to achieve the determination of the target pixel value of the pixel at each position. Finally, the pixel points corresponding to the target pixel values ​​obtained at each position are combined into a target frame image, which can easily, stably and efficiently obtain a complete, accurate and unobstructed image including road markings, and then determine the wear state of the road markings based on the target frame image, thereby improving the recognition accuracy and detection efficiency of road markings, thereby increasing the success rate of illegal capture and reducing the accident rate.

[0012] In some exemplary embodiments, after the pixel points corresponding to the target pixel values ​​obtained at the respective positions are combined into a target frame image, the processor is further configured to execute:

[0013] If the determined target frame image cannot determine the wear state of the road marking, increasing the number of single frame images obtained by sampling;

[0014] Re-determine the target frame image according to each single frame image;

[0015] The wear state of the road marking is determined according to the re-determined target frame image.

[0016] In the above embodiment, only when the determined target frame image cannot determine the wear state of the road marking, the target frame image is re-determined by increasing the number of single frame images obtained by sampling, and then the state of the road marking is re-determined. In this way, whether the number of single frame images needs to be increased to re-determine the target frame image is determined in combination with whether the state of the road marking can be determined, and there is no need to use too many single frame images each time. On the one hand, the data processing amount is saved and the data processing speed is improved, and on the other hand, it is more flexible.

[0017] In some exemplary embodiments, the processor is configured to increase the number of single-frame images obtained by sampling in the following manner:

[0018] Increase the sampling rate when sampling the video stream including road markings captured by the video capture device; or

[0019] The video stream is reacquired, and the reacquired video stream is resampled, and the resampled single-frame image is superimposed with the initial single-frame image to obtain the number of single-frame images.

[0020] In the above embodiment, by adopting any one of the above two strategies, the number of single-frame images can be increased, and the operation is simple.

[0021] In some exemplary embodiments, the processor is configured to execute:

[0022] performing edge processing on the target frame image to determine the wear state of the road marking included in the target frame image; or

[0023] The type of each road marking included in the target frame image is determined, and the road marking is matched with a standard road marking of a corresponding type to determine a wear state of the road marking.

[0024] In the above-mentioned embodiment, any one of the above two strategies can be used to determine the wear state of the road markings. In this way, the wear state of the road markings can be accurately determined by determining the target frame image, so that the operation and maintenance personnel can understand the road conditions in time, and then deal with the worn road markings, thereby improving the success rate of illegal capture and reducing the occurrence rate of accidents.

[0025] In some exemplary embodiments, for each pixel point at the same position in each single-frame image, grouping the pixel points to obtain each pixel group includes:

[0026] Determine a reference pixel point among the pixels at the same position in each single-frame image; wherein the pixel value of the reference pixel point is the reference pixel value;

[0027] Comparing the pixel value of each pixel point with the reference pixel value in turn, determining each pixel point whose absolute value of difference with the reference pixel value is within a preset range and the reference pixel point as a pixel group;

[0028] If there are ungrouped pixels, the reference pixel is re-determined among the ungrouped pixels, and all the ungrouped pixels are traversed according to the reference pixel value of the re-determined reference pixel until the pixel points at the same position in each single-frame image are grouped.

[0029] The above embodiment is a pixel point grouping process. Applying this method for grouping can ensure that the pixel values ​​of the pixels in the same group are infinitely close, that is, the corresponding objects (whether cars, people, or road markings) are roughly the same, so as to determine that the target pixel value at each position is sufficiently accurate.

[0030] In some exemplary embodiments, the reference pixel value includes a reference pixel value of an R channel, a reference pixel value of a G channel, and a reference pixel value of a B channel; the preset range includes a preset range of an R channel, a preset range of a G channel, and a preset range of a B channel;

[0031] Comparing the pixel value of each pixel point with the reference pixel value in turn, determining each pixel point whose absolute value of difference with the reference pixel value is within a preset range and the reference pixel point as a pixel group, comprising:

[0032] Compare the pixel value of the R channel of each pixel point with the reference pixel value of the R channel to obtain the difference of the R channel pixel value; compare the pixel value of the G channel of each pixel point with the reference pixel value of the G channel to obtain the difference of the G channel pixel value; compare the pixel value of the B channel of each pixel point with the reference pixel value of the B channel to obtain the difference of the B channel pixel value;

[0033] If the absolute value of the difference in the R channel pixel values ​​is within the R channel preset range, the absolute value of the difference in the G channel pixel values ​​is within the G channel preset range, and the absolute value of the difference in the B channel pixel values ​​is within the B channel preset range, then it is determined that the pixel points involved in the comparison and the reference pixel point are a pixel group.

[0034] In the above embodiment, since the pixel value usually includes three channels of RGB, during the grouping process, the pixel value of each pixel point to be grouped is compared with the reference value of the corresponding channel, and the pixel points whose differences between the pixel values ​​and the reference pixel values ​​on the three channels are within a preset range and the reference pixel points are determined as a pixel group. This method improves the accuracy of grouping.

[0035] In some exemplary embodiments, the step of applying the pixel values ​​of each pixel point in the target pixel group to determine the target pixel value at the position corresponding to the target pixel group includes:

[0036] For each pixel in the target pixel group, determine a color value according to the pixel value of the pixel;

[0037] A mode value is determined based on each color value, and the pixel value of the pixel point corresponding to the mode value is determined as the target pixel value at the position corresponding to the target pixel group; wherein the mode value is the color value with the most identical color values ​​in the target pixel group.

[0038] The above embodiment is a process for determining the target pixel value at each position. For the same position, it is ensured that the target pixel value at the position is the pixel value with the highest frequency of pixel values ​​appearing at the position in all single-frame images. The target pixel value is applied as the pixel value of the position, and pixels with less frequent occurrence are discarded, thereby improving the accuracy of the target pixel value at each position.

[0039] In some exemplary embodiments, the pixel value includes a pixel value of an R channel, a pixel value of an R channel, and a pixel value of a B channel;

[0040] The step of determining the color value of each pixel in the target pixel group according to the pixel value of the pixel includes:

[0041] For each pixel in the target pixel group, the color value of the pixel is determined according to the pixel value of the R channel of the pixel, the pixel value of the R channel and the pixel value of the B channel, and the weight parameters corresponding to each channel.

[0042] The above embodiment utilizes the correspondence between color values ​​and pixel values ​​to convert the pixel values ​​of the three channels into one-dimensional color value data. In this way, the color values ​​are compared separately, which reduces the amount of calculation in the pixel value comparison process and improves the comparison speed.

[0043] In some exemplary embodiments, the method further comprises:

[0044] If the majority value cannot be determined based on the individual color values, the target pixel value is determined according to the pixel value of each pixel point in the target pixel group.

[0045] In the above embodiment, the majority value cannot be determined based on each color value, indicating that the pixel values ​​of each pixel point are different. At this time, the target pixel value is determined based on the pixel value of each pixel point in the target pixel group, which can also ensure the accuracy of the target pixel value determined in this case.

[0046] According to a second aspect of an exemplary embodiment, a device for determining a road marking is provided, the terminal comprising a processor and a data acquisition unit, wherein:

[0047] The data acquisition unit is configured to perform:

[0048] Obtaining a video stream including road markings captured by a video capture device;

[0049] The processor is configured to perform:

[0050] Determine a plurality of single-frame images obtained by sampling a video stream including road markings captured by the video capture device; wherein each single-frame image has the same resolution;

[0051] For each pixel point at the same position in each single-frame image, the pixel points are grouped to obtain each pixel group; wherein the difference between the pixel value of each pixel point in each pixel group and the determined reference pixel value is within a preset range;

[0052] Determine the target pixel value at the position corresponding to the target pixel group by using the pixel value of each pixel point in the target pixel group; wherein the target pixel group is the pixel group including the largest number of pixel points among the pixel groups;

[0053] The pixel points corresponding to each target pixel value obtained at each position form a target frame image;

[0054] The wear state of the road marking included in the target frame image is determined according to the target frame image.

[0055] In some exemplary embodiments, after the pixel points corresponding to the target pixel values ​​obtained at the respective positions are combined into a target frame image, the processor is further configured to execute:

[0056] If the determined target frame image cannot determine the wear state of the road marking, increasing the number of single frame images obtained by sampling;

[0057] Re-determine the target frame image according to each single frame image;

[0058] The wear state of the road marking is determined according to the re-determined target frame image.

[0059] In some exemplary embodiments, the processor is configured to increase the number of single-frame images obtained by sampling in the following manner:

[0060] Increase the sampling rate when sampling the video stream including road markings captured by the video capture device; or

[0061] The video stream is reacquired, and the reacquired video stream is resampled, and the resampled single-frame image is superimposed with the initial single-frame image to obtain the number of single-frame images.

[0062] In some exemplary embodiments, the processor is configured to execute:

[0063] performing edge processing on the target frame image to determine the wear state of the road marking included in the target frame image; or

[0064] The type of each road marking included in the target frame image is determined, and the road marking is matched with a standard road marking of a corresponding type to determine a wear state of the road marking.

[0065] In some exemplary embodiments, the processor is configured to execute:

[0066] Determine a reference pixel point among the pixels at the same position in each single-frame image; wherein the pixel value of the reference pixel point is the reference pixel value;

[0067] Comparing the pixel value of each pixel point with the reference pixel value in turn, determining each pixel point whose absolute value of difference with the reference pixel value is within a preset range and the reference pixel point as a pixel group;

[0068] If there are ungrouped pixels, the reference pixel is re-determined among the ungrouped pixels, and all the ungrouped pixels are traversed according to the reference pixel value of the re-determined reference pixel until the pixel points at the same position in each single-frame image are grouped.

[0069] In some exemplary embodiments, the reference pixel value includes a reference pixel value of an R channel, a reference pixel value of a G channel, and a reference pixel value of a B channel; the preset range includes a preset range of an R channel, a preset range of a G channel, and a preset range of a B channel;

[0070] The processor is configured to perform:

[0071] Compare the pixel value of the R channel of each pixel point with the reference pixel value of the R channel to obtain the difference of the R channel pixel value; compare the pixel value of the G channel of each pixel point with the reference pixel value of the G channel to obtain the difference of the G channel pixel value; compare the pixel value of the B channel of each pixel point with the reference pixel value of the B channel to obtain the difference of the B channel pixel value;

[0072] If the absolute value of the difference in the R channel pixel values ​​is within the R channel preset range, the absolute value of the difference in the G channel pixel values ​​is within the G channel preset range, and the absolute value of the difference in the B channel pixel values ​​is within the B channel preset range, then it is determined that the pixel points involved in the comparison and the reference pixel point are a pixel group.

[0073] In some exemplary embodiments, the processor is configured to execute:

[0074] For each pixel in the target pixel group, determine a color value according to the pixel value of the pixel;

[0075] A mode value is determined based on each color value, and the pixel value of the pixel point corresponding to the mode value is determined as the target pixel value at the position corresponding to the target pixel group; wherein the mode value is the color value with the most identical color values ​​in the target pixel group.

[0076] In some exemplary embodiments, the processor is specifically configured to perform;

[0077] The step of determining the color value of each pixel in the target pixel group according to the pixel value of the pixel includes:

[0078] For each pixel in the target pixel group, the color value of the pixel is determined according to the pixel value of the R channel of the pixel, the pixel value of the R channel and the pixel value of the B channel, and the weight parameters corresponding to each channel.

[0079] In some exemplary embodiments, if the mode value cannot be determined based on each color value, the processor is further configured to execute:

[0080] The target pixel value is determined according to the pixel value of each pixel point in the target pixel group.

[0081] According to a third aspect of an exemplary embodiment, a device for determining a road marking is provided, the device comprising:

[0082] A video processing module, used for determining a plurality of single-frame images obtained by sampling a video stream including road markings captured by a video capture device; wherein each single-frame image has the same resolution;

[0083] A grouping module, for grouping each pixel point at the same position in each single-frame image to obtain each pixel group; wherein the difference between the pixel value of each pixel point in each pixel group and the determined reference pixel value is within a preset range;

[0084] A target pixel value determination module, used to determine a target pixel value at a position corresponding to the target pixel group by applying the pixel value of each pixel point in the target pixel group; wherein the target pixel group is a pixel group including the largest number of pixel points among the pixel groups;

[0085] An image generation module, used for composing a target frame image from pixel points corresponding to each target pixel value obtained at each position;

[0086] The first wear state determination module is used to determine the wear state of the road marking included in the target frame image according to the target frame image.

[0087] In some exemplary embodiments, a second wear state determination module is further included, which is used to increase the number of single frame images obtained by sampling when the determined target frame image cannot determine the wear state of the road marking after the pixel points corresponding to the target pixel values ​​obtained at the respective positions are formed into a target frame image;

[0088] Re-determine the target frame image according to each single frame image;

[0089] The wear state of the road marking is determined according to the re-determined target frame image.

[0090] In some exemplary embodiments, the second wear state determination module is specifically configured to increase the number of single-frame images obtained by sampling in the following manner:

[0091] Increase the sampling rate when sampling the video stream including road markings captured by the video capture device; or

[0092] The video stream is reacquired, and the reacquired video stream is resampled, and the resampled single-frame image is superimposed with the initial single-frame image to obtain the number of single-frame images.

[0093] In some exemplary embodiments, the first wear state determination module is specifically configured to:

[0094] performing edge processing on the target frame image to determine the wear state of the road marking included in the target frame image; or

[0095] The type of each road marking included in the target frame image is determined, and the road marking is matched with a standard road marking of a corresponding type to determine a wear state of the road marking.

[0096] In some exemplary embodiments, the grouping module is specifically configured to:

[0097] Determine a reference pixel point among the pixels at the same position in each single-frame image; wherein the pixel value of the reference pixel point is the reference pixel value;

[0098] Comparing the pixel value of each pixel point with the reference pixel value in turn, determining each pixel point whose absolute value of difference with the reference pixel value is within a preset range and the reference pixel point as a pixel group;

[0099] If there are ungrouped pixels, the reference pixel is re-determined among the ungrouped pixels, and all the ungrouped pixels are traversed according to the reference pixel value of the re-determined reference pixel until the pixel points at the same position in each single-frame image are grouped.

[0100] In some exemplary embodiments, the grouping module is specifically used to:

[0101] Comparing the pixel value of each pixel point with the reference pixel value in turn, determining each pixel point whose absolute value of difference with the reference pixel value is within a preset range and the reference pixel point as a pixel group, comprising:

[0102] Compare the pixel value of the R channel of each pixel point with the reference pixel value of the R channel to obtain the difference of the R channel pixel value; compare the pixel value of the G channel of each pixel point with the reference pixel value of the G channel to obtain the difference of the G channel pixel value; compare the pixel value of the B channel of each pixel point with the reference pixel value of the B channel to obtain the difference of the B channel pixel value;

[0103] If the absolute value of the difference in the R channel pixel values ​​is within the R channel preset range, the absolute value of the difference in the G channel pixel values ​​is within the G channel preset range, and the absolute value of the difference in the B channel pixel values ​​is within the B channel preset range, then it is determined that the pixel points involved in the comparison and the reference pixel point are a pixel group.

[0104] In some exemplary embodiments, the target pixel value determination module is specifically used to:

[0105] For each pixel in the target pixel group, determine a color value according to the pixel value of the pixel;

[0106] A mode value is determined based on each color value, and the pixel value of the pixel point corresponding to the mode value is determined as the target pixel value at the position corresponding to the target pixel group; wherein the mode value is the color value with the most identical color values ​​in the target pixel group.

[0107] In some exemplary embodiments, the pixel value includes a pixel value of an R channel, a pixel value of an R channel, and a pixel value of a B channel;

[0108] The target pixel value determination module is specifically used for:

[0109] For each pixel in the target pixel group, the color value of the pixel is determined according to the pixel value of the R channel of the pixel, the pixel value of the R channel and the pixel value of the B channel, and the weight parameters corresponding to each channel.

[0110] In some exemplary embodiments, the target pixel value determination module is further configured to determine the target pixel value according to the pixel value of each pixel point in the target pixel group when the mode value cannot be determined based on the respective color values.

[0111] According to a fourth aspect of an exemplary implementation, a computer storage medium is provided, wherein computer program instructions are stored in the computer storage medium. When the instructions are executed on a computer, the computer executes the method for determining road markings as described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0112] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0113] Figure 1 A schematic diagram of a road marking determination system provided by an embodiment of the present invention is exemplarily shown;

[0114] Figure 2 A flowchart of a method for determining a road marking provided by an embodiment of the present invention is exemplarily shown;

[0115] Figure 3 A schematic diagram of capturing C single-frame images according to video stream sampling provided by an embodiment of the present invention is exemplarily shown;

[0116] Figure 4 A schematic diagram of dividing a single-frame image into a plurality of pixel points according to the resolution of the single-frame image provided by an embodiment of the present invention is exemplarily shown;

[0117] Figure 5 A schematic diagram exemplarily shows a grouping process provided by an embodiment of the present invention;

[0118] Figure 6 A schematic diagram of dividing into four pixel groups provided by an embodiment of the present invention is exemplarily shown;

[0119] Figure 7 A flowchart of a method for determining a road marking provided by an embodiment of the present invention is exemplarily shown;

[0120] Figure 8An exemplary effect diagram after a road marking line is determined according to an embodiment of the present invention is shown;

[0121] Fig. 9 Another effect diagram after road markings are determined according to an embodiment of the present invention is exemplarily shown;

[0122] Fig.10 A schematic diagram of the structure of a device for determining road markings provided by an embodiment of the present invention is exemplarily shown;

[0123] Fig.11 The following is a schematic diagram showing the structure of a road marking determination device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0124] The following will be combined with the accompanying drawings to clearly and thoroughly describe the technical solutions in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of the associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0125] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0126] In actual applications, unclear or damaged road markings and zebra crossings may lead to traffic accidents and threaten the safety of pedestrians, while also affecting the success rate of illegal capture. Therefore, it is particularly important to obtain clear and complete road markings.

[0127] In the related technologies, three methods are usually used to solve this problem: manual on-site inspection, manual video inspection, and intelligent video inspection. Among them, manual on-site inspection is the most primitive solution, which has a long cycle, difficult management, time-consuming and labor-intensive, and poor results. Both manual video inspection and intelligent video inspection are based on video inspection, but due to interference from moving objects such as passing vehicles and personnel, the analysis efficiency and judgment accuracy are directly affected. The reason is that online inspection personnel or intelligent inspection programs cannot obtain a complete and unobstructed picture of intersection facilities. If all facilities are to be inspected, they need to be checked area by area in the gaps between people or vehicles, which is time-consuming and labor-intensive, easy to miss, and cannot provide a picture to prove the current status of the inspection.

[0128] In addition, in the related technology, the road traffic markings can be obtained from the low-altitude aerial images of drones, and the connected domain can be screened according to the stroke width of the image pixels to sort the road traffic markings and roads; the position of the road traffic markings is determined by using a cubic B-spline curve fitting algorithm based on random sampling consistency in the screened connected domain of the road traffic markings. However, this method requires that the markings are not blocked by vehicles or pedestrians. If the road markings are blocked, this method is not applicable.

[0129] To this end, the present application provides a method for determining road markings, in which a plurality of single-frame images obtained by sampling a video stream including road markings captured by a video capture device are determined; wherein the resolution of each single-frame image is the same; for each pixel point at the same position in each single-frame image, each pixel point is grouped to obtain each pixel group; wherein the difference between the pixel value of each pixel point in each pixel group and the determined reference pixel value is within a preset range; the pixel value of each pixel point in the target pixel group is used to determine the target pixel value at the position corresponding to the target pixel group; wherein the target pixel group is the pixel group with the largest number of pixels included in each pixel group; the pixel points corresponding to each target pixel value obtained at each position are combined into a target frame image; and the wear state of the road markings included in the target frame image is determined according to the target frame image. In this way, the occlusion of the road markings by dynamic objects such as vehicles and pedestrians is excluded, and the image composed after processing retains static elements such as road markings.

[0130] After introducing the design ideas of the embodiments of the present application, the following briefly introduces the application scenarios to which the technical solutions of the embodiments of the present application can be applied. It should be noted that the application scenarios introduced below are only used to illustrate the embodiments of the present application and are not limited. In specific implementation, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0131] refer to Figure 1, shows a schematic diagram of a road marking determination system, which includes a video capture device 110, a server 120 and a road marking determination device 130. Among them, the video capture device 110 is used to capture intersections, the server 120 is a server for acquiring and storing a video stream including road markings captured by the video capture device, and the road marking determination device 130 can be, for example, a computer. Specifically, the video capture device 110 sends the captured video stream to the server, and after the server stores the video stream, it sends it to the road marking determination device 130 through a communication network. The road marking determination device 130 applies the method in the embodiment of the present application to obtain a clear, complete and unobstructed image including road markings, and then uses the image to improve the success rate of illegal capture on the one hand, and on the other hand, maintain the road markings according to the image, which can also reduce the incidence of traffic accidents.

[0132] Next, the technical solutions of the embodiments of the present application are described through various embodiments. Figure 2 .

[0133] S201. Determine a plurality of single-frame images obtained by sampling a video stream including road markings captured by a video capture device; wherein each single-frame image has the same resolution.

[0134] S202, grouping each pixel point at the same position in each single-frame image to obtain each pixel group; wherein the difference between the pixel value of each pixel point in each pixel group and the determined reference pixel value is within a preset range.

[0135] S203 , applying the pixel values ​​of each pixel point in the target pixel group to determine the target pixel value at the position corresponding to the target pixel group; wherein the target pixel group is the pixel group including the largest number of pixel points among the pixel groups.

[0136] S204, composing a target frame image with pixel points corresponding to each target pixel value obtained at each position.

[0137] S205 : Determine the wear state of the road marking included in the target frame image according to the target frame image.

[0138] In the above embodiment, the video stream captured by the video capture device includes the static characteristic object of the road marking, and it can be known from the actual situation that it also includes other dynamic characteristic objects. Therefore, firstly, according to the principle of the same single frame resolution, a plurality of single frame images obtained by sampling the video stream are determined. Since the resolution of each single frame image is the same, that is, the number of pixels in each single frame image is the same, in this way, for each pixel at the same position in each single frame image, each pixel is grouped to ensure that the difference between the pixel value of each pixel in each pixel group and the determined reference pixel value is within a preset range, that is, the pixel value of the pixel in each pixel group is relatively close. At this time, the pixel group with the largest number of pixels in each pixel group is determined as the target pixel group, so that the difference point in the blocked image (for example, a part of the vehicle body appears in the image, and the pixel point corresponding to the vehicle body is the difference point) can be discarded, and then the pixel value of each pixel in the target pixel group is used to determine the target pixel value at the position corresponding to the target pixel group, so as to achieve the determination of the target pixel value of the pixel at each position. Finally, the pixel points corresponding to the target pixel values ​​obtained at each position are combined into a target frame image, which can easily, stably and efficiently obtain a complete, accurate and unobstructed image including road markings, and then determine the wear state of the road markings based on the target frame image, thereby improving the recognition accuracy and detection efficiency of road markings, thereby increasing the success rate of illegal capture and reducing the accident rate.

[0139] In S201, the video capture equipment is, for example, a bayonet camera, an electronic police (abbreviated as electronic police, the common name for the intelligent traffic violation monitoring management system), etc. Usually, the video captured by the video capture equipment includes not only road markings, but also moving vehicles and passers-by, etc. Dynamic objects such as vehicles and pedestrians may block the road markings. Specifically, the road marking determination device receives the electronic police video from the electronic police, the bayonet video from the bayonet camera, and the monitoring video resources from other social monitoring equipment (such as the camera outside the store at the intersection), etc. For example, the video stream F within the time period [T1, T2] is obtained, and the video stream is sampled according to the time interval Td to obtain a single frame image, and a total of C = (T2-T1) / Td single frame images are intercepted, which are recorded as Pc (c = 1, 2, ..., C).

[0140] refer to Figure 3 , shows a schematic diagram of sampling and intercepting C single-frame images according to a video stream, wherein each small square is a pixel point, such as 301 is a pixel point, and the pixel points in each single-frame image are only used for example and have nothing to do with the actual number of pixel points. The single-frame image is divided into X*Y pixel points according to the resolution of the single-frame image, referring to Figure 4 .

[0141] In order to ensure the accuracy of subsequent processing, the resolution of each single frame image is the same, so that the number of pixels in each single frame image can be the same, thereby ensuring the accuracy of processing pixels at the same position in each single frame image. It should be noted that the operation of sampling the video stream to obtain a single frame image can be completed by the road marking determination device, or it can be completed by the server and then each single frame image is sent to the road marking determination device. This is just an example and does not form a specific limitation.

[0142] In S202, the resolution of each single frame image is the same, so the number of pixels in each single frame image is the same, where the same position means, for example, taking the pixel a at the (x, y)th position in each single frame image. (x,y) When x and y are 1 respectively, it indicates that the pixels at 1*1 in each single frame image are grouped. x is an integer between 1 and X, y is an integer between 1 and Y, X is the number of pixels in the row direction, Y is the number of pixels in the column direction, and X*Y is the number of pixels in each single frame image.

[0143] For each pixel point at the same position in each single-frame image, each pixel point is grouped to obtain each pixel group; wherein the principle of grouping is to ensure that the difference between the pixel value of each pixel point in each pixel group and the determined reference pixel value is within a preset range.

[0144] Specifically, the grouping process is achieved in the following way:

[0145] A reference pixel is determined among the pixels at the same position in each single-frame image; wherein the pixel value of the reference pixel is the reference pixel value; the pixel value of each pixel is compared with the reference pixel value in turn, and the pixels whose absolute values ​​of the differences with the reference pixel value are within a preset range and the reference pixel are determined as a pixel group; if there are ungrouped pixels, the reference pixel is re-determined among the ungrouped pixels, and all the ungrouped pixels are traversed according to the reference pixel values ​​of the re-determined reference pixel until the pixel groups at the same position in each single-frame image are completed.

[0146] Exemplarily, taking the pixel point at 1*1 as an example, the pixel point at 1*1 in each single-frame image can be expressed as a total of C pixels. Among these C pixels, a reference pixel is determined, and the pixel value of the reference pixel is the reference pixel value. Among them, any point can be selected as the reference pixel, and the reference pixel can also be determined according to the first order rule. Specifically, for each pixel point at the same position in each single-frame image, the order of the single-frame image to which the pixel point belongs in the video stream (for example, there are a total of C single-frame images, and the single-frame image to which a certain pixel point belongs is the third) is determined as the order of the pixel point in each pixel point; according to the order of each pixel point in each pixel point, the pixel point determined according to the first order rule is used as the reference pixel point. In this specific example, the first order rule is, for example, to select a pixel point at a determined position in the first single-frame image as the reference pixel point.

[0147] For example, determine the pixel at 1*1 in the first single frame image as the reference pixel, expressed as a (1,1) , compare the pixel values ​​of the pixels at 1*1 in the 1st to Cth single frame images with the reference pixel value in turn, and determine the absolute value of the difference between the two. If the absolute value is within a preset range, the pixels whose absolute values ​​of the differences are within the preset range and the reference pixel are divided into a pixel group.

[0148] In the above embodiment, in the process of determining the reference pixel point, the order of the pixel point among all the pixel points is taken into consideration, and the reference pixel point is determined according to the first order rule related to the order, which is more flexible and less likely to cause omission of pixel points.

[0149] In the above process, for example, among C pixels, C / 3 of the pixels and the reference pixel are grouped into a pixel group, and the remaining C / 2 pixels are not grouped. At this time, among the remaining ungrouped pixels, the reference pixel is re-determined, and according to the reference pixel value of the re-determined reference pixel, all the ungrouped pixels are traversed until the pixel points at the same position in each single-frame image are grouped.

[0150] Exemplarily, the principle of redetermining the reference pixel point is the same as the principle of determining the pixel point as described above, with the difference that the reference pixel point is redetermined among all the ungrouped pixels. Specifically, for each pixel point at the same position in each single-frame image, the order of the single-frame image to which the ungrouped pixel point belongs in the video stream is determined to be the order of the ungrouped pixel point among all the ungrouped pixel points; and according to the order of each ungrouped pixel point among all the ungrouped pixel points, the ungrouped pixel point determined according to the second order rule is redetermined as the reference pixel point. In this specific example, the second order rule is, for example, to select a pixel point at a determined position in the first single-frame image as the reference pixel point.

[0151] In the above embodiment, in the process of determining the reference pixel point, the order of the ungrouped pixel points among all the ungrouped pixel points is taken into consideration, and the reference pixel point is determined according to a second order rule related to the order, which is more flexible and less likely to cause omission of pixels.

[0152] In actual application, usually, the pixel value includes the pixel value of the R channel, the pixel value of the G channel and the pixel value of the B channel. Therefore, the benchmark pixel value includes the benchmark pixel value of the R channel, the benchmark pixel value of the G channel and the benchmark pixel value of the B channel; the corresponding preset range includes the R channel preset range, the G channel preset range and the B channel preset range.

[0153] In this way, in the process of comparing pixel values, the pixel values ​​can be compared by channel to complete the grouping of each pixel point in each single frame image. Specifically, the pixel value of the R channel of each pixel point is compared with the reference pixel value of the R channel to obtain the difference △R of the R channel pixel value; the pixel value of the G channel of each pixel point is compared with the reference pixel value of the G channel to obtain the difference △G of the G channel pixel value; the pixel value of the B channel of each pixel point is compared with the reference pixel value of the B channel to obtain the difference △B of the B channel pixel value; if the absolute value of the difference of the R channel pixel value |△R| is within the preset range of the R channel, the absolute value of the difference of the G channel pixel value |△G| is within the preset range of the G channel, and the absolute value of the difference of the B channel pixel value |△B| is within the preset range of the B channel, then the pixels participating in the comparison and the reference pixel point are determined to be a pixel group. Among them, the preset range of the R channel, the preset range of the G channel and the preset range of the B channel are pre-set according to actual needs (such as different needs of different scenes), and take values ​​between 0-255, such as 10, 10 and 15 respectively.

[0154] The above is an example of the pixel points at 1*1 of C single-frame images to illustrate the grouping process. In actual application, pixel points at other positions are also grouped in the same way. In this way, each pixel point at the same position is grouped to obtain each pixel group.

[0155] In a specific example, C is 15, that is, there are 15 single-frame images sampled from the video stream, and 15 pixels at 1*1 in the 15 single-frame images are grouped, and the 15 pixels are represented as a 1(1,1) 、a 2(1,1) 、a 3(1,1) 、a 4(1,1) 、a 5(1,1) 、a 6(1,1) 、a 7(1,1) 、a 8(1,1) 、a 9(1,1) 、a10(1,1) 、a 11(1,1) 、a 12(1,1) 、a 13(1,1) 、a 14(1,1) and a 15(1,1) . 1(1,1) , Ga 1(1,1) and Ba 1(1,1) The pixel values ​​of the R, G, and B channels of the pixel point are represented respectively. In the grouping process of this example, |△R2|=|Ra is determined as the reference pixel value. 2(1,1) -Ra 1(1,1) |、|△G2|=|Ga 2(1,1) -Ga 1(1,1) | and |△B2|=|Ba 2(1,1) -Ba 1(1,1) |, and so on, the pixel difference between each pixel and the reference pixel is calculated respectively, and then the absolute value is compared with the preset range of the corresponding channel.

[0156] In this example, after the first round of grouping, determine a 1(1,1) 、a 2(1,1) 、a 4(1,1) 、a 10(1,1) As a group, next, among the 11 ungrouped pixels, a 3(1,1) Re-determine the reference pixel point, and continue to group according to the above comparison logic until all pixel points are grouped.

[0157] In S203, the target pixel value at the position corresponding to the target pixel group is determined by applying the pixel value of each pixel point in the target pixel group; wherein the target pixel group is the pixel group including the largest number of pixel points among the pixel groups.

[0158] Specifically, for C single-frame images, all 1*1, 1*2, 1*3, ..., X*Y pixels in each single-frame image are grouped. Since the operation at each position is the same, the operation of each pixel group obtained after grouping C pixels at any position (such as 4*9) is described below. Figure 5 , shows a schematic diagram of a grouping process, C takes 15 as an example, and the pixel point at 4*9 is represented by 501. Among them, the number of groups is represented by 2. In the actual application process, the number of groups is usually related to the length of the cycle of collecting video streams and the number of single-frame images obtained by sampling. For example, taking a certain pixel as an example, in the entire cycle, there are two situations, including the road marking line being blocked by vehicles and not blocked. One possibility is that there are relatively few vehicles passing through in this cycle, and the color of the car is relatively single, then the number of possible groups is two; another possibility is that there are relatively many vehicles passing through in this cycle, and the color of the car is relatively many, then the number of possible groups is multiple, but usually, the number of groups is 3 or 4.

[0159] For example, C takes 15 as an example, that is, there are 15 single-frame images sampled from the video stream, and the 15 pixels at 4*9 in the 15 single-frame images are divided into 2 groups, and the 15 pixels are represented by a 1(4,9) 、a 2(4,9) 、a 3(4,9) 、a 4(4,9) 、a 5(4,9) 、a 6(4,9) 、a 7(4,9) 、a 8(4,9) 、a 9(4,9) 、a 10(4,9) 、a 11(4,9) 、a 12(4,9) 、a 13(4,9) 、a 14(4,9) 、a 15(4,9) . refer to Figure 6 , the grouping is the first group A group a 1(4,9) 、a 2(4,9) 、a 3(4,9) 、a 4(4,9) 、a 5(4,9) 、a 6(4,9) 、a 7(4,9) 、a 11(4,9) 、a 12(4,9) 、a 13(4,9) 、a 15(4,9) ; Group 2a 8(4,9) 、a 9(4,9) 、a 10(4,9) 、a 14(4,9) . It can be seen that the number of pixels in group A is 11 (for example, all corresponding to road markings), and the number of pixels in group B is 4 (all corresponding to the vehicle body). The pixel group A, which includes the largest number of pixels, is taken as the target pixel group; then the target pixel value at the position is determined according to the values ​​of the aforementioned 11 pixels in group A, and the target pixel value is recorded as the pixel value at x*y in the final target frame image.

[0160] Next, it is explained how to use the pixel values ​​of each pixel point in the target pixel group to determine the target pixel value at the position corresponding to the target pixel group. Specifically, for each pixel point in the target pixel group, a color value is determined according to the pixel value of the pixel point; a mode value is determined based on each color value, and the pixel value of the pixel point corresponding to the mode value is determined as the target pixel value at the position corresponding to the target pixel group; wherein the mode value is the color value with the most identical color values ​​in the target pixel group.

[0161] For example, the target pixel group is group A, which includes the following 11 pixel points: 1(4,9) 、a 2(4,9) 、a 3(4,9) 、a 4(4,9)、a 5(4,9) 、a 6(4,9) 、a 7(4,9) 、a 11(4,9) 、a 12(4,9) 、a (4,9) 、a 15(4,9) , the pixel values ​​of these 11 pixels are processed to obtain 11 color values, and then based on the color values, the color value with the most identical color values ​​in the target pixel group is determined as the majority value, and the pixel value of the pixel corresponding to the majority value is determined as the target pixel value at the corresponding position of the target pixel group. In a specific example, the color values ​​corresponding to these 11 pixels are Y2, Y2, Y2, Y2, Y2, Y2, Y1, Y3, Y3, Y1, Y2, Y2, Y1, Y3, Y3, and the majority value is Y2, so the pixel a corresponding to Y2 is determined 1(4,9) 、a 2(4,9) 、a 3(4,9) 、a 4(4,9) 、a 5(4,9) 、a 6(4,9) 、a 11(4,9) 、a 12(4,9) , the pixel value (the color values ​​of these 8 pixels are the same, indicating that their pixel values ​​are also the same) is the target pixel value at that position.

[0162] The color value of each pixel can be determined as follows:

[0163] Since the pixel value includes the pixel value of the R channel, the pixel value of the G channel and the pixel value of the B channel, for each pixel point in the target pixel group, the color value of the pixel point is determined according to the pixel value of the R channel, the pixel value of the G channel and the pixel value of the B channel of the pixel point, as well as the weight parameters corresponding to each channel.

[0164] In a specific example, for example, the weight parameter corresponding to the R channel is 65536, the weight parameter corresponding to the G channel is 256, and the weight parameter corresponding to the B channel is 1, then the color value can be calculated according to Yk=65536*R+256*G+B, where k is the number of pixels in the target pixel group. In the above example, k is 4.

[0165] In the process of determining the target pixel value, there is also a situation where the mode value cannot be determined based on each color value. In this case, the target pixel value is determined according to the pixel value of each pixel point in the target pixel group. For example, still taking the 5 pixels in the above target pixel group as an example, the corresponding color values ​​are Y1, Y2, Y3, Y4, and Y5, that is, 5 different color values ​​are obtained, indicating that there is no mode value.

[0166] In this case, the number of pixels in the target pixel group is obtained; a first average value of the pixel value of the R channel of each pixel point, a second average value of the pixel value of the G channel of each pixel point, and a third average value of the pixel value of the B channel of each pixel point are determined; and the target pixel value is determined based on the first average value, the second average value, and the third average value.

[0167] In the above embodiment, when the pixel values ​​in a target pixel group are different, it is more accurate to respectively calculate the average values ​​of the pixel values ​​on each channel and then combine the average values ​​into the target pixel value.

[0168] For example, the number of pixels in the target pixel group in this example is 5, and the first average value Ravg of the pixel value of the R channel of each pixel point is determined; the second average value Gavg of the pixel value of the G channel of each pixel point, and the third average value Bavg of the pixel value of the B channel of each pixel point are determined; the target pixel value is determined according to the first average value Ravg, the second average value Gavg and the third average value Bavg, referring to the following formula:

[0169] Ravg=(Ra 3(1,1) +Ra 5(1,1) +Ra 6(1,1) +Ra 7(1,1) +Ra 11(1,1) ) / 5;

[0170] Gavg=(Ga 3(1,1) +Ga 5(1,1) +Ga 6(1,1) +Ga 7(1,1) +Ga 11(1,1) ) / 5;

[0171] Bavg=(Ba 3(1,1) +Ba 5(1,1) +Ba 6(1,1) +Ba 7(1,1) +Ba 11(1,1) ) / 5.

[0172] It should be noted that, in the process of determining the target pixel value, it is also possible not to calculate the color value of each pixel point, and directly search whether there are pixels with the same pixel value in the current target pixel group, and then determine their number. However, since the pixel value is divided into three channels of RGB, this method requires the three channels to be calculated and compared separately, which requires a lot of calculation. In the above method of calculating the color value, for each pixel point, since the color value is calculated based on the pixel point and then compared, the amount of calculation is saved.

[0173] The above is explained by taking the pixel point at 1*1 as an example, and the calculation method of the pixel points at other positions is the same. Therefore, through the above embodiment, the target pixel value at each position (a total of X*Y pixel points, that is, X*Y positions) can be obtained.

[0174] In S204, since the image is composed of various pixels, the pixels corresponding to the target pixel values ​​obtained at various positions are combined into a target frame image. In this way, the obtained target image not only includes the road markings, but also removes the obstructions.

[0175] In S205, after obtaining the target frame image, the target image frame can be processed to obtain the wear state of the road markings included in the target frame image.

[0176] Exemplarily, the first method is to perform edge processing on the target frame image to determine the wear status of the road markings included in the target frame image; the second method is to determine the type of each road marking included in the target frame image, and match the road markings with standard road markings of corresponding types to determine the wear status of the road markings.

[0177] In the first method, when the road marking is worn, the pixel values ​​of the worn part and the surrounding pixels may be quite different. Taking advantage of the fact that the edge in the image is where the pixel value changes, the edge processing method is applied to mark the pixel points where the pixel value changes, and then determine the worn part of the road marking, and then determine the degree of wear based on the difference between the pixel values ​​of the worn part and the surrounding pixels.

[0178] In the second method, the types of road markings included in the target frame image are determined, and the types mainly include instruction markings, prohibition markings, and warning markings, etc. For example, each type of road marking corresponds to a standard road marking, and the size of the standard road marking is known. Therefore, after determining the types of road markings included in the target frame image, they can be matched with the corresponding type of standard road markings. In this way, the wear state of the road markings can be determined according to the matching results. In a specific example, during the matching process, the image overlap ratio of the current road marking and the corresponding standard road marking is determined, and the overlap ratio is used as the wear degree.

[0179] In order to make the technical solution of the present application more complete, a complete flowchart is used below to illustrate the embodiment of the present application. Figure 7 .

[0180] S701: Determine a plurality of single-frame images obtained by sampling a video stream including road markings captured by a video capture device.

[0181] Among them, the resolution of each single frame image is the same.

[0182] S702: Determine a reference pixel point among the pixels at the same position in each single-frame image.

[0183] The pixel value of the reference pixel point is the reference pixel value.

[0184] S703 , comparing the pixel value of each pixel point with the reference pixel value in turn, and determining each pixel point whose absolute value of difference with the reference pixel value is within a preset range and the reference pixel point as a pixel group.

[0185] S704. If there are ungrouped pixels, redefine the reference pixel among the ungrouped pixels, and traverse all the ungrouped pixels according to the reference pixel value of the redetermined reference pixel until the pixel points at the same position in each single-frame image are grouped.

[0186] S705. For each pixel in the target pixel group, determine the color value of the pixel according to the pixel value of the R channel, the pixel value of the R channel and the pixel value of the B channel, and the weight parameters corresponding to each channel.

[0187] S7061. Determine a mode value based on each color value, and determine the pixel value of the pixel point corresponding to the mode value as the target pixel value at the position corresponding to the target pixel group; wherein the mode value is the color value with the most identical color values ​​in the target pixel group.

[0188] S7062. If the mode value cannot be determined based on each color value, obtain the number of pixels in the target pixel group; determine a first average value of the pixel value of the R channel of each pixel, a second average value of the pixel value of the G channel of each pixel, and a third average value of the pixel value of the B channel of each pixel; determine the target pixel value based on the first average value, the second average value and the third average value.

[0189] S707 , composing a target frame image with pixel points corresponding to each target pixel value obtained at each position.

[0190] S708: Determine the wear state of the road marking included in the target frame image according to the target frame image.

[0191] In summary, in order to be able to analyze the situation of road markings at any time, the embodiment of the present application converts frame images according to the intercepted video stream based on the static characteristics of road markings in the video stream, while vehicles and pedestrians have dynamic characteristics, and obtains pixel points that meet the preset range through probability statistics based on the color characteristics of the pixel point RGB value, and finally synthesizes and outputs the completed frame image of unobstructed road markings. There is no need to wait for the intersection without cars and pedestrians to obtain video resources of road markings, avoiding unnecessary waiting time. Compared with traditional methods, it is more efficient and convenient, saving unnecessary waiting time, and providing a complete, unobstructed picture of intersection facilities. The wear state of the road markings is then determined based on the picture, and the worn road markings are processed to reduce the accident rate and increase the success rate of illegal capture.

[0192] See also Figure 8 , shows a processing effect after applying the road marking determination method in an embodiment of the present application. Figure 8 In the figure, the number of single-frame images is 15, and the pixel point processed in each single-frame image is taken as the pixel point at position 81. Referring to the above grouping principle, the 1st, 2nd, 3rd, 4th, 5th, 6th, 7th, 11th, 12th, 13th, and 15th images are group A, and the 8th, 9th, 10th, and 14th images are group B. It can be seen that in the processed images, the road markings are clearly and completely displayed without being blocked by vehicles or pedestrians.

[0193] In actual application, in order to improve the processing speed and ensure the clarity of the road markings, a shorter period video stream or a smaller number of single-frame images is usually used to determine the target frame image including clear and complete road markings. However, a situation may also occur where, if the number of single-frame images is too small, the target frame image may not be able to determine the wear state of the road markings. In this case, this problem can be solved by increasing the number of single-frame images obtained by sampling, that is, after increasing the number of single-frame images, the target frame image is re-determined based on each single-frame image; and the wear state of the road markings is determined based on the re-determined target frame image.

[0194] refer to Fig. 9 , Fig. 9 The number of single-frame images is 30, through Fig. 9 It can be seen that in the processed image, the road markings are clearly and completely displayed without being blocked by vehicles or pedestrians.

[0195] Specifically, the number of single-frame images obtained by sampling can be increased in the following ways:

[0196] The first method is to increase the sampling rate when sampling the video stream including the road markings captured by the video capture device. In this method, the video stream remains unchanged, the sampling rate is increased, and the number of single-frame images is increased.

[0197] The second method is to reacquire the video stream, resample the reacquired video stream, and superimpose the resampled single-frame image with the initial single-frame image to increase the number of single-frame images. In this method, the video stream is reacquired, resampled, and the resampled single-frame image and the initial single-frame image are used together as the re-determined single-frame image to increase the number of single-frame images.

[0198] In this way, only when the determined target frame image cannot determine the wear state of the road marking, the target frame image is re-determined by increasing the number of single frame images obtained by sampling, and then the state of the road marking is re-determined. In this way, whether the state of the road marking can be determined is determined by determining whether the number of single frame images needs to be increased to re-determine the target frame image, without using too many single frame images each time, which saves data processing volume and improves data processing speed on the one hand, and is more flexible on the other hand.

[0199] like Fig.10 As shown, based on the same inventive concept, an embodiment of the present invention provides a device for determining road markings, which includes a video processing module 1001, a grouping module 1002, a target pixel value determination module 1003, an image generation module 1004 and a first wear state determination module 1005.

[0200] The video processing module 1001 is used to determine a plurality of single-frame images obtained by sampling a video stream including road markings captured by a video capture device; wherein each single-frame image has the same resolution;

[0201] The grouping module 1002 is used to group the pixels at the same position in each single-frame image to obtain pixel groups; wherein the difference between the pixel value of each pixel in each pixel group and the determined reference pixel value is within a preset range;

[0202] The target pixel value determination module 1003 is used to determine the target pixel value at the position corresponding to the target pixel group by using the pixel value of each pixel point in the target pixel group; wherein the target pixel group is the pixel group including the largest number of pixel points in each pixel group;

[0203] An image generation module 1004 is used to form a target frame image by combining pixel points corresponding to each target pixel value obtained at each position;

[0204] The first wear state determining module 1005 is configured to determine the wear state of the road marking included in the target frame image according to the target frame image.

[0205] In some exemplary embodiments, a second wear state determination module is further included, which is used to increase the number of single frame images obtained by sampling when the determined target frame image cannot determine the wear state of the road marking after the pixel points corresponding to the target pixel values ​​obtained at the respective positions form a target frame image;

[0206] Re-determine the target frame image according to each single frame image;

[0207] The wear state of the road marking is determined according to the re-determined target frame image.

[0208] In some exemplary embodiments, the second wear state determination module is specifically configured to increase the number of single-frame images obtained by sampling in the following manner:

[0209] Increase the sampling rate when sampling the video stream including road markings captured by the video capture device; or

[0210] The video stream is reacquired, and the reacquired video stream is resampled, and the resampled single-frame image is superimposed with the initial single-frame image to obtain the number of single-frame images.

[0211] In some exemplary implementations, the first wear state determination module 1005 is specifically configured to:

[0212] performing edge processing on the target frame image to determine the wear state of the road marking included in the target frame image; or

[0213] The type of each road marking included in the target frame image is determined, and the road marking is matched with a standard road marking of a corresponding type to determine a wear state of the road marking.

[0214] In some exemplary implementations, the grouping module 1002 is specifically configured to:

[0215] Determine a reference pixel point among the pixels at the same position in each single-frame image; wherein the pixel value of the reference pixel point is the reference pixel value;

[0216] Compare the pixel value of each pixel point with the reference pixel value in turn, and determine that each pixel point whose absolute value of difference with the reference pixel value is within a preset range and the reference pixel point are a pixel group;

[0217] If there are ungrouped pixels, the reference pixels are re-determined among the ungrouped pixels, and all the ungrouped pixels are traversed according to the reference pixel values ​​of the re-determined reference pixels until the pixel points at the same position in each single-frame image are grouped.

[0218] In some exemplary embodiments, the grouping module 1002 is specifically configured to:

[0219] Comparing the pixel value of each pixel point with the reference pixel value in turn, determining each pixel point whose absolute value of difference with the reference pixel value is within a preset range and the reference pixel point as a pixel group, including:

[0220] The pixel value of the R channel of each pixel point is compared with the reference pixel value of the R channel to obtain the difference of the pixel value of the R channel; the pixel value of the G channel of each pixel point is compared with the reference pixel value of the G channel to obtain the difference of the pixel value of the G channel; the pixel value of the B channel of each pixel point is compared with the reference pixel value of the B channel to obtain the difference of the pixel value of the B channel;

[0221] If the absolute value of the difference in the R channel pixel values ​​is within the R channel preset range, the absolute value of the difference in the G channel pixel values ​​is within the G channel preset range, and the absolute value of the difference in the B channel pixel values ​​is within the B channel preset range, then the pixel points involved in the comparison and the reference pixel point are determined to be a pixel group.

[0222] In some exemplary implementations, the target pixel value determination module 1003 is specifically configured to:

[0223] For each pixel in the target pixel group, a color value is determined according to the pixel value of the pixel;

[0224] A mode value is determined based on each color value, and the pixel value of the pixel point corresponding to the mode value is determined as the target pixel value at the position corresponding to the target pixel group; wherein the mode value is the color value with the most identical color values ​​in the target pixel group.

[0225] In some exemplary embodiments, the pixel value includes a pixel value of an R channel, a pixel value of an R channel, and a pixel value of a B channel;

[0226] The target pixel value determination module 1003 is specifically used for:

[0227] For each pixel in the target pixel group, the color value of the pixel is determined according to the pixel value of the R channel, the pixel value of the R channel and the pixel value of the B channel, and the weight parameters corresponding to each channel.

[0228] In some exemplary embodiments, the target pixel value determination module 1003 is further configured to determine the target pixel value according to the pixel value of each pixel point in the target pixel group when the mode value cannot be determined based on each color value.

[0229] Since the device is the device in the method in the embodiment of the present invention, and the principle of solving the problem by the device is similar to that of the method, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0230] like Fig.11 As shown, based on the same inventive concept, an embodiment of the present invention provides a road marking determination device, which includes: a processor 1101 and a data acquisition unit 1102.

[0231] The data acquisition unit 1102 is configured to perform:

[0232] Obtaining a video stream including road markings captured by a video capture device;

[0233] The processor 1101 is configured to execute:

[0234] Determine a plurality of single-frame images obtained by sampling a video stream including road markings captured by a video capture device; wherein each single-frame image has the same resolution;

[0235] For each pixel point at the same position in each single-frame image, each pixel point is grouped to obtain each pixel group; wherein the difference between the pixel value of each pixel point in each pixel group and the determined reference pixel value is within a preset range;

[0236] Determine the target pixel value at the position corresponding to the target pixel group by using the pixel value of each pixel point in the target pixel group; wherein the target pixel group is the pixel group including the largest number of pixel points in each pixel group;

[0237] The pixel points corresponding to each target pixel value obtained at each position form a target frame image;

[0238] A wear state of a road marking included in the target frame image is determined based on the target frame image.

[0239] In some exemplary embodiments, after the pixel points corresponding to the target pixel values ​​obtained at the respective positions are combined into a target frame image, the processor 1101 is further configured to execute:

[0240] If the determined target frame image cannot determine the wear state of the road marking, then the number of single frame images obtained by sampling is increased;

[0241] Re-determine the target frame image according to each single frame image;

[0242] The wear state of the road marking is determined according to the re-determined target frame image.

[0243] In some exemplary embodiments, the processor 1101 is configured to increase the number of single-frame images obtained by sampling in the following manner:

[0244] Increase the sampling rate when sampling the video stream including road markings captured by the video capture device; or

[0245] The video stream is reacquired, and the reacquired video stream is resampled, and the resampled single-frame image is superimposed with the initial single-frame image to obtain the number of single-frame images.

[0246] In some exemplary embodiments, the processor 1101 is configured to execute:

[0247] Performing edge processing on the target frame image to determine the wear state of the road marking included in the target frame image; or

[0248] The type of each road marking included in the target frame image is determined, and the road marking is matched with a standard road marking of a corresponding type to determine the wear state of the road marking.

[0249] In some exemplary embodiments, the processor 1101 is configured to execute:

[0250] Determine a reference pixel point among the pixels at the same position in each single-frame image; wherein the pixel value of the reference pixel point is the reference pixel value;

[0251] Compare the pixel value of each pixel point with the reference pixel value in turn, and determine that each pixel point whose absolute value of difference with the reference pixel value is within a preset range and the reference pixel point are a pixel group;

[0252] If there are ungrouped pixels, the reference pixels are re-determined among the ungrouped pixels, and all the ungrouped pixels are traversed according to the reference pixel values ​​of the re-determined reference pixels until the pixel points at the same position in each single-frame image are grouped.

[0253] In some exemplary embodiments, the reference pixel value includes a reference pixel value of an R channel, a reference pixel value of a G channel, and a reference pixel value of a B channel; the preset range includes a preset range of an R channel, a preset range of a G channel, and a preset range of a B channel;

[0254] The processor 1101 is configured to execute:

[0255] The pixel value of the R channel of each pixel point is compared with the reference pixel value of the R channel to obtain the difference of the pixel value of the R channel; the pixel value of the G channel of each pixel point is compared with the reference pixel value of the G channel to obtain the difference of the pixel value of the G channel; the pixel value of the B channel of each pixel point is compared with the reference pixel value of the B channel to obtain the difference of the pixel value of the B channel;

[0256] If the absolute value of the difference in the R channel pixel values ​​is within the R channel preset range, the absolute value of the difference in the G channel pixel values ​​is within the G channel preset range, and the absolute value of the difference in the B channel pixel values ​​is within the B channel preset range, then the pixel points involved in the comparison and the reference pixel point are determined to be a pixel group.

[0257] In some exemplary embodiments, the processor 1101 is configured to execute:

[0258] For each pixel in the target pixel group, a color value is determined according to the pixel value of the pixel;

[0259] A mode value is determined based on each color value, and the pixel value of the pixel point corresponding to the mode value is determined as the target pixel value at the position corresponding to the target pixel group; wherein the mode value is the color value with the most identical color values ​​in the target pixel group.

[0260] In some exemplary embodiments, the processor 1101 is specifically configured to execute;

[0261] For each pixel in the target pixel group, a color value is determined according to the pixel value of the pixel, including:

[0262] For each pixel in the target pixel group, the color value of the pixel is determined according to the pixel value of the R channel, the pixel value of the R channel and the pixel value of the B channel, and the weight parameters corresponding to each channel.

[0263] In some exemplary embodiments, if the majority value cannot be determined based on each color value, the processor 1101 is further configured to execute:

[0264] The target pixel value is determined according to the pixel value of each pixel point in the target pixel group.

[0265] An embodiment of the present invention further provides a computer storage medium, in which computer program instructions are stored. When the instructions are executed on a computer, the computer executes the steps of the above-mentioned network configuration method for electronic home devices.

[0266] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0267] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0268] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0269] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0270] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A road marking determination device, characterized in that: Including data acquisition unit and processor: The data acquisition unit is configured to perform: Obtaining a video stream including road markings captured by a video capture device; The processor is configured to perform: Determine a plurality of single-frame images obtained by sampling a video stream including road markings captured by the video capture device; wherein each single-frame image has the same resolution; Determine a reference pixel point among the pixels at the same position in each single-frame image; wherein the pixel value of the reference pixel point is the reference pixel value; Comparing the pixel value of each pixel point with the reference pixel value in turn, determining that each pixel point whose absolute value of difference with the reference pixel value is within a preset range and the reference pixel point are a pixel group; wherein the difference between the pixel value of each pixel point in each pixel group and the determined reference pixel value is within the preset range; If there are ungrouped pixels, redetermine the reference pixel among the ungrouped pixels, and traverse all the ungrouped pixels according to the reference pixel value of the redetermined reference pixel until the pixel points at the same position in each single-frame image are grouped; Determine the target pixel value at the position corresponding to the target pixel group by using the pixel value of each pixel point in the target pixel group; wherein the target pixel group is the pixel group including the largest number of pixel points among the pixel groups; The pixel points corresponding to each target pixel value obtained at each position form a target frame image; The wear state of the road marking included in the target frame image is determined according to the target frame image.

2. The device according to claim 1, characterized in that After the pixel points corresponding to the target pixel values ​​obtained at the respective positions are combined into a target frame image, the processor is further configured to execute: If the determined target frame image cannot determine the wear state of the road marking, increasing the number of single frame images obtained by sampling; Re-determine the target frame image according to each single frame image; The wear state of the road marking is determined according to the re-determined target frame image.

3. The device according to claim 2, characterized in that The processor is configured to increase the number of single-frame images obtained by sampling in the following manner: Increase the sampling rate when sampling the video stream including road markings captured by the video capture device; or The video stream is reacquired, and the reacquired video stream is resampled, and the resampled single-frame image is superimposed with the initial single-frame image to obtain the number of single-frame images.

4. The device according to claim 1, characterized in that The processor is configured to perform: performing edge processing on the target frame image to determine the wear state of the road marking included in the target frame image; or The type of each road marking included in the target frame image is determined, and the road marking is matched with a standard road marking of a corresponding type to determine a wear state of the road marking.

5. The device according to claim 1, characterized in that The reference pixel value includes a reference pixel value of an R channel, a reference pixel value of a G channel, and a reference pixel value of a B channel; the preset range includes a preset range of an R channel, a preset range of a G channel, and a preset range of a B channel; The processor is configured to perform: Compare the pixel value of the R channel of each pixel point with the reference pixel value of the R channel to obtain the difference of the R channel pixel value; Compare the pixel value of the G channel of each pixel point with the reference pixel value of the G channel to obtain the difference of the pixel value of the G channel; Compare the pixel value of the B channel of each pixel point with the reference pixel value of the B channel to obtain the difference of the pixel value of the B channel; If the absolute value of the difference in the R channel pixel values ​​is within the R channel preset range, the absolute value of the difference in the G channel pixel values ​​is within the G channel preset range, and the absolute value of the difference in the B channel pixel values ​​is within the B channel preset range, then it is determined that the pixel points involved in the comparison and the reference pixel point are a pixel group.

6. The device according to claim 1, characterized in that The processor is configured to perform: For each pixel in the target pixel group, determine a color value according to the pixel value of the pixel; A mode value is determined based on each color value, and the pixel value of the pixel point corresponding to the mode value is determined as the target pixel value at the position corresponding to the target pixel group; wherein the mode value is the color value with the most identical color values ​​in the target pixel group.

7. The device according to claim 6, characterized in that The processor is specifically configured to execute; The step of determining the color value of each pixel in the target pixel group according to the pixel value of the pixel includes: For each pixel in the target pixel group, the color value of the pixel is determined according to the pixel value of the R channel of the pixel, the pixel value of the R channel and the pixel value of the B channel, and the weight parameters corresponding to each channel.

8. The device according to claim 6, characterized in that If the majority value cannot be determined based on the individual color values, the processor is further configured to execute: The target pixel value is determined according to the pixel value of each pixel point in the target pixel group.

9. A method for determining road markings, characterized in that: include: Determine a plurality of single-frame images obtained by sampling a video stream including road markings captured by a video capture device; wherein each single-frame image has the same resolution; Determine a reference pixel point among the pixels at the same position in each single-frame image; wherein the pixel value of the reference pixel point is the reference pixel value; Comparing the pixel value of each pixel point with the reference pixel value in turn, determining that each pixel point whose absolute value of difference with the reference pixel value is within a preset range and the reference pixel point are a pixel group; wherein the difference between the pixel value of each pixel point in each pixel group and the determined reference pixel value is within the preset range; If there are ungrouped pixels, redetermine the reference pixel among the ungrouped pixels, and traverse all the ungrouped pixels according to the reference pixel value of the redetermined reference pixel until the pixel points at the same position in each single-frame image are grouped; Determine the target pixel value at the position corresponding to the target pixel group by using the pixel value of each pixel point in the target pixel group; wherein the target pixel group is the pixel group including the largest number of pixel points among the pixel groups; The pixel points corresponding to each target pixel value obtained at each position form a target frame image; The wear state of the road marking included in the target frame image is determined according to the target frame image.

Images