A method and device for determining road visibility
The road acquisition device determines the visibility mark position and conversion relationship in the image frame, calculates the road visibility, solves the problems of inaccurate detection and strict requirements on the lane line in the prior art, and achieves high-accurate road visibility detection .
Patent Information
- Application Number
- CN202210648992.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-09
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2042-06-09
AI Technical Summary
The prior art requires additional signage devices when detecting road visibility or fails to obtain accurate visible distances, and strictly require the inclination angle of the lane line, resulting in poor detection accuracy and reliability.
Image frames are collected through road acquisition equipment, visibility marks are determined, and horizontal and vertical transformation relationships are established based on the lane line and pixel row relationships in the image frame, and the visible distance is calculated to achieve accurate detection of road visibility without being affected by lanes Line inclination angle influences.
Accurate detection of road visibility can be achieved without additional devices, improving the accuracy and reliability of detection, simplifying parameter acquisition and reducing errors.
Smart Images

Figure CN115035485B_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present invention relate to the field of computer vision technology, and in particular, to a method, apparatus, computing device, and computer-readable storage medium for determining road visibility. Background Art
[0002] Road visibility seriously affects traffic safety. Therefore, if the road visibility can be judged in advance by road data collection equipment installed on the road and passing vehicles can be reminded, traffic accidents can be greatly avoided.
[0003] There are many methods for detecting visibility using video image information, but most of them require the use of additional signboard devices for detection; or they can only detect the approximate area of visibility and cannot obtain the precise visible distance, with poor accuracy and reliability; or they have strict requirements on specific implementation scenarios. For example, the road acquisition equipment needs to be placed in the center of the road, and the plane where the vertical field of view of the lane and the road acquisition equipment is located cannot have an inclination angle, otherwise the visibility error increases with the increase of the inclination angle of the lane. Summary of the invention
[0004] The embodiment of the present invention provides a method for determining road visibility, which does not require an additional sign device and improves the accuracy of determining road visibility.
[0005] In a first aspect, an embodiment of the present invention provides a method for determining road visibility, comprising:
[0006] Determine a visibility flag through at least one image frame collected by the road collection device within a collection period; the visibility flag is a position in the image frame used to characterize the visibility within the collection period;
[0007] For any image frame collected during the collection period, a horizontal conversion relationship of any pixel row in the image frame is determined according to a plurality of lane lines in the image frame; the horizontal conversion relationship is used to characterize a conversion relationship between a pixel distance and a physical distance on any pixel row in the image frame;
[0008] Determining a vertical conversion relationship according to the number of pixel rows of the image frame and the vertical field angle of the road acquisition device; the vertical conversion relationship is used to characterize the conversion relationship between the vertical field angle and the number of pixel rows;
[0009] The visible distance of the road collection device within the collection period is determined based on the visibility flag, the horizontal conversion relationship and the vertical conversion relationship.
[0010] According to the visibility flag, the vertical conversion relationship and the horizontal conversion relationship of the pixel row where the visibility flag is located, the visible distance of the road acquisition equipment during the acquisition period is determined, and the visibility is accurately detected. The method is simple, the detection can be completed without the help of additional equipment, and the parameters used are easy to obtain. And the visibility error is not affected by the inclination angle of the lane line.
[0011] Optionally, determining the visible distance of the road collection device within the collection period based on the visibility flag, the horizontal conversion relationship, and the vertical conversion relationship includes:
[0012] Constructing a first perpendicular foot from the visibility flag to the vertical symmetry axis of the image frame in the image frame; determining a first physical distance between the visibility flag and the first perpendicular foot according to a horizontal conversion relationship of a pixel row where the visibility flag is located;
[0013] Determine a second physical distance between the first vertical foot and a projection point of the road collection device on the ground based on the height of the road collection device from the ground, the pitch angle of the road collection device and the vertical conversion relationship;
[0014] The distance between the road acquisition device and the physical position of the visibility mark in the three-dimensional space is determined based on the first physical distance, the second physical distance and the height of the image acquisition device from the ground.
[0015] The visibility flag is introduced to the first vertical foot of the vertical symmetry axis of the image frame. The first vertical foot must be located on the vertical symmetry axis of the image frame. Therefore, the pitch angle, vertical conversion relationship and other parameters of the image acquisition device can be used. The second physical distance between the first vertical foot and the projection point of the road acquisition device on the ground can be determined in combination with the height of the image acquisition device from the ground. The first physical distance between the visibility flag and the first vertical foot can be obtained by the horizontal conversion relationship of the pixel row where the visibility flag is located. Therefore, the visible distance is determined according to the first physical distance, the second physical distance and the height of the road acquisition device from the ground. The geometric principle is applied, and the photographic imaging principle is used at the same time to achieve accurate detection of visibility. The method is simple, the detection can be completed without the aid of additional devices, and the parameters used are easy to obtain. And the visibility error is not affected by the inclination angle of the lane line.
[0016] Optionally, determining a second physical distance between the first vertical foot and a projection point of the road collection device on the ground based on the height of the road collection device from the ground, the pitch angle of the road collection device and the vertical conversion relationship includes:
[0017] Based on the location of the road collection device and the physical location of the first perpendicular foot in the three-dimensional space, construct a right triangle perpendicular to the plane where the road is located;
[0018] Determine a first angle corresponding to the number of pixel rows between the first perpendicular foot and the center point of the image frame according to the vertical conversion relationship;
[0019] Determine a second angle in the right triangle according to the pitch angle of the road collection device and the first angle;
[0020] A second physical distance between the first perpendicular foot and a projection point of the road collection device on the ground is determined according to the second angle and the height of the road collection device from the ground.
[0021] By using geometric principles to construct a right triangle in three-dimensional space, accurate visibility detection is achieved. The method is simple and can be completed without the help of additional equipment. The parameters used are easy to obtain. And the visibility error is not affected by the inclination angle of the lane line.
[0022] Optionally, determining a horizontal conversion relationship of any pixel row in the image frame according to a plurality of lane lines in the image frame includes:
[0023] Determining a horizontal inverse perspective image of the image frame according to a plurality of numbers of pixels between intersections of a plurality of lane lines and a plurality of pixel rows in the image frame;
[0024] For two intersection points of any two lane lines and any pixel row in the horizontal reverse perspective image, determine the physical distance between the two intersection points according to the physical lane widths of the two lane lines;
[0025] determining a pixel distance between the two intersection points in the image frame;
[0026] According to the physical distance between the two intersection points and the pixel distance between the two intersection points, the horizontal conversion relationship of the pixel rows where the two intersection points are located is determined.
[0027] According to the number of pixels between the intersections of the lane lines and the pixel rows in the image frame, the image frame is converted into a horizontal inverse perspective image. The obtained horizontal inverse perspective image eliminates the visual error of "near is big and far is small" in the horizontal direction. The lane lines that are parallel to each other in the three-dimensional space are also parallel to each other in the horizontal inverse perspective image. Therefore, the physical distance between the two intersections can be determined according to the physical lane width of the lane line, and the horizontal conversion relationship on the pixel row where the two intersections are located can be determined by using the physical distance and pixel distance between the two intersections.
[0028] Optionally, determining a horizontal inverse perspective image of the image frame according to a plurality of numbers of pixels between intersections of a plurality of lane lines and a plurality of pixel rows in the image frame comprises:
[0029] For a target pixel row in the image frame, determining a first number of pixels between intersections of any two lane lines and the target pixel row; determining a second number of pixels between intersections of any two lane lines and a reference pixel row; the target pixel row is any pixel row among the plurality of pixel rows; the reference pixel row is located above the target pixel row in the image frame;
[0030] Determine an inverse perspective coefficient of the target pixel row according to one or more first pixel quantities and one or more second pixel quantities;
[0031] A horizontal inverse perspective view of the image frame is determined according to inverse perspective coefficients of the plurality of target pixel rows and pixel coordinates on the plurality of target pixel rows.
[0032] Since the lane lines are parallel in three-dimensional space, according to the principle of perspective, the lane lines are not parallel in the image frame. Therefore, in the image frame, the number of first pixels between the intersection of any two lane lines and the target pixel row is different from the number of second pixels between any two lane lines and the reference pixel row. The inverse perspective coefficient of the target pixel row can be determined based on the difference in the number of pixels. The inverse perspective coefficients of multiple pixel rows can be obtained according to the same method, thereby making a horizontal inverse perspective view of the image frame.
[0033] Optionally, before determining the horizontal inverse perspective image of the image frame according to the number of pixels between the intersections of the plurality of lane lines and the plurality of pixel rows in the image frame, the method further includes:
[0034] Extending the plurality of lane lines detected in the image frame respectively, and determining whether the intersection points of the plurality of extension lines are the same point; if they are not the same point, re-determining a corrected intersection point according to the positions of the plurality of intersection points;
[0035] The positions of the multiple lane lines are corrected according to the corrected intersection point and the multiple lane lines.
[0036] If the lane lines are not strictly parallel to each other in three-dimensional space due to operational errors, or there are errors in lane line detection on image frames, resulting in the detected lane lines not being parallel to each other, then the determination of the horizontal inverse perspective will be inaccurate, and the subsequent determination of the horizontal transformation relationship will also be inaccurate, so the lane lines need to be parallel in three-dimensional space. If the lane lines are parallel in three-dimensional space, then the lane lines will intersect at one point in the image frame. In other words, when the intersection point is the same point, it means that the lane lines are parallel in three-dimensional space. Using this visual principle, the position of the lane lines can be corrected to ensure the accuracy of the subsequent determination of visibility.
[0037] Optionally, the vertical conversion relationship is determined in the following manner, including:
[0038] Acquiring a magnification of the image frame;
[0039] Determining the focal length corresponding to the image frame according to the corresponding relationship between the magnification and the focal length of the road acquisition device;
[0040] Determining a vertical field of view angle of the road acquisition device according to a focal length corresponding to the image frame and an image height of the road acquisition device;
[0041] The vertical conversion relationship is determined according to the vertical field of view angle and the number of pixel rows of the image frame.
[0042] The focal length of the image frame can be determined according to the magnification of the image frame, thereby determining the vertical field angle of the image acquisition device corresponding to the image frame. Since the vertical field angle has a certain relationship with the number of pixel rows of the image frame, the vertical conversion relationship can be determined.
[0043] Optionally, determining the visibility flag through at least one image frame collected by the road collection device within a collection period includes:
[0044] Acquire multiple image frames with the same magnification collected during a collection period through a road collection device;
[0045] Performing target tracking on the multiple image frames to determine the driving trajectories of the multiple targets;
[0046] If the driving trajectory of any target is close to the road collection device, the starting point of the driving trajectory of the target is extracted; if the driving trajectory of any target is far away from the road collection device, the end point of the driving trajectory of the target is extracted;
[0047] The visibility flag is determined according to the extracted multiple starting points and / or end points.
[0048] The positions of the visibility marking points determined in this way are more intelligent, automated and accurate.
[0049] In a second aspect, an embodiment of the present invention further provides a device for determining road visibility, comprising:
[0050] Identify units for:
[0051] Determine a visibility flag through at least one image frame collected by the road collection device within a collection period; the visibility flag is a position in the image frame used to characterize the visibility within the collection period;
[0052] For any image frame collected during the collection period, a horizontal conversion relationship of any pixel row in the image frame is determined according to a plurality of lane lines in the image frame; the horizontal conversion relationship is used to characterize a conversion relationship between a pixel distance and a physical distance on any pixel row in the image frame;
[0053] Determining a vertical conversion relationship according to the number of pixel rows of the image frame and the vertical field angle of the road acquisition device; the vertical conversion relationship is used to characterize the conversion relationship between the vertical field angle and the number of pixel rows;
[0054] The visible distance of the road collection device within the collection period is determined based on the visibility flag, the horizontal conversion relationship and the vertical conversion relationship.
[0055] Optionally, the determining unit is specifically configured to:
[0056] Constructing a first perpendicular foot from the visibility flag to the vertical symmetry axis of the image frame in the image frame; determining a first physical distance between the visibility flag and the first perpendicular foot according to a horizontal conversion relationship of a pixel row where the visibility flag is located;
[0057] Determine a second physical distance between the first vertical foot and a projection point of the road collection device on the ground based on the height of the road collection device from the ground, the pitch angle of the road collection device and the vertical conversion relationship;
[0058] The distance between the road acquisition device and the physical position of the visibility mark in the three-dimensional space is determined based on the first physical distance, the second physical distance and the height of the image acquisition device from the ground.
[0059] Optionally, the determining unit is specifically configured to:
[0060] Based on the location of the road collection device and the physical location of the first perpendicular foot in the three-dimensional space, construct a right triangle perpendicular to the plane where the road is located;
[0061] Determine a first angle corresponding to the number of pixel rows between the first perpendicular foot and the center point of the image frame according to the vertical conversion relationship;
[0062] Determine a second angle in the right triangle according to the pitch angle of the road collection device and the first angle;
[0063] A second physical distance between the first perpendicular foot and a projection point of the road collection device on the ground is determined according to the second angle and the height of the road collection device from the ground.
[0064] Optionally, the determining unit is specifically configured to:
[0065] Determining a horizontal inverse perspective image of the image frame according to a plurality of numbers of pixels between intersections of a plurality of lane lines and a plurality of pixel rows in the image frame;
[0066] For two intersection points of any two lane lines and any pixel row in the horizontal reverse perspective image, determine the physical distance between the two intersection points according to the physical lane widths of the two lane lines;
[0067] determining a pixel distance between the two intersection points in the image frame;
[0068] According to the physical distance between the two intersection points and the pixel distance between the two intersection points, the horizontal conversion relationship of the pixel rows where the two intersection points are located is determined.
[0069] Optionally, the determining unit is specifically configured to:
[0070] For a target pixel row in the image frame, determining a first number of pixels between intersections of any two lane lines and the target pixel row; determining a second number of pixels between intersections of any two lane lines and a reference pixel row; the target pixel row is any pixel row among the plurality of pixel rows; the reference pixel row is located above the target pixel row in the image frame;
[0071] Determine an inverse perspective coefficient of the target pixel row according to one or more first pixel quantities and one or more second pixel quantities;
[0072] A horizontal inverse perspective view of the image frame is determined according to inverse perspective coefficients of the plurality of target pixel rows and pixel coordinates on the plurality of target pixel rows.
[0073] Optionally, the determining unit is further configured to:
[0074] Extending the plurality of lane lines detected in the image frame respectively, and determining whether the intersection points of the plurality of extension lines are the same point; if they are not the same point, re-determining a corrected intersection point according to the positions of the plurality of intersection points;
[0075] The positions of the multiple lane lines are corrected according to the corrected intersection point and the multiple lane lines.
[0076] Optionally, the determining unit is specifically configured to:
[0077] Acquiring a magnification of the image frame;
[0078] Determining the focal length corresponding to the image frame according to the corresponding relationship between the magnification and the focal length of the road acquisition device;
[0079] Determining a vertical field of view angle of the road acquisition device according to a focal length corresponding to the image frame and an image height of the road acquisition device;
[0080] The vertical conversion relationship is determined according to the vertical field of view angle and the number of pixel rows of the image frame.
[0081] Optionally, the determining unit is specifically configured to:
[0082] Acquire multiple image frames with the same magnification collected during a collection period through a road collection device;
[0083] Performing target tracking on the multiple image frames to determine the driving trajectories of the multiple targets;
[0084] If the driving trajectory of any target is close to the road collection device, the starting point of the driving trajectory of the target is extracted; if the driving trajectory of any target is far away from the road collection device, the end point of the driving trajectory of the target is extracted;
[0085] The visibility flag is determined according to the extracted multiple starting points and / or end points.
[0086] In a third aspect, an embodiment of the present invention further provides a computing device, including:
[0087] Memory for storing computer programs;
[0088] The processor is used to call the computer program stored in the memory and execute the road visibility determination method listed in any of the above-mentioned ways according to the obtained program.
[0089] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer-executable program, and the computer-executable program is used to enable a computer to execute the method for determining road visibility listed in any of the above-mentioned ways. BRIEF DESCRIPTION OF THE DRAWINGS
[0090] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0091] Figure 1a A schematic diagram of a system architecture provided for an embodiment of the present invention;
[0092] Figure 1b A schematic diagram of a possible setting relationship between a road data collection device 100 and a lane provided in an embodiment of the present invention;
[0093] Figure 1cA schematic diagram of a possible setting relationship between a road data collection device 100 and a lane provided in an embodiment of the present invention;
[0094] Figure 2 A schematic flow chart of a possible method for determining road visibility provided by an embodiment of the present invention;
[0095] Figure 3 A schematic diagram of determining a visibility flag provided by an embodiment of the present invention;
[0096] Figure 4a A schematic diagram of lane lines in a possible image frame provided by an embodiment of the present invention;
[0097] Figure 4b A schematic diagram of a possible redefinition of lane lines provided by an embodiment of the present invention;
[0098] Figure 5a A schematic diagram of an imaging area of a road acquisition device in three-dimensional space provided by an embodiment of the present invention;
[0099] Figure 5b A schematic diagram of any image frame collected by a road collection device provided by an embodiment of the present invention;
[0100] Figure 6a A schematic flow chart of a possible method for determining road visibility provided by an embodiment of the present invention;
[0101] Figure 6b A schematic diagram of a possible method of determining a first perpendicular foot provided by an embodiment of the present invention;
[0102] Figure 7 A schematic diagram of a possible method for determining a second physical distance provided by an embodiment of the present invention;
[0103] Figure 8 A schematic diagram of a possible method for determining a horizontal conversion relationship provided by an embodiment of the present invention;
[0104] Figure 9a A schematic diagram of a possible horizontal reverse perspective view provided by an embodiment of the present invention;
[0105] Figure 9b A schematic diagram of a possible method for determining a horizontal reverse perspective view provided by an embodiment of the present invention;
[0106] Fig.10 A schematic flow chart of a possible method for determining road visibility provided by an embodiment of the present invention;
[0107] Fig.11A schematic structural diagram of a possible device for determining road visibility provided by an embodiment of the present invention;
[0108] Fig.12 A schematic diagram of the structure of a possible computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0109] In order to make the purpose, implementation mode and advantages of the present application clearer, the exemplary implementation mode of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, rather than all the embodiments.
[0110] Based on the exemplary embodiments described in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the claims attached to this application. In addition, although the disclosure in this application is introduced according to one or several exemplary examples, it should be understood that each aspect of the disclosure can also constitute a complete implementation method separately.
[0111] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.
[0112] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise indicated. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, for example, they can be implemented according to orders other than those given in the diagrams or descriptions of the embodiments of this application.
[0113] In addition, the terms "include" and "have" and any variations thereof are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to those components expressly listed but may include other components not expressly listed or inherent to such products or devices.
[0114] Figure 1a A system architecture applicable to an embodiment of the present invention is exemplarily shown, and the system architecture includes a road collection device 100 and a server 200 .
[0115] The road collection device 100 is set above the road, takes images or videos of passing vehicles during the collection period, and uploads the taken images or videos to the server 200, so that the server 200 determines the visibility distance of the road collection device during the collection period based on the image information or video information.
[0116] There are various ways to set the collection period. One possible implementation method is to set each hour of the 24 hours of a day as a collection period. The road collection device 100 uploads the image or video taken in any hour to the server 200. After the server 200 analyzes the image or video, it determines the visible distance of the road within this hour. Another possible implementation method is to set a specific period of the day as the collection period. For example, the daily traffic peak 8:00-9:00 is set as the collection period. The road collection device 100 uploads the image or video taken from 8:00 to 9:00 to the server 200. After the server 200 analyzes the image or video, it determines the visible distance of the road within 8:00-9:00. Another possible implementation method is to set a weather recognition module. When it is recognized that the weather is bad weather that may affect visibility, the image or video collection mode is turned on. After the server 200 analyzes the image or video, it determines the visible distance of the road within the collection period. The bad weather includes but is not limited to rainy days, fog, snow and other weather. The above is only an example, and the embodiment of the present invention does not limit this.
[0117] The road data collection device 100 may be arranged in the center or on both sides of the road, and the field of view of the road data collection device 100 may cover the road. Figure 1b and Figure 1c A top view showing a possible setting relationship between the road acquisition device 100 and the lane provided by an embodiment of the present invention is shown. Figure 1b In the figure, the road collection device 100 is set in the center of the road, and the dotted line represents the field of view of the road collection device 100. Figure 1c In the figure, the road collection device 100 is arranged on the left side of the road, and the dotted line indicates the field of view of the road collection device 100. The above is only an example, and the arrangement position of the road collection device 100 is not limited to the two listed in the embodiment of the present invention.
[0118] Server 200 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), as well as big data and artificial intelligence platforms.
[0119] It is understandable that the functions of the server 200 can also be integrated in the road collection device 100, that is, the road collection device 100 has the functions of taking images and / or videos and analyzing and processing the taken images and / or videos to obtain the visible distance. The above is only an example, and the embodiment of the present invention does not limit this.
[0120] Figure 2 A possible method for determining road visibility provided by an embodiment of the present invention is shown, including:
[0121] Step 201, determining a visibility flag through at least one image frame collected by a road collection device within a collection period; the visibility flag is a position in the image frame used to characterize the visibility within the collection period.
[0122] Step 202, for any image frame collected during the collection period, determine the horizontal conversion relationship of any pixel row in the image frame according to the multiple lane lines in the image frame; the horizontal conversion relationship is used to characterize the conversion relationship between the pixel distance and the physical distance on any pixel row in the image frame.
[0123] Step 203, determining a vertical conversion relationship according to the number of pixel rows of the image frame and the vertical field angle of the road acquisition device; the vertical conversion relationship is used to characterize the conversion relationship between the vertical field angle and the number of pixel rows.
[0124] Step 204: determining the visible distance of the road collection device within the collection period based on the visibility flag, the horizontal conversion relationship, and the vertical conversion relationship.
[0125] In step 201, a visibility flag is determined by analyzing at least one image frame collected by a road collection device during a collection period. The visibility flag is a position in an image frame used to characterize visibility during the collection period.
[0126] The visibility flag in the embodiment of the present invention is not determined by the distance signs pre-set on both sides of the road. The visibility flag can be determined by the position of the dividing point / dividing line between the clear area and the fuzzy area in the image frame. For example, the visibility flag can be the position of the dividing point / dividing line between the clear area and the fuzzy area in the image frame. The visibility flag can also be a sub-area containing the dividing point / dividing line between the clear area and the fuzzy area in the image frame, and the distance from the boundary point of the sub-area to the aforementioned dividing point / dividing line is less than the preset distance; wherein, the clear area can be but not limited to including the image area in the image frame whose clarity is greater than the clarity threshold, and the fuzzy area can be but not limited to including the image area in the image frame whose clarity is less than or equal to the aforementioned clarity threshold; in the embodiment of the present application, the position of the dividing point between the clear area and the fuzzy area in an image frame can be determined by detecting and identifying at least one collected image frame, and the position can reflect the visibility of the road collection device during the collection period. Above this position, the identified target (such as a person or vehicle) is relatively blurred, and below this position, the identified target is relatively clear.
[0127] The embodiment of the present invention does not limit the method for determining the visibility flag.
[0128] A possible way to determine the visibility flag is to obtain multiple image frames with the same magnification collected during a collection period through a road collection device; perform target tracking on the multiple image frames to determine the driving trajectories of multiple targets; if the driving trajectory of any target is close to the road collection device, extract the starting point of the driving trajectory of the target; if the driving trajectory of any target is away from the road collection device, extract the end point of the driving trajectory of the target; and determine the visibility flag based on the extracted multiple starting points and / or end points.
[0129] For example, target tracking is performed on multiple image frames collected during the acquisition period, and the driving trajectories of five targets are determined, such as Figure 3 For any target, if the target's driving trajectory is far away from the road collection device, the end point of the target's driving trajectory is extracted; if the target's driving trajectory is close to the road collection device, the starting point of the target's driving trajectory is extracted. In this way, 5 starting points and / or end points can be obtained, as shown in Figure 3 As shown in the dotted box in . According to the coordinates of the five starting points and / or end points, the coordinates of the visibility flag are determined. Specifically, the coordinates of the five starting points and / or end points can be averaged, median or calculated in other ways to obtain the coordinates of the visibility flag. Figure 3 , the visibility flag T obtained by averaging the coordinates of the five starting points and / or end points is shown in FIG. It can be seen that the visibility flag can represent the farthest distance that the image acquisition device can clearly capture.
[0130] Another possible way to determine the visibility flag is to train a visibility recognition model using a large number of training samples. Input an image frame collected during the collection period into the visibility recognition model, and the image clarity and blurriness boundary line can be automatically output. Any point on the boundary line is the visibility flag. It is understandable that multiple image frames can also be input into the visibility recognition model to obtain the image clarity or blurriness boundary lines of multiple image frames, and the coordinates of these boundary lines are averaged to obtain a final boundary line. Then any point on the final boundary line is the visibility flag.
[0131] Another possible way to determine the visibility flag is that the user defines the visibility flag in an image frame according to the clarity of the object in the image frame, or the user defines the visibility flag based on the clarity of the objects in multiple image frames.
[0132] In step 202, for any image frame collected during the collection period, the horizontal conversion relationship of any pixel row in the image frame is determined according to multiple lane lines in the image frame; the horizontal conversion relationship is used to characterize the conversion relationship between the pixel distance and the physical distance on any pixel row in the image frame.
[0133] Determining the horizontal transformation relationship of any pixel row in an image frame requires determining multiple lane lines in the image frame. Perform image detection on any image frame and detect multiple lane lines in the image frame. Generally speaking, the lane lines are parallel to each other in three-dimensional space. According to the perspective principle, the extension lines of the lane lines in the image frame will intersect at one point. Figure 4a FIG. 1 is a schematic diagram of each lane line in a possible image frame. It can be seen that the extension lines of each lane line intersect at a point W.
[0134] In reality, the following situations may occur: the lane lines are not parallel to each other in three-dimensional space, or the lane lines are parallel to each other in three-dimensional space, but the image detection result is not very accurate, resulting in the extension lines of each lane line in the detected image frame not intersecting at one point. Therefore, it is necessary to recalibrate each lane line.
[0135] The specific method may be: extending the multiple lane lines detected in the image frame respectively, and determining whether the intersection points of the multiple extension lines are the same point; if they are not the same point, re-determining a corrected intersection point based on the positions of the multiple intersection points; and correcting the positions of the multiple lane lines based on the corrected intersection point and the multiple lane lines.
[0136] For example, Figure 4bA method for re-determining lane lines provided by an embodiment of the present invention is shown in FIG. Multiple lane lines detected in the image frame are extended respectively, and it is determined that the intersection points of the multiple extended lines are not the same point, and the intersection points are W1 and W2 respectively. Therefore, it is shown that the lane lines detected in the current image detection are not parallel in three-dimensional space, which will affect the accuracy of the subsequent determination of the horizontal transformation relationship. The midpoint of the intersection points W1 and W2 is taken to obtain the corrected intersection point W. According to the corrected intersection point and the bottom ends of the multiple lane lines detected before, the position of the lane line is re-determined, such as Figure 4b The dashed line in the figure shows the re-determined lane line. It is understandable that the determination of the corrected intersection point W is not limited to taking the midpoint of the intersection points W1 and W2, and other methods can also be used; the position of the re-determined lane line can also be determined based on the corrected intersection point and the top or other positions of the previously detected multiple lane lines, and the embodiment of the present invention does not limit this.
[0137] The re-determined lane lines have the same intersection point W, so in three-dimensional space, the lane lines are parallel to each other, which improves the accuracy of the subsequent determination of the horizontal conversion relationship of each pixel row in the image frame.
[0138] The method for determining the horizontal transformation relationship of any pixel row is to determine the horizontal inverse perspective image of the image frame according to multiple lane lines in the image frame. The physical distance between any two points in any pixel row can be determined based on the horizontal inverse perspective image. The pixel distance between the two points in the pixel row is determined based on the image frame. The horizontal transformation relationship of the pixel row can be determined through the pixel distance and physical distance of the two points.
[0139] For example, if the number of pixel rows in an image frame is 1000, due to the perspective principle, the conversion relationship between the pixel distance and the physical distance in each row is different. If the bottom row of the image frame is the 1st row, the top row of the image frame is the 1000th row. Generally speaking, as the number of rows increases, the physical distance that can be represented by the same length of pixel distance will increase. If the horizontal conversion relationship is expressed as the ratio of physical distance to pixel distance: η i =d / p; where η i is the horizontal conversion relationship of the i-th row, d is the physical distance, and p is the pixel distance. As the number of rows increases, the horizontal conversion relationship will also increase. For example, the horizontal conversion relationship of the 600th row is greater than the horizontal conversion relationship of the 500th row.
[0140] The above method can be used to determine the horizontal conversion relationship between the pixel distance and the physical distance on each pixel row in the image frame.
[0141] In step 203, a vertical conversion relationship is determined according to the number of pixel rows of the image frame and the vertical field angle of the road acquisition device; the vertical conversion relationship is used to characterize the conversion relationship between the vertical field angle and the number of pixel rows.
[0142] The vertical field of view of the road acquisition device is the field of view of the road acquisition device in the vertical direction. Reflected in the image frame, the number of pixel rows in the image frame is also the field of view of the road acquisition device in the vertical direction. Therefore, the number of pixel rows in the image frame and the vertical field of view of the road acquisition device have a certain conversion relationship, that is, a vertical conversion relationship.
[0143] It is expressed in the following formula: γ=θ / k; wherein γ is the vertical conversion relationship, θ is the vertical field angle of the road acquisition device, and k is the number of pixel rows in the image frame, for example, 1000 rows.
[0144] In step 204, the visible distance of the road collection device within the collection period is determined based on the visibility flag, the horizontal conversion relationship, and the vertical conversion relationship.
[0145] Figure 5a A schematic diagram showing an imaging area of a road acquisition device in three-dimensional space provided by an embodiment of the present invention is shown. Figure 5b A schematic diagram of any image frame collected by a road collection device provided by an embodiment of the present invention.
[0146] Combination Figure 5a and Figure 5b Make an introduction. Figure 5a In the figure, the pitch angle of the road acquisition device O is α, that is, the inclination angle of the road acquisition device O relative to the ground plane is α. The height of the road acquisition device from the ground is H. Plane ACDF is the shooting field of view of the road acquisition device O, that is, the image frame shot by the road acquisition device O corresponds to plane ACDF in the three-dimensional space, and the four vertices of the shot image frame correspond to A, C, D and F in the three-dimensional space respectively.
[0147] The vertical field of view of the road acquisition device O is the field of view of the road acquisition device O in the vertical direction, which corresponds to the straight line BE in the three-dimensional space and corresponds to the vertical symmetry axis of the image frame in the image frame. The center of the field of view of the road acquisition device O is O' in the three-dimensional space, which is the center point O' of the image frame, and the center point O' must be located on the vertical symmetry axis of the image frame.
[0148] Figure 5b The figure shows the relationship between each point in the image frame and Figure 5a The corresponding relationship between the points in . Figure 5b The visibility flag T is also shown in FIG. The method for determining the visibility flag has been introduced above and will not be repeated here.
[0149] The following describes a method for determining the visible distance of road data collection equipment during the collection period. Figure 6a As shown, including:
[0150] Step 601, constructing a first perpendicular foot from the visibility flag to the vertical symmetry axis of the image frame in the image frame; determining a first physical distance between the visibility flag and the first perpendicular foot according to a horizontal conversion relationship of a pixel row where the visibility flag is located.
[0151] like Figure 6b As shown, a perpendicular line is drawn through the visibility flag T to the vertical symmetry axis of the image frame to obtain the first perpendicular foot N.
[0152] Since the pixel row where the visibility flag T is located is known, and the horizontal conversion relationship of the pixel row where the visibility flag T is located is known, the first pixel distance between NT is also known, so the first physical distance between NT can be determined by the following formula, that is, Figure 5a The distance between NTs in:
[0153] L NT =η i ·χ; where L NT is the first physical distance, χ is the first pixel distance between NTs; η i is the horizontal transformation relationship of the i-th pixel row.
[0154] For example, it is known that the visibility flag is located at the 600th pixel row, the first pixel distance between NT is 2 cm, and the horizontal conversion relationship η of the 600th pixel row has been determined in step 202. 600 , so the first pixel distance 2cm multiplied by the horizontal conversion relationship η of the 600th pixel row 600 , we can get the first physical distance L between NT NT :
[0155] L NT =η 600 ·χ; where L NT is the first physical distance, χ is the first pixel distance between NTs; η 600 This is the horizontal conversion relationship of the 600th pixel row.
[0156] Step 602: Determine a second physical distance between the first vertical foot and a projection point of the road collection device on the ground based on the height of the road collection device from the ground, the pitch angle of the road collection device and the vertical conversion relationship.
[0157] The second physical distance determined is Figure 5a The specific method for determining the second physical distance is as follows: Figure 7 As shown, including:
[0158] Step 701, based on the location of the road collection device and the physical location of the first perpendicular foot in the three-dimensional space, construct a right triangle perpendicular to the plane where the road is located, such as Figure 5a The right triangle OMN in.
[0159] Step 702: Determine a first angle corresponding to the number of pixel rows between the first perpendicular foot and the center point of the image frame according to the vertical conversion relationship.
[0160] In triangle ONO', ω can be determined according to the vertical conversion relationship, γ = ω / b, where γ is the vertical conversion relationship, ω is the first angle, and b is the second pixel distance of O'N. Because O' is the center point of the image frame, the second pixel distance b of O'N is known, and γ is known, so ω, that is, the first angle, can be determined according to the above formula.
[0161] Step 703, determining a second angle in the right triangle according to the pitch angle of the road collection device and the first angle;
[0162] The pitch angle α of the road collection device is an external angle of the triangle ONO', so That is, the second angle is determined. is the second angle, α is the pitch angle of the road collection equipment, and ω is the first angle.
[0163] Step 704: Determine a second physical distance between the first perpendicular foot and the projection point of the road collection device on the ground according to the second angle and the height of the road collection device from the ground.
[0164] According to the triangle related formula, the second physical distance can be obtained by the following formula:
[0165] Among them, L MN is the second physical distance between the first perpendicular foot and the projection point of the road collection device on the ground, H is the height of the road collection device from the ground, is the second angle.
[0166] Step 603: Determine the distance between the road acquisition device and the physical position of the visibility mark in the three-dimensional space based on the first physical distance, the second physical distance and the height of the image acquisition device from the ground.
[0167] In the right triangle MNT, the first physical distance and the second physical distance are known. According to the Pythagorean theorem, the physical distance between MT can be obtained:
[0168] Among them, L MNL is the second physical distance between the first perpendicular foot and the projection point of the road collection device on the ground; NT is the first physical distance between the first vertical foot and the visibility mark; L MT It is the physical distance between the visibility mark and the projection point of the road collection equipment on the ground.
[0169] In the right triangle OMT, since the height H of the road collection equipment from the ground is known, the Pythagorean theorem is used again to obtain the physical distance between OT:
[0170] Among them, L MN L is the second physical distance between the first perpendicular foot and the projection point of the road collection device on the ground; NT is the first physical distance between the first vertical foot and the visibility mark; L OT is the physical distance between the road data collection device and the visibility mark; H is the height of the road data collection device from the ground.
[0171] That is, the distance between the physical position of the road collection device and the visibility mark in the three-dimensional space, that is, the visible distance of the road collection device during the collection period.
[0172] Optionally, since the physical distance between MTs is much greater than H, the second physical distance L can be directly used as MT As visible distance.
[0173] The following describes a method for determining the horizontal conversion relationship through a specific embodiment. Figure 8 As shown, including:
[0174] Step 801: Determine a horizontal inverse perspective image of the image frame according to a plurality of numbers of pixels between intersections of a plurality of lane lines and a plurality of pixel rows in the image frame.
[0175] exist Figure 9a In the example, any pixel row, such as the pixel row where OPQR is located, is selected as the target pixel row, and the first pixel number between any two lane lines and the intersection of the target pixel row is determined. For example, the first pixel number x1 between the intersection of lane lines l1 and l2 with the target pixel row, i.e., OP, is determined, the first pixel number x2 between lane lines l2 and l3 with the target pixel row, i.e., PQ, is determined, and the first pixel number x3 between lane lines l3 and l4 with the target pixel row, i.e., QR, is determined.
[0176] A reference pixel row is determined, and the reference pixel row is located above the target pixel row in the image frame. The second pixel number between any two lane lines and the intersection of the reference pixel row is determined. The determination method is the same as the first pixel number, and will not be repeated here. For example, the multiple second pixel numbers determined are y1, y2 and y3. y1, y2 and y3 correspond to x1, x2 and x3 respectively.
[0177] The embodiment of the present invention does not limit the method for determining the reference pixel row. The reference pixel row should be as close to the top of the image frame as possible, for example, the last pixel row of the image frame is taken as the reference pixel row. Alternatively, the reference pixel row can also be taken in the middle of the image frame. In short, after the reference pixel row is determined, the target pixel row can only be located below the reference pixel row. In this way, only the inverse perspective coefficients of the pixel rows below the reference pixel row can be determined, and then the final determined horizontal inverse perspective image also only includes the pixel rows below the reference pixel row. Therefore, those skilled in the art can determine the reference pixel row as needed.
[0178] The inverse perspective coefficient of the target pixel row is determined as follows: 1. The inverse perspective coefficient of the target pixel row is determined according to the ratio of one of the second pixel numbers to the corresponding first pixel number, for example, y1 / x1. 2. The inverse perspective coefficient of the target pixel row is determined according to the ratio of multiple second pixel numbers to multiple first pixel numbers. For example, the final inverse perspective coefficient of the target pixel row is obtained by taking the average of y1 / x1, y2 / x2, and y3 / x3.
[0179] In a possible implementation, according to the above method, the inverse perspective coefficients of all pixel rows are determined one by one. For example, if there are 1000 pixel rows in the image frame, 1000 calculations are required.
[0180] Another possible implementation is to calculate the inverse perspective coefficients of only n pixel rows, where n≥3. After determining the inverse perspective coefficients of the n pixel rows, the inverse perspective coefficient of each pixel row in the image frame is fitted by the least squares method.
[0181] After the inverse perspective coefficient of each pixel row is determined, the pixel coordinates on each pixel row can be transformed to obtain a horizontal inverse perspective image.
[0182] The conversion method is:
[0183]
[0184] Among them, X and Y are the pixel coordinates of the original image, are the pixel coordinates after inverse perspective. δ is the inverse perspective coefficient corresponding to the pixel row where the pixel coordinates of the original image are located.
[0185] Figure 9bA possible horizontal reverse perspective is shown. It can be found that compared with Figure 9a , Figure 9b The horizontal coordinates of each pixel in the image have changed, eliminating the visual error of "near objects are larger and far objects are smaller" in the horizontal direction. The lane lines that are parallel to each other in the three-dimensional space are also parallel to each other in the horizontal inverse perspective image.
[0186] Step 802: for two intersection points of any two lane lines and any pixel row in the horizontal reverse perspective image, determine the physical distance between the two intersection points according to the physical lane widths of the two lane lines.
[0187] For example, for lane lines l2 and l3, determine the two intersection points of these two lane lines with any pixel row, P' and Q', draw a perpendicular line from P' to lane line l3, and obtain the second perpendicular foot R'. Since the pixel distance of P'R' is known, the pixel distance of P'Q' is known, and the physical distance of P'R' is known (the physical distance of P'R' is the lane width l between lane lines l2 and l3), the physical distance of P'Q' can be obtained according to the proportional relationship.
[0188] Step 803: Determine the pixel distance between the two intersection points in the image frame.
[0189] According to the mapping relationship between the image frame and each pixel of the horizontal inverse perspective image, the two intersection points determined in step 802, for example, P and Q, can be determined in the image frame.
[0190] Only in the horizontal inverse perspective map can the physical distance between P and Q be determined, and in the image frame the pixel distance between P and Q can be determined.
[0191] Step 804: Determine the horizontal conversion relationship of the pixel rows where the two intersection points are located according to the physical distance between the two intersection points and the pixel distance between the two intersection points.
[0192] According to the ratio of the physical distance between P and Q and the pixel distance between P and Q, the horizontal conversion relationship of the pixel rows where P and Q are located is determined.
[0193] In a possible implementation, the horizontal conversion relationship of all pixel rows in the image frame is determined one by one in the above manner. For example, if there are 1000 pixel rows in the image frame, 1000 calculations are required.
[0194] Another possible implementation is to calculate only the horizontal transformation relationship of n pixel rows, where n≥3. After the horizontal transformation relationship of n pixel rows is determined, the horizontal transformation relationship of each pixel row in the image frame is fitted by the least squares method.
[0195] In determining the vertical conversion relationship, the vertical field of view of the road acquisition device is used. The zoom camera has a variety of different magnifications, and the vertical field of view corresponding to different magnifications is different. Therefore, the vertical conversion relationship is determined in the following way:
[0196] Acquire the magnification of the image frame; determine the focal length corresponding to the image frame according to the corresponding relationship between the magnification and focal length of the road acquisition device; determine the vertical field of view of the road acquisition device according to the focal length corresponding to the image frame and the image height of the road acquisition device; determine the vertical conversion relationship according to the vertical field of view and the number of pixel rows of the image frame.
[0197] The equipment parameters of the road acquisition equipment store the correspondence between magnification and focal length. After obtaining the magnification of the image frame, the focal length corresponding to the image frame can be determined through the correspondence. The vertical field of view of the road acquisition equipment is determined by the following formula h = EFL*tan(half FOV), where h is half the height of the imaging area of the road acquisition equipment sensor, EFL is the focal length, and FOV is the vertical field of view of the road acquisition equipment.
[0198] By determining the vertical field of view of the road acquisition device in the above manner, the vertical conversion relationship can be determined based on the vertical field of view and the number of pixel rows of the image frame. In this way, if the camera has different magnifications and captures image frames of various magnifications, it is only necessary to analyze the image frame with the same magnification as the image frame for determining the visibility flag, obtain the magnification corresponding to the image frame, and then determine the vertical conversion relationship corresponding to the image frame.
[0199] In order to better explain the embodiment of the present invention, the process of determining the above road visibility will be described below in a specific implementation scenario. Fig.10 A schematic flow chart of a method for determining road visibility provided by an embodiment of the present invention is shown.
[0200] Step 1001, obtaining relevant parameters of road data collection equipment.
[0201] Including the type of road acquisition equipment (zoom / fixed focus), the pitch angle α of the road acquisition equipment, the height H of the road acquisition equipment from the ground, and the lane width l; if the type of road acquisition equipment is fixed focus, it is also necessary to obtain the vertical field of view angle of the road acquisition equipment; if the type of road acquisition equipment is zoom, it is also necessary to obtain the mapping relationship between the magnification and focal length of the road acquisition equipment, and determine the mapping relationship between the magnification and the vertical field of view angle based on the mapping relationship between the magnification and the focal length.
[0202] Step 1002, obtain any image frame taken by any road acquisition device, detect the lane lines in the image frame by using an image detection algorithm, and correct the detection result to obtain lane lines with the same intersection.
[0203] Step 1003 , determining a horizontal inverse perspective image of the image frame according to a plurality of numbers of pixels between intersections of a plurality of lane lines and a plurality of pixel rows in the image frame.
[0204] Step 1004 , determining the horizontal conversion relationship of each pixel row in the image frame according to the physical distance between any two points in the horizontal inverse perspective image and the pixel distance between the two points in the image frame.
[0205] Step 1005, determining the vertical conversion relationship of the image frame according to the vertical field angle of the road acquisition device and the number of pixel rows of the image frame.
[0206] Step 1006, start weather recognition.
[0207] Step 1007, determine whether it is bad weather such as fog, rain, snow, etc. that can affect visibility. If so, proceed to step 1008, if not, continue to determine.
[0208] Step 1008, collect image frames within a period of time, start the target tracking algorithm for the collected multiple image frames, and determine the visibility flag.
[0209] The image frames collected in this step should have the same magnification as the image frames whose horizontal conversion relationship and vertical conversion relationship are determined in the previous step.
[0210] Step 1009: determining the visible distance of the road collection device within the collection period based on the visibility flag, the horizontal conversion relationship, and the vertical conversion relationship.
[0211] Step 1010 , determine whether the visible distance is less than or equal to the warning distance, if so, proceed to step 1011 , if not, return to step 1007 .
[0212] Step 1011, start the early warning program.
[0213] (2) When the visibility level is level 1, no warning action is required;
[0214] (3) When the visibility level is level 2, sound and light warning measures such as road electronic screens are activated to remind passing vehicles to pay attention to their speed and drive carefully;
[0215] (4) When the visibility level reaches Level 3, the road electronic screen and other sound and light warning measures are activated and warning information is broadcast to vehicles in the area through the wireless network.
[0216] (5) When the visibility level is Level 4, all warning measures that are taken when visibility reaches Level 3 are initiated, and warning information is sent to the traffic control department.
[0217] Based on the same technical concept, Fig.11 The structure of a device for determining road visibility provided by an embodiment of the present invention is exemplarily shown, and the structure can execute the process of determining road visibility.
[0218] like Fig.11 As shown, the device specifically includes:
[0219] The determining unit 1101 is configured to:
[0220] Determine a visibility flag through at least one image frame collected by the road collection device within a collection period; the visibility flag is a position in the image frame used to characterize the visibility within the collection period;
[0221] For any image frame collected during the collection period, a horizontal conversion relationship of any pixel row in the image frame is determined according to a plurality of lane lines in the image frame; the horizontal conversion relationship is used to characterize a conversion relationship between a pixel distance and a physical distance on any pixel row in the image frame;
[0222] Determining a vertical conversion relationship according to the number of pixel rows of the image frame and the vertical field angle of the road acquisition device; the vertical conversion relationship is used to characterize the conversion relationship between the vertical field angle and the number of pixel rows;
[0223] The visible distance of the road collection device within the collection period is determined based on the visibility flag, the horizontal conversion relationship and the vertical conversion relationship.
[0224] Optionally, the determining unit 1101 is specifically configured to:
[0225] Constructing a first perpendicular foot from the visibility flag to the vertical symmetry axis of the image frame in the image frame; determining a first physical distance between the visibility flag and the first perpendicular foot according to a horizontal conversion relationship of a pixel row where the visibility flag is located;
[0226] Determine a second physical distance between the first vertical foot and a projection point of the road collection device on the ground based on the height of the road collection device from the ground, the pitch angle of the road collection device and the vertical conversion relationship;
[0227] The distance between the road acquisition device and the physical position of the visibility mark in the three-dimensional space is determined based on the first physical distance, the second physical distance and the height of the image acquisition device from the ground.
[0228] Optionally, the determining unit 1101 is specifically configured to:
[0229] Based on the location of the road collection device and the physical location of the first perpendicular foot in the three-dimensional space, construct a right triangle perpendicular to the plane where the road is located;
[0230] Determine a first angle corresponding to the number of pixel rows between the first perpendicular foot and the center point of the image frame according to the vertical conversion relationship;
[0231] Determine a second angle in the right triangle according to the pitch angle of the road collection device and the first angle;
[0232] A second physical distance between the first perpendicular foot and a projection point of the road collection device on the ground is determined according to the second angle and the height of the road collection device from the ground.
[0233] Optionally, the determining unit 1101 is specifically configured to:
[0234] Determining a horizontal inverse perspective image of the image frame according to a plurality of numbers of pixels between intersections of a plurality of lane lines and a plurality of pixel rows in the image frame;
[0235] For two intersection points of any two lane lines and any pixel row in the horizontal reverse perspective image, determine the physical distance between the two intersection points according to the physical lane widths of the two lane lines;
[0236] determining a pixel distance between the two intersection points in the image frame;
[0237] According to the physical distance between the two intersection points and the pixel distance between the two intersection points, the horizontal conversion relationship of the pixel rows where the two intersection points are located is determined.
[0238] Optionally, the determining unit 1101 is specifically configured to:
[0239] For a target pixel row in the image frame, determining a first number of pixels between intersections of any two lane lines and the target pixel row; determining a second number of pixels between intersections of any two lane lines and a reference pixel row; the target pixel row is any pixel row among the plurality of pixel rows; the reference pixel row is located above the target pixel row in the image frame;
[0240] Determine an inverse perspective coefficient of the target pixel row according to one or more first pixel quantities and one or more second pixel quantities;
[0241] A horizontal inverse perspective view of the image frame is determined according to inverse perspective coefficients of the plurality of target pixel rows and pixel coordinates on the plurality of target pixel rows.
[0242] Optionally, the determining unit 1101 is further configured to:
[0243] Extending the plurality of lane lines detected in the image frame respectively, and determining whether the intersection points of the plurality of extension lines are the same point; if they are not the same point, re-determining a corrected intersection point according to the positions of the plurality of intersection points;
[0244] The positions of the multiple lane lines are corrected according to the corrected intersection point and the multiple lane lines.
[0245] Optionally, the determining unit 1101 is specifically configured to:
[0246] Acquiring a magnification of the image frame;
[0247] Determining the focal length corresponding to the image frame according to the corresponding relationship between the magnification and the focal length of the road acquisition device;
[0248] Determining a vertical field of view angle of the road acquisition device according to a focal length corresponding to the image frame and an image height of the road acquisition device;
[0249] The vertical conversion relationship is determined according to the vertical field of view angle and the number of pixel rows of the image frame.
[0250] Optionally, the determining unit 1101 is specifically configured to:
[0251] Acquire multiple image frames with the same magnification collected during a collection period through a road collection device;
[0252] Performing target tracking on the multiple image frames to determine the driving trajectories of the multiple targets;
[0253] If the driving trajectory of any target is close to the road collection device, the starting point of the driving trajectory of the target is extracted; if the driving trajectory of any target is far away from the road collection device, the end point of the driving trajectory of the target is extracted;
[0254] The visibility flag is determined according to the extracted multiple starting points and / or end points.
[0255] Based on the same technical concept, the embodiment of the present application provides a computer device, such as Fig.12 As shown, it includes at least one processor 1201 and a memory 1202 connected to the at least one processor. The specific connection medium between the processor 1201 and the memory 1202 is not limited in the embodiment of the present application. Fig.12 For example, the processor 1201 and the memory 1202 are connected via a bus. The bus can be divided into an address bus, a data bus, a control bus, etc.
[0256] In the embodiment of the present application, the memory 1202 stores instructions that can be executed by at least one processor 1201. The at least one processor 1201 can perform the steps of the above-mentioned road visibility determination method by executing the instructions stored in the memory 1202.
[0257] The processor 1201 is the control center of the computer device, and can use various interfaces and lines to connect various parts of the computer device, and determine the road visibility by running or executing instructions stored in the memory 1202 and calling data stored in the memory 1202. Optionally, the processor 1201 may include one or more processing units, and the processor 1201 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 1201. In some embodiments, the processor 1201 and the memory 1202 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on independent chips.
[0258] Processor 1201 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware processor for execution, or can be executed by a combination of hardware and software modules in the processor.
[0259] The memory 1202 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 1202 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 1202 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 1202 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.
[0260] Based on the same technical concept, an embodiment of the present invention further provides a computer-readable storage medium, which stores a computer-executable program. The computer-executable program is used to enable a computer to execute the method for determining the road visibility listed in any of the above-mentioned ways.
[0261] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0262] The present application is described with reference to the flowchart and / or block diagram of the method, device (system), and computer program product according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the process and / or box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the function specified in one process or multiple processes in the flowchart and / or one box or multiple boxes in the block diagram.
[0263] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.
[0264] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.
[0265] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A method for determining road visibility, characterized in that: include: Determine the visibility flag through at least one image frame collected by the road collection device within the collection period; The visibility flag is a position in the image frame used to characterize the visibility within the acquisition period; For any image frame collected during the collection period, a horizontal conversion relationship of any pixel row in the image frame is determined according to a plurality of lane lines in the image frame; the horizontal conversion relationship is used to characterize a conversion relationship between a pixel distance and a physical distance on any pixel row in the image frame; Determining a vertical conversion relationship according to the number of pixel rows of the image frame and the vertical field angle of the road acquisition device; the vertical conversion relationship is used to characterize the conversion relationship between the vertical field angle and the number of pixel rows; Determine the visible distance of the road acquisition device within the acquisition period based on the visibility flag, the horizontal conversion relationship, and the vertical conversion relationship; wherein, determining the visible distance of the road acquisition device within the acquisition period based on the visibility flag, the horizontal conversion relationship, and the vertical conversion relationship includes: constructing a first perpendicular foot from the visibility flag to the vertical symmetry axis of the image frame in the image frame; determining a first physical distance between the visibility flag and the first perpendicular foot according to the horizontal conversion relationship of the pixel row where the visibility flag is located; determining a second physical distance between the first perpendicular foot and the projection point of the road acquisition device on the ground based on the height of the road acquisition device from the ground, the pitch angle of the road acquisition device, and the vertical conversion relationship; determining the distance between the road acquisition device and the physical position of the visibility flag in the three-dimensional space based on the first physical distance, the second physical distance, and the height of the road acquisition device from the ground, wherein the distance between the road acquisition device and the physical position of the visibility flag in the three-dimensional space is the visible distance of the road acquisition device within the acquisition period.
2. The method according to claim 1, characterized in that Determining a second physical distance between the first vertical foot and a projection point of the road collection device on the ground based on the height of the road collection device from the ground, the pitch angle of the road collection device, and the vertical conversion relationship includes: Based on the location of the road collection device and the physical location of the first perpendicular foot in the three-dimensional space, construct a right triangle perpendicular to the plane where the road is located; Determine a first angle corresponding to the number of pixel rows between the first perpendicular foot and the center point of the image frame according to the vertical conversion relationship; Determine a second angle in the right triangle according to the pitch angle of the road collection device and the first angle; A second physical distance between the first perpendicular foot and a projection point of the road collection device on the ground is determined according to the second angle and the height of the road collection device from the ground.
3. The method according to claim 1, characterized in that Determining a horizontal conversion relationship of any pixel row in the image frame according to a plurality of lane lines in the image frame includes: Determining a horizontal inverse perspective image of the image frame according to a plurality of numbers of pixels between intersections of a plurality of lane lines and a plurality of pixel rows in the image frame; For two intersection points of any two lane lines and any pixel row in the horizontal reverse perspective image, determine the physical distance between the two intersection points according to the physical lane widths of the two lane lines; determining a pixel distance between the two intersection points in the image frame; According to the physical distance between the two intersection points and the pixel distance between the two intersection points, the horizontal conversion relationship of the pixel rows where the two intersection points are located is determined.
4. The method according to claim 3, characterized in that Determining a horizontal inverse perspective image of the image frame according to a plurality of numbers of pixels between intersections of a plurality of lane lines and a plurality of pixel rows in the image frame includes: For a target pixel row in the image frame, determining a first number of pixels between intersections of any two lane lines and the target pixel row; determining a second number of pixels between intersections of any two lane lines and a reference pixel row; the target pixel row is any pixel row among the plurality of pixel rows; the reference pixel row is located above the target pixel row in the image frame; Determine an inverse perspective coefficient of the target pixel row according to one or more first pixel quantities and one or more second pixel quantities; A horizontal inverse perspective view of the image frame is determined according to inverse perspective coefficients of the plurality of target pixel rows and pixel coordinates on the plurality of target pixel rows.
5. The method according to claim 3, characterized in that Before determining the horizontal inverse perspective image of the image frame according to the number of pixels between the intersections of the plurality of lane lines and the plurality of pixel rows in the image frame, the method further includes: Extending the plurality of lane lines detected in the image frame respectively, and determining whether the intersection points of the plurality of extension lines are the same point; if they are not the same point, re-determining a corrected intersection point according to the positions of the plurality of intersection points; The positions of the multiple lane lines are corrected according to the corrected intersection point and the multiple lane lines.
6. The method according to claim 1, characterized in that The vertical conversion relationship is determined by: Acquiring a magnification of the image frame; Determining the focal length corresponding to the image frame according to the corresponding relationship between the magnification and the focal length of the road acquisition device; Determining a vertical field of view angle of the road acquisition device according to a focal length corresponding to the image frame and an image height of the road acquisition device; The vertical conversion relationship is determined according to the vertical field of view angle and the number of pixel rows of the image frame.
7. The method according to claim 1, characterized in that Determining the visibility flag through at least one image frame collected by the road collection device within the collection period includes: Acquire multiple image frames with the same magnification collected during a collection period through a road collection device; Performing target tracking on the multiple image frames to determine the driving trajectories of the multiple targets; If the driving trajectory of any target is close to the road collection device, the starting point of the driving trajectory of the target is extracted; if the driving trajectory of any target is far away from the road collection device, the end point of the driving trajectory of the target is extracted; The visibility flag is determined according to the extracted multiple starting points and / or end points.
8. A device for determining road visibility, characterized in that: include: Identify units for: Determine the visibility flag through at least one image frame collected by the road collection device within the collection period; The visibility flag is a position in the image frame used to characterize the visibility within the acquisition period; For any image frame collected during the collection period, a horizontal conversion relationship of any pixel row in the image frame is determined according to a plurality of lane lines in the image frame; the horizontal conversion relationship is used to characterize a conversion relationship between a pixel distance and a physical distance on any pixel row in the image frame; Determining a vertical conversion relationship according to the number of pixel rows of the image frame and the vertical field angle of the road acquisition device; the vertical conversion relationship is used to characterize the conversion relationship between the vertical field angle and the number of pixel rows; Determine the visible distance of the road acquisition device within the acquisition period based on the visibility flag, the horizontal conversion relationship, and the vertical conversion relationship; wherein, determining the visible distance of the road acquisition device within the acquisition period based on the visibility flag, the horizontal conversion relationship, and the vertical conversion relationship includes: constructing a first perpendicular foot from the visibility flag to the vertical symmetry axis of the image frame in the image frame; determining a first physical distance between the visibility flag and the first perpendicular foot according to the horizontal conversion relationship of the pixel row where the visibility flag is located; determining a second physical distance between the first perpendicular foot and the projection point of the road acquisition device on the ground based on the height of the road acquisition device from the ground, the pitch angle of the road acquisition device, and the vertical conversion relationship; determining the distance between the road acquisition device and the physical position of the visibility flag in the three-dimensional space based on the first physical distance, the second physical distance, and the height of the road acquisition device from the ground, wherein the distance between the road acquisition device and the physical position of the visibility flag in the three-dimensional space is the visible distance of the road acquisition device within the acquisition period.
9. A computing device, characterized in that include: Memory for storing computer programs; A processor, configured to call a computer program stored in the memory, and execute the method according to any one of claims 1 to 7 according to the obtained program.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer-executable program, and the computer-executable program is used to enable a computer to execute the method according to any one of claims 1 to 7.
