A road condition recognition method, device and equipment

Abstract

The application discloses a road condition recognition method, device and equipment. The method comprises the following steps: acquiring a first image of road condition information of a first road collected by a monitoring device, and performing feature recognition on the first image to obtain a first recognition result. The first recognition result comprises at least one target area and attribute information of each target area. The first image and the first recognition result are displayed on a first interface. In response to a trigger operation, the first recognition result is adjusted to obtain a target recognition result.

Description

Technical Field

[0001] This application relates to the field of computer display technology, and to, but is not limited to, a road condition recognition method, apparatus, and device. Background Technology

[0002] In related technologies, one approach to road condition recognition is to collect images of vehicles on-site, determine a database of vehicle trajectories, input them into an initial recognition model for training, and generate a recognition model to identify road conditions. Another approach is to use onboard cameras to collect images in real time to identify road conditions, primarily for navigation, autonomous driving, and other functions.

[0003] However, none of the above methods are accurate enough for road condition identification. Summary of the Invention

[0004] In view of this, embodiments of this application provide a road condition recognition method, apparatus, and device.

[0005] In a first aspect, embodiments of this application provide a road condition recognition method, the method comprising: acquiring a first image representing road condition information of the first road collected by a monitoring device on the first road, and performing feature recognition on the first image to obtain a first recognition result; the first recognition result includes at least one target area and attribute information of each target area; displaying at least the first image and the first recognition result on a first interface; and adjusting the first recognition result in response to a triggered operation to obtain a target recognition result.

[0006] Secondly, embodiments of this application provide a road condition recognition device, comprising: a first acquisition module, configured to acquire a first image representing road condition information of the first road collected by a monitoring device on the first road, and perform feature recognition on the first image to obtain a first recognition result; the first recognition result includes at least one target area and attribute information of each target area; a first display module, configured to display at least the first image and the first recognition result on a first interface; and a first adjustment module, configured to adjust the first recognition result in response to an acquired trigger operation to obtain a target recognition result.

[0007] Thirdly, embodiments of this application provide an electronic device, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the program to implement the steps in the road condition recognition method.

[0008] In this embodiment, the first image containing road condition information is first automatically identified to obtain a first identification result. Then, the user manually adjusts the displayed first identification result, thereby improving the accuracy and reliability of road condition identification. Attached Figure Description

[0009] Figure 1 This is a schematic flowchart of a road condition recognition method according to an embodiment of this application;

[0010] Figure 2a This is a schematic diagram of a first image according to an embodiment of this application;

[0011] Figure 2b This is a schematic diagram of a method for establishing a coordinate system for a first image according to an embodiment of this application;

[0012] Figure 2c This is a schematic diagram of a first interface according to an embodiment of this application;

[0013] Figure 3 This is a schematic diagram of another first interface according to an embodiment of this application;

[0014] Figure 4 This is a schematic diagram of yet another first interface according to an embodiment of this application;

[0015] Figure 5 This is a schematic diagram of another first interface according to an embodiment of this application;

[0016] Figure 6 A schematic diagram of a first interface is also provided for an embodiment of this application;

[0017] Figure 7 This is a flowchart illustrating another road condition recognition method according to an embodiment of this application;

[0018] Figure 8 This is a schematic diagram illustrating the rendering and display of a graphical user interface on a terminal device according to an embodiment of this application;

[0019] Figure 9 This is a schematic diagram of the composition structure of a road condition recognition device according to an embodiment of this application;

[0020] Figure 10 This is a schematic diagram of the composition structure of another road condition recognition device according to an embodiment of this application;

[0021] Figure 11 This is a schematic diagram of the composition structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0022] The technical solution of this application will be further described in detail below with reference to the accompanying drawings and embodiments.

[0023] Figure 1 This is a flowchart illustrating a road condition recognition method according to an embodiment of this application, as shown below. Figure 1 As shown, the method includes the following steps:

[0024] Step 102: Acquire a first image representing the road condition information of the first road collected by the monitoring device on the first road, and perform feature recognition on the first image to obtain a first recognition result; the first recognition result includes at least one target area and attribute information of each target area;

[0025] The monitoring equipment can be a camera or other hardware device with video monitoring capabilities. It can be installed on the first road to capture a first image containing road condition information. The first image may include the sky, clouds, trees, lanes and non-lanes on the first road, vehicles traveling in lanes, and pedestrians in non-lanes. The target area can be an area in the first image related to road condition information, including lane areas and both lane and non-lane areas. Installers and debuggers can obtain the first image through a terminal device and perform installation and debugging based on it. Traffic management personnel can also obtain the first image through a terminal device and determine abnormal behavior on the road based on it. The terminal device includes, but is not limited to, mobile phones, tablets, laptops, desktop computers, or other terminal devices with display screens.

[0026] The first image can be automatically identified using a recognition model to obtain a first recognition result. The recognition model can be generated by training an initial recognition model using second images of vehicles traveling on roads under various road conditions. The second image can be first used to perform feature recognition to form a database containing target regions and attribute information of the target regions in the second image. The database is then input into the initial recognition model to train the initial recognition model and generate a trained recognition model. The first image is then input into the trained recognition model, and the trained recognition model can output at least one target region in the first image and attribute information of each target region.

[0027] Step 104: Display at least the first image and the first recognition result on the first interface;

[0028] The first image can be displayed on the first interface of the terminal device, and the first recognition result can be superimposed on the first image.

[0029] Step 106: In response to the acquired trigger operation, adjust the first recognition result to obtain the target recognition result.

[0030] The first identification result may be incorrect or inaccurate, such as incorrect or inaccurate identification of the target area's location or attribute information. The first identification result is adjustable; if the identification is inaccurate or incorrect, it can be adjusted to match the actual situation in the first image. The triggering operation can be a user-initiated operation on the first interface. The user can determine the true location and attribute information of the target area based on the displayed first image, and then adjust the first identification result according to the true location or attribute information to obtain the target identification result.

[0031] In this embodiment, the first image containing road condition information is first automatically identified to obtain a first identification result. Then, the user manually adjusts the first identification result based on the displayed first image, thereby improving the accuracy and reliability of road condition identification.

[0032] In some embodiments, the target area includes a lane area, and the attribute information includes the vehicle driving direction of the lane area; step 104, "displaying at least the first image and the first recognition result on the first interface," can be achieved through the following steps 1041a to 1044a:

[0033] Step 1041a: Display the first image on the first layer of the first interface;

[0034] Step 1042a: Based on the first recognition result, establish the coordinate system of the first image to generate a first set of coordinates for each lane region;

[0035] Among them, such as Figure 2a As shown, the first image 20 can be displayed on the first layer of the first interface; as Figure 2b As shown, a coordinate system can be established with the lower left vertex of the first image as the origin. The coordinates of the four vertices of the first image can be [0, 0], [0, b], [a, b], and [a, 0], respectively, where a and b are the length and height of the first image, respectively. Figure 2c As shown, each identified lane region can be represented by a first set of coordinates. Assuming two lane regions are identified: the first lane region and the second lane region, the first set of coordinates for the first lane region can be represented as... The first set of coordinates for the second lane region can be represented as:

[0036] Step 1043a: Based on at least one of the first coordinate sets, display at least one lane area on the second layer of the first interface; the second layer is located above the first layer;

[0037] Step 1044a: Display the vehicle travel direction in each lane area.

[0038] Specifically, when performing feature recognition on the second image used for model training, a database containing lane areas and vehicle driving directions can be formed. The database is then input into an initial recognition model to train the initial recognition model and generate a trained recognition model. When the first image is input into the trained recognition model, the recognition model can identify at least one lane area and vehicle driving direction.

[0039] like Figure 2c As shown, a first image 20 can be displayed on the first layer of the first interface, and a first lane area 21 and a second lane area 22 can be displayed on the second layer of the first interface 20. The identified vehicle driving direction (upward) can be displayed in the first lane area 21, and the identified vehicle driving direction (downward) can be displayed in the second lane area 22 (the vehicle driving direction is indicated by the direction of the arrow). The area identifier corresponding to the lane area can also be displayed in the lane area. The area identifier can be an area number. To facilitate the differentiation of different areas, the area numbers can be displayed from left to right in the first image 20 as 1, 2...n. For example, if the area number corresponding to the first lane area 21 is 1, then the area number "①" can be displayed in the first lane area 21. If the area number corresponding to the second lane area 22 is 2, then the number "②" can be displayed in the second lane area 22.

[0040] In this embodiment of the application, by displaying a first image on a first interface and overlaying at least one lane area and the vehicle driving direction and area identifier of each lane area on the first image, the identified lane area and vehicle driving direction can be displayed more intuitively, so that the user can adjust the lane area or the vehicle driving direction based on the first image.

[0041] In some embodiments, the target area includes a lane area and a non-lane area, the attribute information includes the vehicle driving direction and a first area identifier of the lane area, and a second area identifier of the non-lane area, and step 104 "displaying at least the first image and the first recognition result on the first interface" can be achieved through the following steps 1041b to 1045b:

[0042] Step 1041b: Display the first image on the first layer of the first interface;

[0043] Step 1042b: Based on the first recognition result, establish the coordinate system of the first image to generate a first coordinate set for each lane area and a second coordinate set for each non-lane area;

[0044] Among them, such asFigure 3 As shown, a coordinate system can be established with the lower left vertex of the first image as the origin. The coordinates of the four vertices of the first image can be [0, 0], [0, b], [a, b], and [a, 0], respectively, where a and b are the length and height of the first image, respectively. Each identified lane region can be represented by a first set of coordinates. Assuming two lane regions are identified: the first lane region and the second lane region, the first set of coordinates for the first lane region can be represented as follows: The first set of coordinates for the second lane region can be represented as: Each identified non-lane region can be represented by a second set of coordinates. Assuming a non-lane region is identified, the second set of coordinates for that non-lane region can be represented as follows:

[0045] Step 1043b: Based on at least one of the first coordinate sets and at least one of the second coordinate sets, display at least one lane area and at least one non-lane area on the second layer of the first interface; the second layer is located above the first layer;

[0046] Specifically, when performing feature recognition on the second image used for model training, a database containing lane areas, non-lane areas, and vehicle driving directions can be formed. The database is input into the initial recognition model to train the initial recognition model and generate a trained recognition model. When the first image is input into the trained recognition model, the recognition model can identify at least one lane area, at least one non-lane area, and the vehicle driving direction of each lane area.

[0047] Step 1044b: Display the vehicle travel direction and the first area identifier for each lane area;

[0048] Step 1045b: Display a second area identifier for each of the non-lane areas.

[0049] The attribute information of the target area may include the area type Tn and the area identifier. The area type Tn may indicate whether the area is a lane or a non-lane. The area identifier of the lane may be represented as a first area identifier, and the area identifier of the non-lane may be represented as a second area identifier. The first area identifier or the second area identifier may be an area number. To facilitate the differentiation of different areas, the area numbers may be displayed from left to right in the first image as 1, 2...n. The attribute information may also include the vehicle travel direction Dn of the lane area, which is divided into uphill and downhill. For the area with area number n, Tn=0 indicates that the area is a non-lane area, and Tn=1 indicates that the area is a lane area. Dn=0 indicates that the vehicle travel direction is uphill, and Dn=1 indicates that the vehicle travel direction is downhill.

[0050] Based on the initial recognition result, the corresponding area number, vehicle direction of travel, and other attribute information can be displayed within the area. If recognition fails, the default attribute information for the area can be displayed.

[0051] like Figure 3 As shown, the first image 30 can be displayed on the first layer of the first interface, and the first lane area 31, the non-lane area 32, and the second lane area 33 can be displayed on the second layer of the first interface. The vehicle driving direction can be displayed in the first lane area 31 and the second lane area 33 respectively. The vehicle driving direction in the first lane area 31 is upward, that is, from the bottom of the first image to the top of the first image, and the vehicle driving direction in the second lane area 33 is downward, that is, from the top of the first image to the bottom of the first image. The attribute information of the first lane area 31 can be represented as Tn=1, Dn=0, the attribute information of the non-lane area 32 can be represented as Tn=0, and the attribute information of the second lane area 33 can be represented as Tn=1, Dn=1.

[0052] The first image, from left to right, consists of the first lane area 31, the non-lane area 32, and the second lane area 33. The area number corresponding to the first lane area 31 is 1, and the area number "①" is displayed in the first lane area 31. The area number corresponding to the non-lane area 32 is 2, and the area number "②" is displayed in the second lane area 32. The area number corresponding to the second lane area 33 is 3, and the area number "③" is displayed in the second lane area 33.

[0053] In this embodiment of the application, by displaying a first image on a first interface and overlaying at least one lane area and at least one non-lane area, as well as the vehicle driving direction of each lane area, the identified lane area, vehicle driving direction, and non-lane area can be displayed more intuitively, so that the user can further adjust the lane area, the non-lane area, or the vehicle driving direction based on the first image.

[0054] In some embodiments, the first recognition result further includes a reference horizon line and a reference region segmentation line; step 104, "displaying at least the first image and the first recognition result on the first interface," further includes steps 1045b to 1047b:

[0055] Step 1045b: Determine the reference horizon line based on a first set of coordinates for at least one lane area, the reference horizon line being used to distinguish the target area from other areas in the first image;

[0056] A reference horizon line can be overlaid and rendered on the first image. The other areas can be sky areas. The reference horizon line can be used to distinguish the lane areas and non-lane areas in the first image from the sky areas. Figure 4 As shown, the maximum value h of the ordinate in the first coordinate set of the lane area can be selected. Using h as the ordinate, a horizontal line 41 is established, intersecting the left and right sides of the first image. The intersection points (0, h) and (a, h) are the manipulation points. This horizontal line 41 serves as a reference horizon line. The reference horizon line can be dragged up and down along the Y-axis of the coordinate system. The position of the reference horizon line can also be adjusted by moving the manipulation points. The rectangle formed by the reference horizon line and the border of the first image is the effective monitoring area of ​​the monitoring device, i.e., the target area. The rectangle's coordinates are {(0, 0), (0, h), (a, h), (a, 0)}, with an upper limit h. max and lower limit h min h max =b,h min For the preset minimum recommended horizon height, if h <h min This indicates that the installation angle or position of the monitoring equipment is too high. The monitoring equipment can be adjusted to a suitable position; otherwise, the monitoring effect will be affected.

[0057] Step 1046b: Determine the reference region segmentation line based on a first set of coordinates of at least one lane region and a second set of coordinates of at least one non-lane region, the reference region segmentation line being used to distinguish different target regions in the first image;

[0058] Specifically, reference region segmentation lines can be overlaid and rendered on the first image. The coordinates of the intersection points of the identified lane and non-lane region boundaries and the edge of the effective monitoring area are taken as the control point positions, distributed on the four sides of a rectangle, with the minimum distance between adjacent control points being d. minThe reference region dividing line can be dragged to move up and down, left and right. The shape and position of the reference region dividing line can also be adjusted by moving the control points, and the reference region dividing line can be scaled. The coordinates of the first control point clockwise from the top left vertex (0, h) (inclusive) are... The coordinates of the first control point in the counterclockwise direction are The line connecting the two points is the first reference region dividing line 42, which can be represented as: The second control point in the clockwise direction is The second control point in the counterclockwise direction is The line connecting the two points is the second reference region dividing line 43, which can be represented as: The third control point in the clockwise direction is The third control point in the counterclockwise direction is The line connecting the two points is the third reference region dividing line 44, which can be represented as: The 4th control point in the clockwise direction is The fourth control point in the counterclockwise direction is The line connecting the two points is the fourth reference region dividing line 45, which can be represented as: This process continues until the Nth reference region segmentation line is generated based on the first recognition result.

[0059] The region is defined by the shape enclosed by two adjacent reference region dividing lines, the border of the first image, and the reference horizon line. This region includes lane areas and non-lane areas. The number of regions is n, which is equal to the number of region dividing lines minus 1, and can be represented as n = N - 1. If recognition fails or N ≤ 1, N = 2 can be set to display two reference region dividing lines. The positions of the reference region dividing lines are as follows: and That is, one end of the control point (subscript 1) is on the reference horizon, and the other end of the control point (subscript 2) is on the bottom edge of the first image.

[0060] For region n, let the midpoint of adjacent control points with the same index (adjacent control points with index 1) at one end be... The endpoint of the adjacent control points with the same index at the other end (adjacent control points with index 2) is If the vehicles in the lane are traveling in the upward direction, then the vehicle travel direction can be represented as: If the vehicles in the lane are traveling in the downward direction, then the vehicle travel direction can be represented as:

[0061] It should be noted that each control point moves clockwise or counterclockwise between adjacent control points within the rectangle of the effective monitoring area enclosed by the reference horizon and the border of the first image, without overlapping or crossing points.

[0062] A region can have 4 manipulation points, and each side (including vertices) of an effective monitoring region can have a maximum of 2 manipulation points.

[0063] Step 1047b: Display the reference horizon line and the reference area dividing line in the second layer of the first interface.

[0064] In this embodiment of the application, by displaying a first image on a first interface, and overlaying at least one lane area, at least one non-lane area, the vehicle driving direction of each lane area, a reference horizon line, and the reference area dividing line on the first image, the identified lane area, vehicle driving direction, non-lane area, reference horizon line, and reference area dividing line can be displayed more intuitively, so that the user can make more flexible and convenient adjustments to the lane area, the non-lane area, the vehicle driving direction, the reference horizon line, and the reference area dividing line based on the first image.

[0065] In some embodiments, step 106, "in response to the acquired triggering operation, adjust the first identification result to obtain the target identification result," can be achieved through the following steps 1061 and 1062:

[0066] Step 1061: In response to the drag operation on the reference horizon line, move the display position of the reference horizon line to a first position, where the first position is the position of the actual horizon line in the first image;

[0067] Among them, such as Figure 6 As shown, the user can determine the position 62 of the real horizon line from the displayed first image, and when the position 61 of the reference horizon line is inconsistent with the position 62 of the real horizon line, the user can drag the reference horizon line to the first position 62 [(0,h),(a,h)] of the real horizon line in the first image, so as to keep the positions of the reference horizon line and the real horizon line consistent.

[0068] Step 1062: In response to the drag operation of the reference region segmentation line, move the display position of the reference region segmentation line to a second position, the second position being the position of the actual region segmentation line in the first image.

[0069] Similarly, such as Figure 6 As shown, the user can determine the position of the real region segmentation line 64 from the displayed first image, and when the position of the reference region segmentation line 63 is inconsistent with the position of the real region segmentation line 64, the user can drag the reference region segmentation line to the second position 64 where the real region segmentation line is located in the first image, thereby keeping the positions of the reference region segmentation line and the real region segmentation line consistent.

[0070] When dragging the reference horizon line and the reference area dividing line, the area's attribute information can be hidden until the drag is complete, thus reducing the interference of the displayed attribute information on the dragging action. After the drag is complete, the area number and attribute information Tn and Dn can be updated.

[0071] In this embodiment, by dragging the reference horizon line and the reference region segmentation line, the reference horizon line and the reference region segmentation line can be adjusted to be consistent with the position of the real horizon line and the real region segmentation line in the first image, thereby improving the accuracy of the first recognition result.

[0072] In some embodiments, step 106, "in response to the acquired triggering operation, adjust the first identification result to obtain the target identification result," can also be achieved through steps 1063 and 1064:

[0073] Step 1063: In response to a click operation on the reference region dividing line, add a Bates curve manipulation point;

[0074] Step 1064: In response to the drag operation of the Bates curve manipulation point, move the display position of the reference region segmentation line to a third position, the third position being the position of the real region segmentation line in the first image, the real region segmentation line being a curve.

[0075] Among them, such as Figure 6 As shown, when there is a curve in the first image, the corresponding real area segmentation line 66 should be a curve. You can click on the reference area segmentation line 65 to add Bates curve manipulation points 67. Where cpx and cpy are joystick coordinates, such as Figure 6 As shown, by dragging the Bates curve manipulation point 67, the reference area dividing line 65 corresponding to the curve can be adjusted to coincide with the actual area dividing line 66. The Bates curve manipulation point can be deleted, and after all points are deleted, the area dividing line can be restored to a straight line.

[0076] In this embodiment, by adding a Bates curve manipulation point when the lane is curved, the reference area segmentation line can be adjusted to a curve. This not only applies to regular effective monitoring areas, but also makes the reference area segmentation line fit the curve better when there is a curve, further improving the recognition accuracy.

[0077] In some embodiments, the first image of the target recognition result obtained after adjusting the first recognition result can also be added to the database as a second image with recognized features, so as to continuously improve the accuracy of the recognition model.

[0078] In some embodiments, the method further includes steps 107 to 109:

[0079] Step 107: Display adjustment components on the first interface, the adjustment components including quantity adjustment components and attribute adjustment components for each target area;

[0080] In one embodiment, such as Figure 5 As shown, the quantity adjustment component is also called the area quantity setting component, and the attribute adjustment component is also called the attribute setting component. The area attribute setting module includes an attribute setting component corresponding to each area. The number of identified areas is 2: area 1 and area 2. The area type of area 1 is lane and the vehicle driving direction is upward. The area type of area 2 is lane and the vehicle driving direction is upward.

[0081] In another embodiment, such as Figure 6 As shown, three areas were identified: Area 1, Area 2, and Area 3. The area type of Area 1, Area 2, and Area 3 are all lanes, and the vehicle travel direction is all upward.

[0082] Step 108: In response to the triggering operation of the quantity adjustment component, adjust the quantity of the target area to a first quantity, and adjust the quantity of the attribute adjustment component based on the first quantity;

[0083] Wherein, the first quantity is the actual quantity of the target area. If it is determined that the quantity of the target area does not match the number of the identified target areas, the user can adjust the quantity of the target area to the first quantity through the quantity adjustment component. Correspondingly, the total number of attribute adjustment components is also adjusted to the first quantity. For example, if the quantity of the target area is adjusted from 2 to 3, then the corresponding number of attribute setting components should be increased to 3. After the quantity of the target area is changed, the reference area dividing line and the reference horizon line can be restored to the automatically identified position or the default position.

[0084] Step 109: In response to the triggering operation of each of the aforementioned attribute adjustment components, adjust the attribute information of the corresponding target area.

[0085] In one embodiment, such as Figure 5 As shown, if the user determines that the vehicle's driving direction in region 2 should be downward based on the first image, the user can adjust the vehicle's driving direction to downward through the attribute setting component corresponding to region 2. Correspondingly, the arrow direction in region 2 in the first recognition result can be automatically adjusted to downward.

[0086] In another embodiment, such as Figure 6As shown, if the user determines from the first image that the region type of region 2 should be non-lane and the vehicle driving direction of region 3 should be downhill, the user can modify the region type to non-lane through the attribute setting component of region 2 and adjust the vehicle driving direction to downhill through the attribute setting component of region 3. Correspondingly, the arrow in region 2 in the first recognition result can be automatically deleted, and the arrow in region 3 can be automatically adjusted to downhill.

[0087] In this embodiment of the application, the number of regions can also be adjusted by the quantity adjustment component, and the attribute information corresponding to the regions can be adjusted by the attribute adjustment component, so that the number of regions and region attributes can be adjusted more flexibly.

[0088] In some embodiments, the method further includes steps 110 and 111:

[0089] Step 110: Based on the target recognition results, perform anomaly analysis on the vehicles on the first road to obtain the analysis results;

[0090] The system can store the coordinates of the control points within the area, the coordinates of the vertices of the first image, and the attribute information of each area, as well as other target recognition results, to be transmitted to the analysis module for subsequent analysis. The analysis results include whether the vehicle has abnormal behaviors such as crossing the line or driving in the wrong direction.

[0091] Step 111: If the analysis results indicate that the vehicle has abnormal behavior, output a prompt message.

[0092] In this embodiment of the application, the vehicle can be analyzed for abnormal behavior based on the target recognition results, and prompt information can be output when abnormal behavior is found, thereby improving road traffic safety.

[0093] With the development of the security industry, the installation of monitoring equipment to monitor vehicles and replace manual vehicle anomaly analysis and early warning is increasingly being applied to key scenarios such as roads and guard posts.

[0094] Figure 7 This is a flowchart illustrating another road condition recognition method according to an embodiment of this application, as shown below. Figure 7 As shown, the method includes the following steps:

[0095] Step 701: Acquire and identify road condition images collected by the monitoring equipment;

[0096] Step 702: Render and display a graphical user interface on the terminal device that includes at least road condition images and adjustment modules;

[0097] The graphical user interface also includes the recognition results. A road condition image is rendered on the terminal, and a coordinate system is established with the lower left vertex as the origin. The coordinates are [(0,0), (0,b), (a,b), (a,0)]. A reference horizon line is overlaid and rendered on the road condition image. The maximum value h of the ordinate in the lane area coordinate set is selected, and the reference horizon line coordinates are [(0,h), (a,h)]. A reference area segmentation line is overlaid and rendered on the road condition image. The intersection point of the recognized lane area and non-lane area boundary lines with the edge of the effective monitoring area is taken as the control point position, such as... Figure 6 As shown, four reference area dividing lines can be generated. The reference area dividing lines, the reference horizon line, and the image border enclose three areas, which are labeled 1, 2, and 3 from left to right, respectively, and it is recognized that all three areas are the uphill lane.

[0098] Step 703: Determine whether the recognition result is accurate; if yes, proceed to step 704; otherwise, proceed to step 705.

[0099] Depending on the actual situation, some inaccuracies in automatic recognition may require adjustment. For curves, you can click on the reference area dividing line to add Bates curve manipulation points, and drag the joystick to make the reference area dividing line coincide with the edge of the curve. After adjusting the monitoring area, set the attributes of areas 1, 2, and 3 to Lane-Upward, Non-Lane, and Lane-Downward.

[0100] Step 704: Store the coordinates of the manipulation points within the region and the values ​​of the region attributes;

[0101] Step 705: Adjust to match the actual situation;

[0102] Step 706: Pass the data to the analysis module for subsequent analysis.

[0103] In some embodiments, the reference horizon line (i.e., the effective rectangle coincides with the live image), Bates curve control points, and non-lane areas can be removed, and only the reference area dividing line is used to set the lane area and vehicle driving direction.

[0104] The training and recognition content in this application includes the horizon line, lane area, non-lane area, and vehicle travel direction. Based on automatic recognition, confirmation and setting methods are added to assist automatic recognition and improve recognition accuracy. This application is for road monitoring equipment and does not require real-time lane calculation.

[0105] This application, based on lane-based scenarios, is intuitive, easy to understand, and simple to operate. It reduces the impact of algorithm models, monitoring equipment installation conditions, road condition complexity, and monitoring environment on the recognition accuracy of existing solutions, thereby improving recognition accuracy. Installers can conveniently and quickly complete the installation and debugging of monitoring equipment on-site using mobile terminals, facilitating the large-scale successful deployment of monitoring equipment.

[0106] Figure 8 This is a schematic diagram illustrating the rendering and display of a graphical user interface on a terminal device according to an embodiment of this application, such as... Figure 8 As shown, the method includes the following steps:

[0107] Step 801: Render the road condition image on the terminal device and establish a coordinate system with the lower left vertex as the origin;

[0108] Step 802: Overlay and render a reference horizon line on the road condition image;

[0109] Step 803: Overlay and render reference area dividing lines on the road condition image;

[0110] Step 804: Overlay and render the attributes and attribute settings module on each area.

[0111] This application embodiment uses a terminal to customize the monitoring area and vehicle driving direction in the images captured by the monitoring equipment. Based on the lane scenario, it is intuitive, easy to understand, and simple to operate, reducing the impact of algorithm models, monitoring equipment installation conditions, road condition complexity, and monitoring environment on the recognition accuracy of existing solutions, and helping to improve recognition accuracy. Installers can conveniently and quickly complete the installation and debugging of monitoring equipment on-site according to the actual situation through mobile terminals, which helps to successfully deploy monitoring equipment on a large scale.

[0112] The embodiments of this application are low-cost. Compared to investing time and effort in improving algorithm models and optimizing for the impact of road conditions and environment on recognition accuracy, the embodiments of this application can quickly help improve the current recognition accuracy, with low R&D costs and good results.

[0113] The embodiments of this application offer a superior setup experience. The process and method for setting custom monitoring areas and vehicle driving directions, designed specifically for lane recognition scenarios, are intuitive, easy to understand, and simple to operate. Compared to polygonal or grid selection, especially for mobile terminals with smaller screens, it is easier to set up road conditions such as curves and multi-lane scenarios, and less prone to errors.

[0114] The embodiments of this application facilitate standardized installation. Installers can conveniently and quickly complete the installation and debugging of monitoring equipment on-site using mobile terminals, based on actual conditions. Features such as the display of road condition images, video adjustment lines, minimum spacing between control points for area division lines, etc., help installers position the monitoring screen at the appropriate location and angle, ensuring standardized installation of the monitoring equipment.

[0115] This application embodiment can be used in security equipment to monitor and warn vehicles in road and guard post scenarios to help improve lane line recognition accuracy. This solution improves recognition performance at a lower cost and significantly enhances the user experience of setting lane lines compared to existing solutions, especially in scenarios with many lanes, curves, or poor road conditions.

[0116] It should be noted that, in the embodiments of this application, if the above-described road condition recognition method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a mobile phone, tablet computer, desktop computer, personal digital assistant, navigator, digital phone, video phone, television, sensor device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0117] Figure 9 This is a schematic diagram of the composition structure of a road condition recognition device according to an embodiment of this application, as shown below. Figure 9 As shown, the device includes:

[0118] Module 91 is used to acquire road condition images captured by monitoring equipment;

[0119] The recognition module 92 is used to recognize road condition information in road condition images;

[0120] Display module 93 is used to display road condition images, reference horizon line, reference area dividing line, area number, and vehicle driving direction.

[0121] Adjustment module 94 is used to adjust the number of regions, region type, and manipulation points.

[0122] Figure 10 This is a schematic diagram of the composition structure of another road condition recognition device according to an embodiment of this application, as shown below. Figure 10 As shown, the device 1000 includes: a first acquisition module 1001, a first display module 1002, and a first adjustment module 1003, wherein:

[0123] The first acquisition module 1001 is used to acquire a first image representing the road condition information of the first road collected by the monitoring equipment on the first road, and to perform feature recognition on the first image to obtain a first recognition result; the first recognition result includes at least one target area and attribute information of each target area;

[0124] The first display module 1002 is used to display at least the first image and the first recognition result on the first interface;

[0125] The first adjustment module 1003 is used to adjust the first recognition result in response to the acquired trigger operation to obtain the target recognition result.

[0126] In some embodiments, the target area includes a lane area, and the attribute information includes the vehicle driving direction of the lane area; the first display module 1002 includes: a first display submodule, configured to display the first image on a first layer of a first interface; a first generation submodule, configured to establish a coordinate system of the first image based on the first recognition result to generate a first coordinate set for each lane area; a second display submodule, configured to display at least one lane area on a second layer of the first interface based on at least one of the first coordinate sets; the second layer is located above the first layer; and a third display submodule, configured to display the vehicle driving direction of the corresponding lane area within each lane area.

[0127] In some embodiments, the target area includes a lane area and a non-lane area, and the attribute information includes the vehicle driving direction and a first area identifier of the lane area, and a second area identifier of the non-lane area. The first display module 1002 includes: a fourth display submodule, used to display the first image on a first layer of a first interface; a second generation submodule, used to establish a coordinate system of the first image based on the first recognition result, to generate a first coordinate set for each lane area and a second coordinate set for each non-lane area; a fifth display submodule, used to display at least one lane area and at least one non-lane area on a second layer of the first interface based on at least one first coordinate set and at least one second coordinate set; the second layer is located above the first layer; a sixth display submodule, used to display the vehicle driving direction and the first area identifier of the corresponding lane area in each lane area; and a seventh display submodule, used to display the second area identifier of the corresponding non-lane area in each non-lane area.

[0128] In some embodiments, the first identification result further includes a reference horizon line and a reference region segmentation line; the first display module 1002 includes: a first determining submodule, configured to determine the reference horizon line based on a first set of coordinates of at least one lane region, the reference horizon line being used to distinguish target regions and other regions in the first image; a second determining submodule, configured to determine the reference region segmentation line based on a first set of coordinates of at least one lane region and a second set of coordinates of at least one non-lane region, the reference region segmentation line being used to distinguish different target regions in the first image; and an eighth display submodule, configured to display the reference horizon line and the reference region segmentation line on the second layer of the first interface.

[0129] In some embodiments, the first adjustment module 1003 includes: a first response submodule, configured to move the display position of the reference horizon line to a first position in response to a drag operation on the reference horizon line, the first position being the position of the actual horizon line in the first image; and a second response submodule, configured to move the display position of the reference region segmentation line to a second position in response to a drag operation on the reference region segmentation line, the second position being the position of the actual region segmentation line in the first image.

[0130] In some embodiments, the first adjustment module 1003 further includes: a third response submodule, configured to add a Bates curve manipulation point in response to a click operation on the reference region segmentation line; and a fourth response submodule, configured to move the display position of the reference region segmentation line to a third position in response to a drag operation on the Bates curve manipulation point, wherein the third position is the position of the real region segmentation line in the first image, and the real region segmentation line is a curve.

[0131] In some embodiments, the apparatus further includes: a second display module, configured to display adjustment components on the first interface, the adjustment components including a quantity adjustment component and an attribute adjustment component for each target area; a first response module, configured to adjust the quantity of the target area to a first quantity in response to a trigger operation on the quantity adjustment component, and adjust the quantity of the attribute adjustment components based on the first quantity; and a second response module, configured to adjust the attribute information of the corresponding target area in response to a trigger operation on each attribute adjustment component.

[0132] In some embodiments, the device further includes: an analysis module, configured to perform anomaly analysis on vehicles on the first road based on the target recognition result, and obtain analysis results; and an output module, configured to output prompt information when the analysis results indicate that the vehicles have abnormal behavior.

[0133] Correspondingly, embodiments of this application provide an electronic device, Figure 11 This is a schematic diagram of a hardware entity of an electronic device according to an embodiment of this application, such as... Figure 11 As shown, the hardware entity of the device 1100 includes a memory 1101 and a processor 1102. The memory 1101 stores a computer program that can run on the processor 1102. When the processor 1102 executes the program, it implements the steps in the road condition recognition method in the above embodiments.

[0134] The memory 1101 is configured to store instructions and applications executable by the processor 1102, and can also cache data to be processed or already processed by the processor 1102 and the various modules in the device 1100 (e.g., image data, audio data, voice communication data and video communication data), which can be implemented by flash memory or random access memory (RAM).

[0135] Correspondingly, embodiments of this application provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps in the road condition recognition method provided in the above embodiments.

[0136] It should be noted that the descriptions of the storage medium and device embodiments above are similar to those of the method embodiments above, and have similar beneficial effects as the device embodiments. For technical details not disclosed in the storage medium and method embodiments of this application, please refer to the descriptions of the device embodiments of this application for understanding.

[0137] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0138] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0139] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0140] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units; some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, the functional units in the embodiments of this application may all be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the integrated unit may be implemented in hardware or in a combination of hardware and software functional units.

[0141] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks. Alternatively, if the integrated units of this application are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to the related technology, can be embodied in the form of software products. These computer software products are stored in a storage medium and include several instructions to cause computer devices (which may be mobile phones, tablets, desktops, personal digital assistants, navigators, digital phones, video phones, televisions, sensing devices, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0142] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined to obtain new method embodiments without conflict. The features disclosed in the several product embodiments provided in this application can be arbitrarily combined to obtain new product embodiments without conflict. The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined to obtain new method embodiments or device embodiments without conflict.

[0143] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A road condition recognition method, characterized in that, The method includes: A first image representing the road condition information of the first road, collected by a monitoring device on the first road, is acquired, and feature recognition is performed on the first image to obtain a first recognition result; the first recognition result includes at least one target area and attribute information of each target area; The first interface displays at least the first image and the first recognition result; In response to the acquired trigger operation, the first recognition result is adjusted to obtain the target recognition result; The target area includes a lane area and a non-lane area. The attribute information includes the vehicle driving direction and a first area identifier of the lane area, and a second area identifier of the non-lane area. Displaying at least the first image and the first recognition result on the first interface includes: The first image is displayed on the first layer of the first interface; Based on the first recognition result, a coordinate system of the first image is established to generate a first coordinate set for each lane area and a second coordinate set for each non-lane area. Based on at least one first coordinate set and at least one second coordinate set, at least one lane area and at least one non-lane area are displayed on a second layer of the first interface; the second layer is located above the first layer. In each lane area, the vehicle travel direction and the first area identifier of the corresponding lane area are displayed; In each of the aforementioned non-lane areas, a second area identifier corresponding to the non-lane area is displayed; The first recognition result further includes a reference horizon line and a reference region segmentation line; the step of displaying at least the first image and the first recognition result on the first interface further includes: Based on a first set of coordinates for at least one lane region, a reference horizon line is determined, which is used to distinguish the target region from other regions in the first image. Based on a first set of coordinates for at least one lane area and a second set of coordinates for at least one non-lane area, a reference region segmentation line is determined, the reference region segmentation line being used to distinguish different target areas in the first image; The reference horizon line and the reference region dividing line are displayed in the second layer of the first interface.

2. The method according to claim 1, characterized in that, The step of adjusting the first recognition result in response to the acquired trigger operation to obtain the target recognition result includes: In response to the drag operation of the reference horizon line, the display position of the reference horizon line is moved to a first position, which is the position of the actual horizon line in the first image; In response to the drag operation of the reference region segmentation line, the display position of the reference region segmentation line is moved to a second position, which is the position of the actual region segmentation line in the first image.

3. The method according to claim 2, characterized in that, The step of adjusting the first recognition result in response to the acquired trigger operation to obtain the target recognition result further includes: In response to a click operation on the reference region dividing line, add a Bates curve manipulation point; In response to a drag operation on the Bates curve manipulation point, the display position of the reference region segmentation line is moved to a third position, which is the position of the real region segmentation line in the first image, and the real region segmentation line is a curve.

4. The method according to claim 1, characterized in that, The method further includes: The first interface displays adjustment components, which include quantity adjustment components and attribute adjustment components for each target area; In response to a trigger operation on the quantity adjustment component, the quantity of the target area is adjusted to a first quantity, and the quantity of the attribute adjustment component is adjusted based on the first quantity; In response to a trigger operation on each of the aforementioned attribute adjustment components, the attribute information of the corresponding target area is adjusted.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: Based on the target recognition results, anomaly analysis is performed on the vehicles on the first road to obtain the analysis results; If the analysis results indicate that the vehicle is exhibiting abnormal behavior, a prompt message will be output.

6. A road condition recognition device, characterized in that, The device includes: The first acquisition module is used to acquire a first image representing the road condition information of the first road collected by the monitoring equipment on the first road, and to perform feature recognition on the first image to obtain a first recognition result; the first recognition result includes at least one target area and attribute information of each target area; A first display module is configured to display at least the first image and the first recognition result on a first interface; The first adjustment module is used to adjust the first recognition result in response to the acquired trigger operation to obtain the target recognition result; The target area includes lane areas and non-lane areas. The attribute information includes the vehicle driving direction and a first area identifier of the lane areas, and a second area identifier of the non-lane areas. The first display module includes: a fourth display submodule for displaying the first image on a first layer of a first interface; a second generation submodule for establishing a coordinate system of the first image based on the first recognition result to generate a first coordinate set for each lane area and a second coordinate set for each non-lane area; a fifth display submodule for displaying at least one lane area and at least one non-lane area on a second layer of the first interface based on at least one first coordinate set and at least one second coordinate set; the second layer is located above the first layer; a sixth display submodule for displaying the vehicle driving direction and a first area identifier of the corresponding lane area in each lane area; and a seventh display submodule for displaying the second area identifier of the corresponding non-lane area in each non-lane area. The first recognition result further includes a reference horizon line and a reference region segmentation line; the first display module includes: a first determining submodule, used to determine the reference horizon line based on a first set of coordinates of at least one lane region, the reference horizon line being used to distinguish target regions and other regions in the first image; a second determining submodule, used to determine the reference region segmentation line based on a first set of coordinates of at least one lane region and a second set of coordinates of at least one non-lane region, the reference region segmentation line being used to distinguish different target regions in the first image; and an eighth display submodule, used to display the reference horizon line and the reference region segmentation line on the second layer of the first interface.

7. An electronic device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the road condition recognition method according to any one of claims 1 to 5.

Images