Road state determination method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application relate to the technical field of intelligent transportation, and in particular to a road state determination method and device, an electronic device, and a storage medium. The method comprises receiving a reference road image sent by a central control terminal; determining a boundary line of a road; constructing a ground curve equation of the boundary line according to positions of multiple calibration points located on the boundary line; and sending the ground curve equation to the central control terminal to instruct the central control terminal to determine a current road state according to a current road image collected in real time according to the ground curve equation. The server can determine the road boundary according to the road image uploaded by the roadside terminal, so that the roadside terminal identifies target objects in the road boundary in the road image with the same field of view and higher real-time performance, removes the invalid recognition range of the roadside terminal, and makes the roadside terminal apply its own computing power to the effective recognition range. The target objects are identified from the road at a faster speed and with higher accuracy, and the traffic state of the road is determined.

Description

Technical Field

[0001] This application relates to the field of intelligent transportation technology, and in particular to a method, apparatus, electronic device, and storage medium for determining road conditions. Background Technology

[0002] In the field of intelligent transportation, identifying targets on roads that may affect traffic flow, such as pedestrians, vehicles, debris, obstacles, and traffic lights, can effectively prevent traffic accidents. Current technologies primarily employ the following two methods to identify targets on roads: 1. Identification using roadside central control terminals. Roadside terminals control and process road images captured by cameras. The advantage of this approach is the high real-time performance of the road images used for recognition. The disadvantage is that roadside terminals have limited computing power, making it difficult to handle too many recognition tasks, resulting in either low recognition accuracy or slow recognition speed. For example, in practical applications, the recognition of objects on the road often uses deep learning-based models for inference. These models typically perform better with more parameters, more complex network structures, and more computations. Roadside terminals, due to their limited computing resources, often cannot achieve satisfactory results.

[0003] II. Identification using a cloud server comprised of control terminals. The cloud-based control terminal identifies road images uploaded by the terminal. The advantage of this method is that the cloud-based control terminal has significantly more computing resources than the terminal, ensuring high recognition accuracy and speed. The disadvantage is that due to communication latency, the real-time performance of the road images used for recognition by the control terminal is poor.

[0004] The inability to simultaneously guarantee the speed, accuracy, and real-time performance of object recognition in road images, and thus to more accurately and in real-time determine the current traffic status of roads, is one of the problems that urgently need to be solved in this field. Summary of the Invention

[0005] To address one of the aforementioned technical deficiencies, this application provides a road condition determination method, apparatus, electronic device, and storage medium.

[0006] According to a first aspect of the embodiments of this application, a road state determination method is provided, applied to a control terminal, comprising: Receive reference road images sent by the central control terminal; Determine the road boundary lines of the road in the reference road image; The ground curve equation of the road boundary line is constructed based on the positions of multiple calibration points located on the road boundary line; The ground curve equation is sent to the central control terminal to instruct the central control terminal to determine the current road status based on the real-time acquired current road image according to the ground curve equation; wherein, the current road image and the reference road image are road images acquired from the same field of view.

[0007] According to a second aspect of the embodiments of this application, a road state determination method is provided, applied to a central control terminal, including: The ground curve equation is received from the control terminal; wherein the ground curve equation is determined according to any of the road condition determination methods mentioned above. Based on the ground curve equation, determine the area within the range defined by the ground curve equation in the current road image acquired in real time by the image acquisition device; Determine the current road status based on the regional image.

[0008] According to a third aspect of the embodiments of this application, a road state determination device is provided, applied to a control terminal, the road state determination device comprising: The first receiving module is used to receive reference road images sent by the central control terminal; The first determining module is used to determine the road boundary line of the road in the reference road image; The module is used to construct the ground curve equation of the road boundary line based on the positions of multiple calibration points located on the road boundary line; The second determining module is used to send the ground curve equation to the central control terminal, so as to instruct the central control terminal to determine the current road status based on the ground curve equation and the current road image collected in real time; wherein, the current road image and the reference road image are road images collected from the same field of view.

[0009] According to a fourth aspect of the embodiments of this application, a road condition determination device is provided, applied to a central control terminal, comprising: The second receiving module is used to receive the ground curve equation sent by the control terminal; wherein the ground curve equation is determined according to any of the above-mentioned road condition determination methods; The third determining module is used to determine the area image within the range defined by the ground curve equation in the current road image acquired in real time by the image acquisition device, based on the ground curve equation. The fourth determination module is used to determine the current road status based on the regional image.

[0010] According to a fifth aspect of the embodiments of this application, an electronic device is provided, comprising: Memory; Processor; and Computer programs; The computer program is stored in memory and configured to be executed by a processor to implement the road state determination method according to any of the above.

[0011] According to a sixth aspect of the present application, a computer-readable storage medium is provided having a computer program stored thereon; the computer program is executed by a processor to implement the road state determination method according to any of the preceding claims.

[0012] By employing the road state determination method provided in this application embodiment, the cloud server can determine the road boundary range based on the road images uploaded by the roadside terminal. This enables the roadside terminal to identify target objects such as pedestrians and vehicles within the road boundary range in road images with high real-time performance and the same field of view. This eliminates invalid recognition ranges for the roadside terminal, reduces its computational load, and allows it to utilize its computing power on the effective recognition range. This results in faster and more accurate identification of target objects within the road, thus determining the road's traffic state. This solves the technical problem of simultaneously ensuring recognition speed, accuracy, and real-time performance in identifying objects in road images. It achieves the technical effect of roadside terminal and cloud server cooperating to identify road images, ensuring both real-time performance and balancing recognition speed and accuracy. Attached Figure Description

[0013] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A flowchart of a road state determination method provided in one embodiment of this application; Figure 2 A flowchart of a road state determination method provided in one embodiment of this application; Figure 3 A flowchart of a road state determination method provided in one embodiment of this application; Figure 4 This is a schematic diagram of the ground boundary curve in an embodiment of this application; Figure 5 A flowchart of a road state determination method provided in one embodiment of this application; Figure 6 A flowchart of a road state determination method provided in one embodiment of this application; Figure 7 A flowchart of a road state determination method provided in one embodiment of this application; Figure 8A flowchart of a road condition determination device provided in one embodiment of this application; Figure 9 A schematic diagram of a road condition determination device provided in one embodiment of this application; Figure 10 This is a schematic diagram of a computer device structure provided in one embodiment of this application. Detailed Implementation

[0014] To make the technical solutions and advantages of the embodiments of this application clearer, the exemplary embodiments of this application will be described in further detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not an exhaustive list of all embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0015] In the process of developing this application, the inventors discovered that when monitoring and analyzing road conditions, roadside central control devices are typically directly connected to image acquisition devices or other types of road information acquisition devices. Therefore, roadside central control devices can obtain road images with relatively high real-time performance. However, due to the limited computing power of the roadside devices, the processing and analysis of road images is slow and the accuracy is low. Cloud servers, on the other hand, have abundant computing resources but cannot directly obtain road images; they need to obtain them through communication, resulting in road images with poor real-time performance.

[0016] In view of the above problems, this application provides a road state determination method applied to a cloud server composed of control terminals. The cloud server analyzes road images uploaded by a central control terminal to obtain road boundary curves, and sends these curves to roadside terminals to instruct the central control terminal. Another method is provided whereby a roadside central control terminal receives curves and analyzes them to identify pedestrians, vehicles, and other target objects within the road boundaries in road images with the same field of view as the uploaded road images and high real-time performance. These methods can narrow the identification and analysis range of the roadside central control terminal, improving its analysis speed and accuracy. Furthermore, since the road images analyzed by the central control terminal are high-real-time images, real-time performance is also taken into account.

[0017] The following is a brief introduction to the application environment of the road state determination method provided in the embodiments of this application: The execution subject of the method in some embodiments of this application is a control terminal. The control terminal can be a server, or a server cluster consisting of multiple servers, or a cloud server or a distributed server network, etc. This is not an exhaustive list, and can be selected or set according to the actual situation.

[0018] In another embodiment of this application, the method is implemented by a central control terminal located on the roadside, which can control an image acquisition device (e.g., a camera) to capture road images. The central control terminal can be a desktop computer, an industrial computer, a mobile terminal, etc.

[0019] like Figure 1 As shown in the embodiment of this disclosure, a road state determination method is applied to a control terminal, including the following steps 101-104: including: Step 101: Receive the reference road image sent by the central control terminal.

[0020] In this embodiment, road conditions refer to various factors affecting road traffic, such as the positions of vehicles and pedestrians, the patterns and colors of traffic lights, and obstacles in the road (debris, road construction, etc.). The reference road image and the current road image were captured at different times, resulting in poorer real-time performance of the reference road image compared to the current road image.

[0021] Step 102: Determine the road boundary lines of the road in the reference road image.

[0022] Road boundary lines are the dividing lines used to distinguish the area inside and outside the road. Road boundary lines typically consist of four lines: two straight lines perpendicular to the road direction, marking the boundaries at both ends of the road, and two curved lines marking the boundaries on either side of the road. Road boundary lines can be determined using a trained road boundary recognition model. This model can be trained using multiple samples of manually labeled road boundaries. Alternatively, road boundary lines can be identified based on the features of lane lines and curbs.

[0023] Step 103: Construct the ground curve equation of the road boundary line based on the positions of multiple calibration points located on the road boundary line; The calibration points are randomly selected pixels on the road boundary lines of the reference road image. The equations for the ground curves correspond to the equations for the four boundary lines mentioned above. To obtain more accurate boundary lines, the calibration points should not be too close together; the distance between adjacent calibration points can be set to be no less than a specific pixel coordinate distance.

[0024] Step 104: Send the ground curve equation to the central control terminal to instruct the central control terminal to determine the current road status based on the real-time collected current road image according to the ground curve equation; wherein, the current road image and the reference road image are road images collected from the same field of view.

[0025] Since the reference road image and the current road image are acquired from the same field of view, the location of the road in the reference road image is the same as that in the current road image. Therefore, the road boundary line in the reference road image and the road boundary line in the current road image are in exactly the same position. The ground curve equation representing the road boundary line in the reference road image can also be used to represent the road boundary line in the current road image. The central control terminal can determine whether each pixel in the current road image is within the road boundary line range based on the ground curve equation, and remove all pixels that are not within the range, reducing the number of pixels used for road condition analysis and improving the efficiency of road image analysis.

[0026] By employing the road state determination method provided in this application embodiment, the cloud server can determine the road boundary range based on the road images uploaded by the roadside terminal. This enables the roadside terminal to identify target objects such as pedestrians and vehicles within the road boundary range in road images with high real-time performance and the same field of view. This eliminates invalid recognition ranges for the roadside terminal, reduces its computational load, and allows it to utilize its computing power on the effective recognition range. This results in faster and more accurate identification of target objects within the road, thus determining the road's traffic state. This solves the technical problem of simultaneously ensuring recognition speed, accuracy, and real-time performance in identifying objects in road images. It achieves the technical effect of roadside terminal and cloud server cooperating to identify road images, ensuring both real-time performance and balancing recognition speed and accuracy.

[0027] In one optional embodiment of this application, such as Figure 2 As shown, a method for determining obstacles on a road is provided, including the following steps 201-202: Step 201: Determine whether the road boundary of the reference road image contains a preset warning object. Preset warning objects can be fixed-location debris, warning signs (such as warning signs placed by disabled vehicle owners), construction signs, etc. A trained warning object recognition model can be used to determine whether the reference road image contains the preset warning object.

[0028] Step 202: If the road boundary of the reference road image contains a preset warning object, then generate warning information and send the warning information to the central control terminal.

[0029] Warning messages can vary depending on the identified pre-defined warning objects. For example, a construction sign indicating that the entire road is under construction might prompt a warning message advising drivers to use alternative routes. Upon receiving the warning message, the central control terminal can forward it to terminals in nearby vehicles. Since the locations of the pre-defined warning objects are fixed, real-time analysis of these objects is not critical. Utilizing servers to analyze these objects can reduce the workload of the roadside central control terminals.

[0030] In one optional embodiment of this application, a method for determining the state of traffic indicator devices such as traffic lights on a road is provided, comprising: if the road boundary of a reference road image contains traffic indicator devices, determining the reference indicator state and the location of the traffic indicator devices, and sending the reference indicator state and the location of the traffic indicator devices to a central control terminal, so as to instruct the central control terminal to determine the current indicator state from the real-time acquired current road image based on the reference indicator state and the location.

[0031] Traffic control devices can be traffic lights or other types of traffic indicators used to direct traffic. A recognition model for traffic control devices can be used to determine if the road boundary of a road image contains traffic control devices. The reference indication state of the traffic control device can be its display state in a reference road image. For example, a traffic light might be green in a reference road image; green represents a reference traffic indication state. The location of the traffic control device can be its outline or the pixel position of its center point.

[0032] For example, a server on a control terminal with high computing power can directly determine that the traffic light color in a reference road image (which has lower real-time requirements) is green. Then, the color data of the traffic light pixels in the current road image (which has higher real-time requirements) is compared with the green color data of the traffic light pixels in the reference image, and a difference operation is performed to obtain the color calculation result. If the difference is small, and the color calculation result is less than a certain threshold, it can be determined that the color of the traffic light in the current road image is the same as the color of the traffic light in the reference road image, and the color of the traffic light in the current road image is also green. If the difference is large, and the color calculation result is greater than a certain threshold, the time difference between the two can be calculated based on the timestamps of the reference road image and the current road image. The order of the traffic light color changes (the order of color changes and the duration of each color) can then determine the color of the traffic light in the current road image. This reduces the computational workload of determining the traffic light color in the current road image.

[0033] This application embodiment utilizes a cloud server to determine the reference indication status and location, which can instruct the central control terminal to directly locate the traffic indication device, reducing the computational workload of the central control terminal in determining the location of the traffic indication device. Furthermore, the central control terminal can obtain the current color of the traffic indication device by judging whether the current color of the traffic indication device has changed compared to the color in the reference road image, which reduces the computational workload compared to the central control terminal independently judging the current color based on the image.

[0034] In one optional embodiment of this application, such as Figure 3 As shown, determining objects above the road within the road boundary area in the radar's laser data includes the following steps 301-304: Step 301: Based on the pre-configured position transformation relationship and ground curve equation, determine the surface equation in the three-dimensional coordinate system of the lidar corresponding to the ground curve equation.

[0035] In this embodiment, the central control terminal can also use a lidar to scan the road area to obtain laser data. The lidar and image acquisition device can be jointly calibrated. The transformation relationship between the position of each pixel in the road image captured by the image acquisition device and the position of the laser point scanned by the lidar is determined (usually a coordinate transformation equation composed of a transformation matrix between pixel coordinates and laser coordinates). The calibration method can be to place a calibration board on the road at laser coordinates, photograph the calibration board using the image acquisition device, and determine the transformation matrix in the transformation equation using the pixel coordinates and laser coordinates of the calibration board in the image, thereby obtaining the transformation equation representing the transformation relationship.

[0036] According to the transformation relationship, the equation of the ground curve can be mapped to the equation of the surface used to define the space enclosing the road boundary in the laser coordinate system.

[0037] Step 302: Determine the ground laser point cloud of all laser points within the surface space bounded by the surface equation in the original laser data; Raw laser data can be obtained by LiDAR scanning the road when there are no people or vehicles on the road. The raw laser data consists of a laser point cloud within a curved surface space, corresponding to points on the road surface. A laser point is a point scanned by the LiDAR. A laser point cloud is a collection of laser points.

[0038] Step 303: Determine the ground fitting equation corresponding to the road surface based on the three-dimensional marker coordinates of at least four laser marker points in the ground laser point cloud that are more than calibrated by a distance between them. The ground fitting equation can be determined based on the coordinates (x-coordinate, y-coordinate, and altitude coordinates) of multiple points on the ground. The solution process can begin by establishing a ground fitting equation Ax + By + Cz = D containing four unknowns (A, B, C, and D), which needs to be solved using the coordinates of at least four points (laser markers). Furthermore, the laser markers should not be too densely packed; to reflect the coordinate characteristics of points across the entire plane, the four laser markers need to be sufficiently far apart, with a distance greater than a calibration threshold.

[0039] Step 304: Send the ground fitting equation to the central control terminal so that the central control terminal removes ground laser point clouds from the current laser point cloud of the current laser data that have a distance of less than a predetermined threshold between the planes corresponding to the ground fitting equation, thereby obtaining the road laser point cloud of the object on the road, and then identifying the target object based on the data of the road laser point cloud. The original laser data and the current laser data are obtained by scanning the same area using a lidar. The scanning area includes the road boundary range. The original laser data is obtained by the lidar scanning the area when there are no objects on the road in the scanning area.

[0040] The method in this application embodiment can obtain the surface equation of the curved surface used to define the road boundary range of the lidar scan, and can determine the ground fitting equation of the corresponding ground plane. This instructs the central control terminal to determine the laser point cloud of all laser points in the road boundary surface space based on the surface equation. Then, based on the ground fitting equation, the data of all laser points on the ground are removed from the laser point cloud. The laser point cloud composed of laser points above the road within the road boundary range is analyzed. Only the laser point cloud of objects above the road is analyzed, which reduces the amount of data used by the central control terminal for analysis and improves the analysis speed and accuracy of the central control terminal.

[0041] In one optional embodiment of this application, a method for determining the equation of a ground curve is proposed, comprising: For each straight line, determine the equation of the straight line based on the coordinates of at least two calibration points on the straight line; for each curve, determine the equation of the curve based on the coordinates of at least five calibration points on the curve.

[0042] like Figure 4 As shown, the ground boundary curve consists of two straight lines and two curves.

[0043] This application provides a method for determining the equation of a ground curve. Based on the characteristics of the lines contained in the ground curve equation, a method is proposed to determine the line equations of each part constituting the road boundary line, which can more accurately determine the ground curve equation.

[0044] In one optional embodiment of this application, such as Figure 5As shown, a method for determining obstacles on a road is provided, applied to a central control terminal, including steps 501-503: Step 501: Receive the ground curve equation sent by the control terminal; wherein the ground curve equation is determined according to any of the road condition determination methods mentioned above; Step 502: Based on the ground curve equation, determine the area image within the range defined by the ground curve equation in the current road image acquired in real time by the image acquisition device. The process involves determining the area within the curve equation's bounded region of the image, specifically whether each pixel's coordinates fall within this bounded area. For example, consider the two straight lines representing the ends of a road perpendicular to its direction and the two curves representing the sides of the road. These curves are located on the left and right sides of the image, respectively. If a pixel's coordinates indicate it's to the left of the left curve, it's considered outside the road boundary; conversely, if a pixel's coordinates indicate it's to the right of the right curve, it's considered outside the road boundary.

[0045] Step 503: Determine the current road status based on the regional image.

[0046] The method applied to the central control terminal in this application embodiment can extract regional images within the road boundary area from the captured road images based on the curve equation used to define the road boundary, and determine the road traffic status based on the regional images. This reduces the analysis range of the central control terminal and improves the analysis speed and accuracy of the central control terminal in analyzing road traffic status.

[0047] In one optional embodiment of this application, determining the area image within the range defined by the ground curve equation in the current road image acquired in real time by the image acquisition device includes: For each pixel in the real-time acquired current road image, if the pixel is within the closed area enclosed by the ground curve equation, the pixel is retained; if the pixel is not within the closed area, the pixel is deleted.

[0048] This application provides a method for extracting regional images within a range defined by a ground curve equation, using pixels as the most basic unit. This method can determine whether each pixel is within the road boundary line based on the ground surface equation, and then determine whether the data of that pixel should be retained. Therefore, it can more accurately obtain the data of all pixels in the corresponding road boundary area in a real-time captured road image based on the ground surface equation, and thus more accurately identify objects in the road area based on the data of that area, thereby obtaining a more accurate traffic status.

[0049] In one optional embodiment of this application, such as Figure 6As shown, a method for determining the color of traffic indicator devices when the current road condition is represented by different colors is provided, including steps 601-604: Step 601: Receive the reference indication status and the location of the traffic indication device sent by the control terminal.

[0050] Step 602: Determine the current indication status of the traffic indication device in the area image based on the location of the traffic indication device.

[0051] Traffic guidance devices can use colors to guide traffic. The colors of traffic guidance devices can indicate whether the current road conditions are suitable for vehicles or pedestrians to pass. Taking traffic lights as an example, the current indication state and the reference indication state are represented by the colors of multiple pixels in the area where the traffic light is located.

[0052] Step 603: Perform a difference operation on the first color component, the second color component, and the third color component of the reference indication state one-to-one with the same color component in the current indication state to obtain three calculation results. Then, sum the three calculation results to obtain the color calculation result.

[0053] Step 604: If the color calculation result is less than or equal to the color change threshold, then determine the color components of the current indication state as the first color component, the second color component, and the third color component of the reference indication state.

[0054] The colors of the current and reference indication states can be composed of three RGB components (corresponding to the first, second, and third color components). Determining the current indication state means determining the RGB components of the pixels in the area where the traffic indication device is located. Differential operations are performed on the three color components of the current and reference indication states, and the results are summed. If the sum of the differences is very small, less than a set threshold, it indicates that the colors of the current and reference indication states are essentially the same and have not changed. The computational complexity of calculating the difference sum for color changes is less than that of determining the actual color. Therefore, this embodiment determines that a reference color is obtained from a server, and then the color is judged based on the reference color to reduce the computational complexity of determining the color of the traffic indication device.

[0055] In an optional embodiment of this application, if the color calculation result is greater than the color change threshold, the color component corresponding to the current indication state is determined based on the current timestamp of the current road image, the reference timestamp of the reference road image, the first color component, the second color component, the third color component, and the color change order of the traffic indication device.

[0056] As mentioned above, the color change sequence of traffic signage includes the duration of each color and the order in which the colors change. Based on the current timestamp of the current road image and the reference timestamp of the reference road image, the time difference between the captured color of the current signage state and the captured color of the reference signage state can be calculated. Based on the time difference and the color change sequence, the color of the current signage state can be determined.

[0057] For example, if a traffic light is green in a reference road image, and the color of the traffic light has changed, the color change sequence is green-yellow-red-yellow-green…, with the green light lasting 30 seconds, the yellow light lasting 10 seconds, and the red light lasting 30 seconds. If the time difference between the timestamp of the reference road image and the timestamp of the current road image is 50 seconds, then the current traffic light color is determined to be red.

[0058] In this embodiment of the application, when it is determined that the color of the traffic sign device has changed, the real-time color of the traffic sign device can be obtained based on the reference color of the server that controls the interruption and the color change pattern of the traffic sign device. This reduces the amount of computation compared to determining the color based on the image of the traffic sign device.

[0059] In one optional embodiment of this application, such as Figure 7 As shown, obtaining the laser point cloud above the road within the road boundary includes steps 701-704: Step 701: Obtain the current laser data for the current road surface scan, as well as the surface equation and ground fitting equation sent by the control terminal; Step 702: Determine the road laser point cloud of all laser points within the surface space bounded by the surface equation in the current laser data based on the surface equation; Step 703: Remove ground laser point clouds from the road laser point cloud where the distance between the planes corresponding to the ground fitting equation and the ground is less than a predetermined distance, to obtain the road laser point cloud of the object above the road. Step 704: Determine the current road status based on the laser coordinates of each laser point in the three-dimensional coordinate system of the laser point cloud on the road.

[0060] The central control terminal in this application embodiment can determine the laser point cloud within the road boundary range in the laser data obtained by LiDAR scanning, and remove the laser point cloud of the road in the laser data according to the road fitting equation obtained by scanning the road to obtain the laser point cloud of the laser points above the road. Therefore, by specifically analyzing the laser data of objects above the road, the amount of data used to analyze objects on the road can be reduced, the analysis speed can be improved, and the real-time performance of the analysis can be enhanced.

[0061] In one optional embodiment of this application, such as Figure 8As shown, a road condition determination device 800 is provided, applied to a control terminal. The road condition determination device 800 includes a first receiving module 810, a first determining module 820, a constructing module 830, and a second determining module 840. The first receiving module 810 is used to receive reference road images sent by the central control terminal; The first determining module 820 is used to determine the road boundary line of the road in the reference road image; The building module 830 is used to construct the ground curve equation of the road boundary line based on the positions of multiple calibration points located on the road boundary line; The second determining module 840 is used to send the ground curve equation to the central control terminal to instruct the central control terminal to determine the current road status based on the ground curve equation and the current road image collected in real time; wherein the current road image and the reference road image are road images collected from the same field of view.

[0062] In an optional embodiment of this application, the first determining module 820 is further configured to determine whether the road boundary of the reference road image contains a preset warning object: if the road boundary of the reference road image contains a preset warning object, then a warning message is generated and the warning message is sent to the central control terminal.

[0063] In an optional embodiment of this application, the first determining module 820 is further configured to, if the road boundary of the reference road image contains a traffic indicating device, determine the reference indicating state of the traffic indicating device and the location of the traffic indicating device, and send the reference indicating state and the location of the traffic indicating device to the central control terminal, so as to instruct the central control terminal to determine the current indicating state from the real-time acquired current road image according to the reference indicating state and the location.

[0064] In an optional embodiment of this application, the construction module 830 is further configured to: determine the surface equation in the three-dimensional coordinate system of the lidar corresponding to the ground curve equation based on the pre-configured position transformation relationship and the ground curve equation; determine the ground laser point cloud of all laser points in the original laser data within the surface space enclosed by the surface equation; determine the ground fitting equation corresponding to the road surface based on the three-dimensional marker coordinates of at least four laser marker points in the ground laser point cloud whose distance to each other is greater than a calibration threshold; send the ground fitting equation to the central control terminal so that the central control terminal removes the ground laser point cloud whose distance to the plane corresponding to the ground fitting equation is less than a predetermined threshold from the current laser point cloud of the current laser data, thereby obtaining the road laser point cloud of the object on the road, and then identifying the target object based on the data of the road laser point cloud; The original laser data and the current laser data are obtained by scanning the same area using a lidar. The scanning area includes the road boundary range. The original laser data is obtained by the lidar scanning the area when there are no objects on the road in the scanning area.

[0065] In an optional embodiment of this application, the construction module 830 is specifically used to: for each straight line, determine the equation of the straight line based on the coordinates of at least two calibration points on the straight line; and for each curve, determine the equation of the curve based on the coordinates of at least five calibration points on the curve.

[0066] In one optional embodiment of this application, such as Figure 9 As shown, a road condition determination device 900 is provided, applied to a central control terminal, including a second receiving module 910, a third determining module 920, and a fourth determining module 930: The second receiving module 910 is used to receive the ground curve equation sent by the control terminal; wherein the ground curve equation is determined according to any of the road condition determination methods mentioned above; The third determining module 920 is used to determine the area image within the range defined by the ground curve equation in the current road image acquired in real time by the image acquisition device, based on the ground curve equation. The fourth determining module 930 is used to determine the current road status based on the regional image.

[0067] In an optional embodiment of this application, the third determining module 920 is specifically used to, for each pixel in the real-time acquired current road image, retain the pixel if it is within the closed area enclosed by the ground curve equation, and delete the pixel if it is not within the closed area.

[0068] In one optional embodiment of this application, the current road state is represented by different colors of the traffic indicator device; the third determining module is further configured to: receive a reference indication state and the location of the traffic indicator device sent by the control terminal; determine the current indication state of the traffic indicator device in the area image based on the location of the traffic indicator device; perform a difference operation on the first color component, the second color component, and the third color component of the reference indication state one-to-one with the same color component in the current indication state to obtain three calculation results, and sum the three calculation results to obtain a color calculation result; if the color calculation result is less than or equal to the color change threshold, then determine the color components of the current indication state as the first color component, the second color component, and the third color component of the reference indication state.

[0069] In an optional embodiment of this application, the third determining module 920 is further configured to, if the color calculation result is greater than the color change threshold, determine the color component corresponding to the current indication state based on the current timestamp of the current road image, the reference timestamp of the reference road image, the first color component, the second color component and the third color component, and the color change order of the traffic indication device.

[0070] In one embodiment, a computer device is provided, the internal structure of which can be as follows: Figure 10 As shown. The computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores data. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements the road state determination method described above. It includes: memory and a processor; the memory stores a computer program; and the processor executes the computer program to implement any step of the road state determination method described above.

[0071] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented in various computer languages, such as C, VHDL, Verilog, the object-oriented programming language Java, and the interpreted scripting language JavaScript.

[0072] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0073] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0074] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0075] In the description of this application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0076] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0077] In this application, unless otherwise expressly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection, an electrical connection, or a connection that allows communication between them; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication between two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0078] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0079] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for determining road conditions, characterized in that, Applied to a cloud server composed of control terminals, the control terminals perform the following steps: Receive reference road images sent by the central control terminal on the roadside; Determine the road boundary lines of the roads in the reference road image; The ground curve equation of the road boundary line is constructed based on the positions of multiple calibration points located on the road boundary line; The ground curve equation is sent to the central control terminal, which then performs the following steps: Receive the ground curve equation sent by the control terminal; Based on the ground curve equation, the area image within the range defined by the ground curve equation in the current road image acquired in real time by the image acquisition device is determined; wherein, the current road image and the reference road image are road images acquired from the same field of view; the current road state is represented by different colors of the traffic signage device; Determining the current road status based on the region image includes: Receive the reference indication status and the location of the traffic indication device sent by the control terminal; Determine the current indication status of the traffic indication device in the area image based on the location of the traffic indication device; The first color component, the second color component, and the third color component of the reference indication state are respectively matched one-to-one with the same color component in the current indication state. Differential operations are performed to obtain three calculation results, and the three calculation results are summed to obtain the color calculation result. If the color calculation result is less than or equal to the color change threshold, then the color components of the current indication state are determined to be the first color component, the second color component, and the third color component of the reference indication state; The step of determining the area image within the range defined by the ground curve equation in the current road image acquired in real time by the image acquisition device, based on the ground curve equation, includes: For each pixel in the real-time acquired current road image, if the pixel is within the closed area enclosed by the ground curve equation, the pixel is retained; if the pixel is not within the closed area, the pixel is deleted.

2. The road condition determination method according to claim 1, characterized in that, After determining the road boundary lines of the road in the reference road image, the method further includes: If the traffic sign is contained within the road boundary of the reference road image, then the reference indication status of the traffic sign and the location of the traffic sign are determined. The reference indication status and the location of the traffic indication device are sent to the central control terminal to instruct the central control terminal to determine the current indication status from the real-time acquired current road image based on the reference indication status and the location.

3. The road condition determination method according to claim 1, characterized in that, After constructing the ground curve equation of the road boundary line based on the positions of multiple calibration points located on the road boundary line, the method further includes: Based on the pre-configured position transformation relationship and the ground curve equation, determine the surface equation in the three-dimensional coordinate system of the lidar corresponding to the ground curve equation; The ground laser point cloud is determined by identifying all laser points within the surface space bounded by the surface equation in the original laser data; the original laser data is the data obtained by the lidar scanning the scanning area when there are no objects above the road in the scanning area. The ground fitting equation corresponding to the road surface is determined based on the three-dimensional marker coordinates of at least four laser marker points in the ground laser point cloud whose distances to each other are greater than a calibration threshold. The ground fitting equation is sent to the central control terminal so that the central control terminal removes ground laser point clouds from the current laser point cloud of the current laser data that have a distance of less than a predetermined threshold between the planes corresponding to the ground fitting equation, thereby obtaining the road laser point cloud of the object above the road. The target object is identified based on the laser point cloud data on the road; the target object is a pedestrian or a vehicle. The original laser data and the current laser data are data obtained by scanning the same scanning area using the lidar. The scanning area includes the road boundary range. The original laser data is data obtained by the lidar scanning the scanning area when there are no objects above the road in the scanning area.

4. The road condition determination method according to claim 1, characterized in that, The road boundary line includes two side curves located on both sides of the road and two straight lines located at both ends of the road. The step of constructing the ground curve equation of the road boundary line based on the positions of multiple calibration points located on the road boundary line includes: For each straight line, the equation of the straight line is determined based on the coordinates of at least two calibration points on the straight line. For each curve, the curve equation is determined based on the coordinates of at least five calibration points on the curve.

5. A road condition determination device, characterized in that, include: The first receiving module is used to receive reference road images sent by the central control terminal on the roadside; The first determining module is used to determine the road boundary line of the road in the reference road image; A construction module is used to construct the ground curve equation of the road boundary line based on the positions of multiple calibration points located on the road boundary line; The second determining module is used to send the ground curve equation to the central control terminal, so as to instruct the central control terminal to determine the current road status based on the ground curve equation and the real-time acquired current road image; wherein, the current road image and the reference road image are road images acquired from the same field of view; The second receiving module is used to receive the ground curve equation sent by the control terminal; the control terminal is a cloud server composed of control terminals. The third determining module is used to determine the area image within the range defined by the ground curve equation in the current road image acquired in real time by the image acquisition device; wherein, the current road image and the reference road image are road images acquired from the same field of view; the current road state is represented by different colors of the traffic indication device; Determining the current road status based on the region image includes: Receive the reference indication status and the location of the traffic indication device sent by the control terminal; Determine the current indication status of the traffic indication device in the area image based on the location of the traffic indication device; The first color component, the second color component, and the third color component of the reference indication state are respectively matched one-to-one with the same color component in the current indication state. Differential operations are performed to obtain three calculation results, and the three calculation results are summed to obtain the color calculation result. If the color calculation result is less than or equal to the color change threshold, then the color components of the current indication state are determined to be the first color component, the second color component, and the third color component of the reference indication state; The step of determining the area image within the range defined by the ground curve equation in the current road image acquired in real time by the image acquisition device, based on the ground curve equation, includes: For each pixel in the real-time acquired current road image, if the pixel is within the closed area enclosed by the ground curve equation, the pixel is retained; if the pixel is not within the closed area, the pixel is deleted. The fourth determining module is used to determine the current road status based on the area image.

6. An electronic device, characterized in that, include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and configured to be executed by the processor to implement the road state determination method according to any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, It stores a computer program; the computer program is executed by a processor to implement the road state determination method according to any one of claims 1-4.

Images