A smart lighting control method and control system based on traffic vision
By deploying crossed radar and cameras in the traffic lighting system, combining vehicle identification and Kalman gain algorithms to build a path prediction model, the problem of vehicle arrival time in the existing traffic lighting system is solved, and more accurate Light up in advance and reduce costs.
Patent Information
- Application Number
- CN202410895422.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-05
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-07-05
AI Technical Summary
In the existing traffic lighting system, the "lights follow the vehicle" control method does not match the time when the light strip is pre-lit and the arrival time of the vehicle, which makes it impossible to meet the actual driving needs, especially the detection of low-speed or stationary vehicles is limited, and The early lighting distance of high-speed vehicles is insufficient and the cost is high.
Using intelligent lighting control method based on traffic vision, by deploying crossing radars and cameras on both sides of the road, combining vehicle recognition, tracking algorithms and Kalman gain algorithms, a path prediction model is constructed, vehicle path prediction is optimized, and the lighting is accurately calculated. Time and distance ensure the accuracy and adaptability of early lighting.
Improve the ability to accurately identify and light up various vehicles, reduce control costs, ensure the matching of the lighting time of the lamp and the driving process, provide sufficient lighting distance and accurate time, and solve the problem of the existing technology. Existing detection blind spots and error problems.
Smart Images

Figure CN118433967B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of traffic lighting, and in particular to an intelligent lighting control method and control system based on traffic vision. Background Art
[0002] In traffic driving, the lighting system deployed on both sides of the road is of great significance in ensuring safe driving and avoiding traffic accidents. The traditional lighting system control method is to turn on and off the lamps at fixed time points, but this type of control method has obvious defects in energy saving and low carbon. Therefore, the "lights follow the car" control method has gradually occupied a dominant position.
[0003] In current traffic lighting, the implementation process of the "light follows vehicle" control method is: use sensors to detect vehicles on the road, and when a vehicle enters the sensor's detection area, the lights in the corresponding area are controlled to light up in advance based on the Internet of Things system and wireless networking system (i.e., forming a pre-lit light strip). The sensors of this type are generally millimeter-wave radars or cameras.
[0004] However, technicians have found that the existing "lights follow the vehicle" control method often has a technical defect that the lighting time of the pre-lit light strip cannot be adapted to the time when the vehicle reaches the pre-lit light strip. The corresponding test analysis found that the reasons for this defect are as follows:
[0005] For millimeter-wave radar, it cannot effectively detect vehicles at low speed or in a stationary state, so invalid detection often occurs; and in order to reduce costs, the effective detection distance of the millimeter-wave radar used is generally only tens of meters, that is, the distance of pre-lighting is limited to "tens of meters". At this time, it cannot provide a sufficiently far pre-lighting distance for high-speed vehicles that can be accurately detected, and introduces interference caused by the change of light and dark of the lamps within the driver's line of sight. For the camera, although it can make up for the shortcomings of millimeter-wave radar in terms of vehicle speed, on the one hand, most of them use monocular cameras. When using it for distance measurement, there is still a problem that the farther the distance between the target vehicle and the camera is, the greater the error in distance estimation; on the other hand, the detection distance is also extremely limited, so it is generally necessary to install a camera on each lamp. However, this will increase the control cost, which is contrary to the original intention of adopting the "light follows the vehicle" control method. Summary of the invention
[0006] The purpose of the present invention is to provide an intelligent lighting control method and control system based on traffic vision, so as to improve the technical defect that the actual control process of the current "light follows the vehicle" traffic lighting control method cannot be consistent with the actual driving process.
[0007] To achieve the above object, the present invention proposes the following technical solutions:
[0008] In the first aspect, the technical solution provides a smart lighting control method based on traffic vision, including:
[0009] Deploy a number of radars and cameras on both sides of the target road; wherein the detection areas of some radars and some cameras overlap, the total detection area covered by all radars and all cameras together is smaller than the area of the target road, and both ends of the target road are within the detection area of the radar or camera;
[0010] In the detection intersection area of the radar and the camera, the test vehicle is identified and tracked based on the vehicle identification algorithm and the target tracking algorithm to obtain several actual values of several driving parameters in time sequence; wherein the driving parameters include: center of gravity coordinates, turning angle and driving speed;
[0011] The path of the test vehicle is predicted using the actual value as the input of the vehicle path prediction algorithm, and in the prediction process, the time error between the test vehicle reaching the same position in the predicted path and the actual path is synchronously optimized based on the Kalman gain algorithm to construct a path prediction model;
[0012] When the actual vehicle enters the detection area of any camera deployed on both sides of the target road, the transfer time from the maximum grayscale value at the previous moment to the maximum grayscale value at the current moment is obtained based on the camera, so as to obtain the total error time corresponding to the lighting response process of the lamp; and then in the current path prediction corresponding to the actual vehicle, the path prediction model is optimized based on the total error time;
[0013] When an actual vehicle enters any intersection area of radars and cameras deployed on both sides of the target road, several current values corresponding to several driving parameters are obtained and input into a targeted optimized path prediction model to obtain several predicted values within the detection blind spot between any intersection area and the next intersection area, and then several lamps at corresponding positions are lit in advance based on the current values and the predicted values.
[0014] Furthermore, the path prediction of the test vehicle is performed using the actual value as the input of the vehicle path prediction algorithm, and in the prediction process, the time error between the test vehicle reaching the same position in the predicted path and the actual path is synchronously optimized based on the Kalman gain algorithm to construct a path prediction model; including:
[0015] Get in t At time t, the vehicle state estimation transfer equation of the test vehicle is z t = H t s t +R+G ;in, H t is the observation matrix, s t is the estimated current state, R is the observation noise covariance matrix, G is a position error correction matrix; wherein the position error correction matrix is used for light control position feedback;
[0016] in, s t = F t s t-1 + B t u t ; F t is the state transfer matrix, t -1 is the previous moment, s t-1 is the estimated state at the previous moment, u t is the velocity motion model, B t To test the internal control quantity of the vehicle on the speed motion model; u t =( v t , ω t ) T , v t is the translation speed at the current moment, ω t is the rotation speed at the current moment, T represents the transposition of the matrix;
[0017] G=s t ·M ;in, M is the position feedback matrix, which is used to describe the position difference between the estimated lighting position and the actual lighting position of the lamp;
[0018] At the same time, get t At time , the center of gravity coordinates of the test vehicle are: ;in, x t For testing vehicles t The X-axis coordinate value at the moment, y t For testing vehicles t The Y-axis coordinate value at the moment, θ tFor testing vehicles t The rotation angle at the moment, Δ t is the time variable, v t-1 for t -1 moment translation speed, ω t-1 for t -1 moment rotation speed, For testing vehicles t -1 approximate motion trajectory at time;
[0019] for t The center coordinates of the vehicle at time -1; where, x c To test the X-axis coordinate value of the vehicle center coordinate, y c To test the Y-axis coordinate value of the vehicle center coordinate, x t-1 For testing vehicles t- The X-axis coordinate value at time 1, y t-1 For testing vehicles t- The Y-axis coordinate value at time 1, θ t-1 for t -1 The turning angle at the moment;
[0020] The corrected speed Introducing the vehicle state estimation transfer equation and the vehicle center of gravity coordinates to obtain the vehicle prediction model state prediction equation is: ;in, For testing vehicles t- Correction value of translation speed at time 1, For testing vehicles t- Correction value of rotation speed at time 1, , are all random errors with a mean of 0. is a random error symbol, α 1. α 2. α 3. α 4 are all error parameters;
[0021] Based on the Jacobian matrix H t and the position feedback matrix M The sum of and the state transfer matrix F t Taking the derivatives, we get J Ht+M 、JFt ,in, J Ht+M is the observation matrix H t and position feedback matrix M The first-order partial derivative of the sum, J Ft is the state transfer matrix F t The first-order partial derivative of ; and then calculate the prediction error covariance matrix as , and the optimal Kalman gain is ;in, for t The prediction error covariance matrix at the moment is an uncertain value; for t The prediction error covariance matrix at time -1 is a fixed value; for J Ft The transposed matrix of Q t is the process noise covariance matrix, for J Ht+M The transposed matrix of R is the observation noise covariance matrix;
[0022] Update the vehicle prediction model state prediction equation by optimal Kalman gain and the prediction covariance equation The path prediction model is obtained; wherein, The predicted state of the test vehicle before the update is an uncertain value; z t Estimating the transfer equation for the vehicle state of the test vehicle, for t Time observation matrix H t The observed value of I is a unit matrix.
[0023] Furthermore, in the process of constructing the path prediction model, it also includes:
[0024] Calculate and obtain the observation matrix based on the camera observation results H t , and calculate the position feedback matrix based on the radar observation results M ;
[0025] The position feedback matrix is evaluated based on whether the light and shadow effect is optimal. M make corrections;
[0026] The principle for determining whether the lighting effect is optimal is as follows: whether the lighting distance is appropriate is determined by whether the light intensity at the preset distance from the test vehicle reaches the preset intensity; if it is less than the preset intensity, the current lighting distance is too far, and the position feedback matrix is determined. M If the intensity is greater than the preset value, the current lighting distance is insufficient, and the position feedback matrix is judged. M Lack of feedback.
[0027] Furthermore, the method of identifying and tracking the test vehicle in the detection intersection area of the radar and the camera based on the vehicle recognition algorithm and the target tracking algorithm to obtain several actual values of several driving parameters in time sequence includes:
[0028] Control all lamps to light up to obtain a number of images corresponding to the target road based on the camera, and obtain all pixel data in each image based on the image processing algorithm;
[0029] The grayscale values of all pixel data in each image are calculated based on the image grayscale algorithm, and the position coordinates of each lamp are estimated according to the position of each maximum grayscale value, and the position coordinates of each lamp are used as reference coordinates when tracking the vehicle target.
[0030] Furthermore, the method of lighting up several lamps at corresponding positions in advance based on the current value and the predicted value includes:
[0031] Calculate the current time t The distance between the predicted or actual position of the actual vehicle and the position of the pre-lit lamp is: ;in, S ( x , y ) is the current time t The coordinates of the predicted or actual position of the actual vehicle, P ( x , y ) is the position coordinate of the pre-lit lamp, L is the actual vehicle path;
[0032] According to the actual vehicle translation speed v t and acceleration a The actual time taken for the vehicle to reach the pre-lit lamp is calculated to be ;
[0033] exist T After the time period, several lamps that are expected to be lit are controlled to light up.
[0034] Furthermore, in the current path prediction corresponding to the actual vehicle, the path prediction model is optimized based on the total error time; including:
[0035] The total error time is set to include a first error time and a second error time; wherein the total error time is Calculated; T The predicted time for the actual vehicle to reach the pre-lit lamp. t’ is the actual time when the actual vehicle reaches the center of the lit street light segment identified by the camera; the first error time is caused by the path prediction model, and the second error time is caused by the mechanical and communication response in the control process;
[0036] The total error time is fed back to the path prediction model to adjust and optimize the position feedback matrix.
[0037] In the second aspect, the technical solution provides a smart lighting control system based on traffic vision, including:
[0038] A sensor deployment module is used to deploy a number of radars and a number of cameras on both sides of a target road; wherein the detection areas of some radars and some cameras overlap, the total detection area covered by all radars and all cameras together is smaller than the area of the target road, and both ends of the target road are within the detection area of the radar or camera;
[0039] The actual path acquisition module is used to identify and track the test vehicle in the detection intersection area of the radar and the camera based on the vehicle recognition algorithm and the target tracking algorithm to obtain several actual values of several driving parameters in time sequence; wherein the driving parameters include: center of gravity coordinates, turning angle and driving speed;
[0040] A prediction model building module, used to predict the path of the test vehicle using actual values as inputs of a vehicle path prediction algorithm, and to synchronously optimize the time error between the test vehicle reaching the same position in the predicted path and the actual path based on a Kalman gain algorithm during the prediction process to build a path prediction model;
[0041] A targeted optimization module is used to obtain the transfer time from the maximum grayscale value at the previous moment to the maximum grayscale value at the current moment based on the camera when the actual vehicle enters the detection area of any camera deployed on both sides of the target road, so as to obtain the total error time corresponding to the lighting response process of the lamp; and then in the current path prediction corresponding to the actual vehicle, the path prediction model is optimized based on the total error time;
[0042] The lighting control module is used to obtain a number of current values corresponding to a number of driving parameters when the actual vehicle enters any intersection area of the radars and cameras deployed on both sides of the target road, and input them into the targeted optimized path prediction model to obtain a number of predicted values within the detection blind spot between any intersection area and the next intersection area, and then light up a number of lamps at the corresponding positions in advance based on the current values and the predicted values.
[0043] Furthermore, the actual path acquisition module includes:
[0044] A pixel acquisition unit, used to control all lamps to light up so as to acquire a number of images corresponding to the target road based on the camera, and to acquire all pixel data in each image based on the image processing algorithm;
[0045] The grayscale calculation unit is used to calculate the grayscale values of all pixel data in each image based on the image grayscale algorithm, and estimate the position coordinates of each lamp according to the position with the maximum grayscale value, and use the position coordinates of each lamp as the reference coordinates when tracking the vehicle target.
[0046] In a third aspect, the technical solution provides an electronic device, comprising at least one processor, wherein the processor is coupled to a memory, wherein a computer program is stored in the memory, and wherein the computer program is configured to execute the method described when executed by the processor.
[0047] In a fourth aspect, the present technical solution provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is used to execute the described method.
[0048] Beneficial Effects
[0049] It can be seen from the above technical solutions that the present invention provides an intelligent lighting control method based on traffic vision to improve the technical problem that the advance lighting distance and time of the lamps cannot meet the actual driving needs in the current "lights follow the vehicle" control process.
[0050] The technicians found that the commonly used "lights follow the car" control method currently considers hardware factors in the design process, so it cannot meet the actual needs in the specific detection and subsequent software analysis and processing process. Therefore, this technical solution considers starting from both hardware and software aspects to effectively improve the problem that the control process of the lamps cannot be fully adapted to the actual driving.
[0051] In terms of hardware deployment, considering that millimeter-wave radar can accurately identify high-speed vehicles, while cameras can accurately identify low-speed or stationary vehicles, millimeter-wave radar and cameras are deployed at the same time to use them to perform fusion identification of vehicles; thereby improving the recognition accuracy of passing vehicles. Further, in the specific deployment of millimeter-wave radar and camera, considering the low cost factor, millimeter-wave radar and camera are not deployed in full coverage, but only a number of millimeter-wave radars and cameras are deployed and their detection areas are crossed so that various vehicles passing through the intersection area can be accurately identified. In order to make up for the detection blind spots and the problem that only millimeter-wave radar or camera detection coverage areas cannot perform vehicle identification, in terms of software deployment, a vehicle path prediction algorithm is introduced to construct a path prediction model; at this time, the light strip can be pre-lit in these areas based on the predicted data of the path prediction model. Specifically, the Kalman filter algorithm is introduced in the construction of the path prediction model to optimize the time error between the actual driving data and the predicted driving data, thereby improving the accuracy of the model results in the subsequent prediction process. At the same time, the accuracy in a specific prediction is also taken into account. Based on the movement time between the maximum grayscale value at the current moment and the maximum grayscale value at the previous moment obtained by the camera, the total error time generated by the lighting response of the lamp is obtained, so as to further optimize the path prediction model in the current path prediction process corresponding to the actual vehicle. At this time, under the deployment of the hardware and software, sufficient advance lighting distance and accurate advance lighting time can be provided while reducing the control cost.
[0052] It should be appreciated that all combinations of the foregoing concepts, as well as additional concepts described in greater detail below, may be considered to be part of the inventive subject matter of the present disclosure, provided such concepts are not mutually inconsistent.
[0053] The foregoing and other aspects, embodiments and features of the present invention can be more fully understood from the following description in conjunction with the accompanying drawings. Other additional aspects of the present invention, such as the features and / or beneficial effects of the exemplary embodiments, will be apparent from the following description or learned from the practice of the specific embodiments according to the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] The drawings are not intended to be drawn to scale. In the drawings, each identical or nearly identical component shown in various figures may be represented by the same reference numeral. For clarity, not every component is labeled in every figure. Embodiments of various aspects of the present invention will now be described by way of example and with reference to the accompanying drawings, in which:
[0055] Figure 1 This is a flow chart of the intelligent lighting control method based on traffic vision described in this embodiment;
[0056] Figure 2 is a schematic diagram of the detection range of this embodiment;
[0057] Figure 3 This is a flow chart for acquiring the position of a target road lamp in this embodiment;
[0058] Figure 4 A flow chart for obtaining a path prediction model for this embodiment;
[0059] Figure 5 A flowchart for performing targeted optimization of the path prediction model for this embodiment;
[0060] Figure 6 This is a flow chart of controlling the early lighting of lamps in this embodiment;
[0061] Figure 7 This is a block diagram of the traffic vision-based intelligent lighting control system described in this embodiment;
[0062] Figure 8 Schematic diagram of the structure of the electronic device described in this embodiment. DETAILED DESCRIPTION
[0063] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein should be the common meaning understood by people with general skills in the field to which the present invention belongs.
[0064] The words "first", "second" and similar words used in the specification and claims of this application do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, unless the context clearly indicates otherwise, the singular form of "a", "an" or "the" and other similar words do not indicate a quantitative limitation, but rather indicate the presence of at least one. Words such as "include" or "comprise" and the like mean that the elements or objects appearing before "include" or "comprise" cover the features, wholes, steps, operations, elements and / or components listed after "include" or "comprise", and do not exclude the existence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.
[0065] In current traffic lighting, the "light follows the vehicle" control method based on millimeter-wave radar detection often has the technical defect that the actual control process cannot be adapted to the actual driving process. Specifically, the main defect of millimeter-wave radar is that it has limitations on the detection of slow-moving or stopped vehicles, and may not be detected or be inaccurate. The main defect of the camera is that the monitoring distance is limited, and the data detected is not accurate enough when the vehicle is driving fast. Therefore, this embodiment aims to provide a smart lighting control method based on traffic vision to improve the above technical defects without introducing new technical defects.
[0066] The following is a detailed introduction to the traffic vision-based intelligent lighting control method described in this embodiment in conjunction with the accompanying drawings.
[0067] like Figure 1 As shown, the method includes:
[0068] Step S102: deploy a number of radars and cameras on both sides of the target road.
[0069] In this embodiment, considering the complementary advantages and disadvantages of radars and cameras in the detection process and the purpose of low-cost detection control, the detection areas of some radars and some cameras are set to intersect, and the total detection area covered by all radars and all cameras together is smaller than the area of the target road, and both ends of the target road are within the detection area of the radar or the camera. Figure 2 As shown, the horizontal axis P (m) is the position of the detectable vehicle, and the vertical axis V (m / s) is the speed of the detectable vehicle; a is the detection area of the millimeter wave radar, b is the detection area of the camera, and c is the detection area after the millimeter wave radar and the camera are integrated. It can be seen that this embodiment effectively expands the detection range, so that various driving parameters of vehicles passing through the detection area can be effectively identified. Specifically, a millimeter wave radar is deployed every two to three street lights; a camera is installed every one hundred to two hundred meters. The camera is specifically a fisheye camera.
[0070] Furthermore, the present embodiment also provides a corresponding hardware controller deployment scheme to control the lighting and extinguishing of lamps. Among them, the hardware controller includes a first single lamp controller, a second single lamp controller and a visual controller. Specifically, the lamps deployed with millimeter wave radars are correspondingly equipped with the first single lamp controller, the lamps not deployed with millimeter wave radars are correspondingly equipped with the second single lamp controller, and the lamps deployed with cameras are correspondingly equipped with the visual controller. The visual controller stores the path prediction model and a variety of data analysis algorithms, and is the core control unit of the intelligent lighting control method based on traffic vision. In the specific control process, if the millimeter wave radar is deployed on the lamp, the first single lamp controller is used to support the transmission of vehicle data identified by the millimeter wave radar to the visual controller, and the visual controller comprehensively analyzes the predicted vehicle driving time to control the on and off adjustment of the street lamp. If the millimeter wave radar sensor is not deployed on the lamp, the second single lamp controller only needs to control the on and off adjustment of the street lamp according to the comprehensive analysis of the predicted vehicle driving time by the visual controller. At the same time, both single lamp controllers include wireless communication modules for transmitting signals between lamps.
[0071] In the specific implementation, for the camera, since it is necessary to deploy multiple cameras for cross-coverage detection in order to perform multi-camera detection of vehicles on the road; therefore, it is necessary to pre-set the deployment position of each camera so that it can be calibrated on the map, and enable the camera to have multiple angle combinations; and then stitch the acquired images. Specifically, the visual controller under each camera is responsible for controlling the single-light controllers (including the first single-light controller and the second single-light controller) on the adjacent 11-stack street lights to perform two operations: brightening and dimming the street lights. With the visual controller as the center, there are 5 lights to the left and 5 lights to the right, covering a distance of approximately 500 meters; the number of lights managed with the visual controller as the center varies with the road conditions (at this point, it can be considered that a light and shadow effect control template has been defined).
[0072] Step S104: in the detection intersection area of the radar and the camera, the test vehicle is identified and tracked based on the vehicle identification algorithm and the target tracking algorithm in turn to obtain a number of actual values of a number of driving parameters in a time series.
[0073] In this embodiment, the test vehicle may be one or more test vehicles. The driving parameters include: center of gravity coordinates, turning angles and driving speed.
[0074] As a specific implementation method, the vehicle position coordinates are set with reference to the lamp installation positions on both sides of the target road. Figure 3 As shown, the following steps are included to obtain the lamp installation location:
[0075] Step S103.2: Control all lamps to light up to obtain a number of images corresponding to the target road based on the camera, and obtain all pixel data in each image based on the image processing algorithm.
[0076] Step S103.4: Calculate the grayscale values of all pixel data in each image based on the image grayscale algorithm, estimate the position coordinates of each lamp according to the position with the maximum grayscale value, and use the position coordinates of each lamp as the reference coordinates for vehicle target tracking.
[0077] In the specific implementation, any pixel data is recorded as ( R,G,B,A ),in, R , G , B They are red, green and blue pixels respectively. A is transparency. The grayscale value is Grap =(max( R , G , B )+min( R , G , B )) / 2. At this time, the obtained lamp coordinate data set is P ={ P 0, P 1, P 2… P N}; where N is the Nth lamp. Since steps S103.2 to S103.4 actually calculate the lamp coordinates, they are more accurate than directly using the engineering data when the lamp is installed; and reduce the error caused by the position error of the reference lamp on the subsequent light strip control.
[0078] Step S106, using the actual value as the input of the vehicle path prediction algorithm to predict the path of the test vehicle, and in the prediction process, synchronously optimizing the time error between the predicted path and the same position in the actual path based on the Kalman gain algorithm to construct a path prediction model.
[0079] like Figure 4 As shown, in the specific implementation, step S106 includes:
[0080] Step S106.2: Get t At time t, the vehicle state estimation transfer equation of the test vehicle is z t = H t s t + R+G .
[0081] in, Ht is the observation matrix, s t is the estimated current state, R is the observation noise covariance matrix, G is a position error correction matrix; wherein the position error correction matrix is used for light control position feedback.
[0082] in, s t = F t s t-1 + B t u t ; F t is the state transfer matrix, t -1 is recorded as the previous moment, s t-1 is the estimated state at the previous moment, u t is the velocity motion model, B t To test the internal control quantity of the vehicle on the speed motion model; u t =( v t , ω t ) T , v t is the translation speed at the current moment, ω t is the rotation speed at the current moment, and T represents the transposition of the matrix.
[0083] G=s t ·M ;in, M is a position feedback matrix used to describe the position difference between the estimated lighting position and the actual lighting position of the lamp.
[0084] Get in t At time , the center of gravity coordinates of the test vehicle are: ;in, x t For testing vehicles t The X-axis coordinate value at the moment, y t For testing vehicles t The Y-axis coordinate value at the moment, θ t For testing vehicles t The rotation angle at the moment, Δt is the time variable, v t-1 for t -1 moment translation speed, ω t-1 for t -1 moment rotation speed, For testing vehicles t Approximate motion trajectory at time -1.
[0085] for t The center coordinates of the vehicle at time -1; where, x c To test the X-axis coordinate value of the vehicle center coordinate, y c To test the Y-axis coordinate value of the vehicle center coordinate, x t-1 For testing vehicles t- The X-axis coordinate value at time 1, y t-1 For testing vehicles t- The Y-axis coordinate value at time 1, θ t-1 for t The rotation angle at the moment -1.
[0086] Step S106.4: The corrected speed Introducing the vehicle state estimation transfer equation and the vehicle center of gravity coordinates to obtain the vehicle prediction model state prediction equation is: .
[0087] in, For testing vehicles t- Correction value of translation speed at time 1, For testing vehicles t- Correction value of rotation speed at time 1, , are all random errors with a mean of 0. is a random error symbol, α 1. α 2. α 3. α 4 are error parameters.
[0088] Step S106.6: Observation matrix based on Jacobian matrix H t and the position feedback matrix M The sum of and the state transfer matrix F t Taking the derivatives, we get J Ht+M、J Ft ,in, J Ht+M is the observation matrix H t and position feedback matrix M The first-order partial derivative of the sum, J Ft is the state transfer matrix F t The first-order partial derivative of ; and then calculate the prediction error covariance matrix , and the optimal Kalman gain .
[0089] The purpose of the derivation is to test the nonlinear motion path requirements of the vehicle. for t The prediction error covariance matrix at the moment is an uncertain value; for t The prediction error covariance matrix at time -1 is a fixed value; for J Ft The transposed matrix of Q t is the process noise covariance matrix, for J Ht+M The transposed matrix of R is the observation noise covariance matrix.
[0090] Step S106.8: Update the vehicle prediction model state prediction equation using the optimal Kalman gain and the prediction covariance equation The path prediction model is obtained.
[0091] in, is the predicted state of the test vehicle before updating, which is an uncertain value; z t Estimate the transfer equation for the vehicle state of the test vehicle, for t Time observation matrix H t The observed value of I is a unit matrix.
[0092] In order to facilitate the corresponding control of hardware signals, in step S106.8, a control matrix of the lamps will eventually be obtained. The control matrix is equal to the current estimated state quantity multiplied by a control signal delay vector, and then combined with the above light and shadow effect control template, the timetable for sending control signals to each lamp is finally obtained. T}={ T 0, T 1,T 2... T N}.
[0093] In the specific implementation, the update process of the path prediction model is also accompanied by the position feedback matrix M Specifically, first observe through the camera and calculate the observation matrix H t The value of , and then calculate the position feedback matrix according to the actual millimeter wave radar observation effect M And continuously accumulate learning with the best light and shadow effect to perform position feedback matrix M Optimization. Specifically, the position feedback matrix M The principle of determining whether the optimal lighting distance is reached during the optimization process is as follows: whether the light intensity at the preset distance from the test vehicle reaches the preset intensity is used as the standard to determine whether the lighting distance is appropriate; if it is less than the preset intensity, the current lighting distance is too far, and the position feedback matrix is determined. M If the intensity is greater than the preset value, the current lighting distance is insufficient, and the position feedback matrix is judged. M In the specific implementation, the preset distance is 250 meters; thus, the position feedback matrix is corrected during the continuous iteration of the path prediction model. M The value of .
[0094] In this embodiment, since the error function of the state prediction equation of the vehicle prediction model is updated by using the optimal Kalman gain, the obtained path prediction model has a smaller error.
[0095] Step S108, when the actual vehicle enters any detection area of the cameras deployed on both sides of the target road, the movement time between the maximum grayscale value at the current moment and the maximum grayscale value at the previous moment is obtained based on the camera to obtain the total error time corresponding to the lamp lighting response, and based on the total error time, the path prediction model is optimized in a targeted manner during the current path prediction process corresponding to the actual vehicle.
[0096] Specific as Figure 5 As shown, targeted optimization is performed in the following ways:
[0097] Step S108.2: Set the total error time to include a first error time and a second error time.
[0098] The total error time is given by Calculated; T The predicted time for the actual vehicle to reach the pre-lit lamp. t’The time taken by the camera to identify the vehicle's arrival at the center of the illuminated street light segment is the first error time caused by the path prediction model, and the second error time caused by mechanical and communication responses during the control process.
[0099] Step S108.4: Feedback the total error time to the path prediction model to adjust and optimize the position feedback matrix.
[0100] Steps S108.2 to S108.4 consider the possible errors in the path prediction model in actual prediction during an actual driving, as well as the impact of errors caused by the recognition response to different driving speeds on the path prediction. The path prediction model is optimized in a targeted manner to reduce the prediction error and improve the prediction accuracy.
[0101] Step S110: When an actual vehicle enters any intersection area of radars and cameras deployed on both sides of the target road, a number of current values corresponding to a number of driving parameters are obtained, and the current values are input into a targeted optimized path prediction model to obtain a number of predicted values corresponding to the detection blind spot between the intersection area and the next intersection area, and then a number of lamps at corresponding positions are lit in advance based on the current values and the predicted values.
[0102] Specific reference Figure 6 As shown, the lamp is lit in advance through the following steps:
[0103] Step S110.2: Calculate the current time t The distance between the predicted or actual position of the actual vehicle and the position of the pre-lit lamp is: .
[0104] in, S ( x , y ) is the current time t The coordinates of the predicted or actual position of the actual vehicle, P ( x , y ) is the position coordinate of the pre-lit lamp, L is the actual vehicle path.
[0105] Step S110.4: according to the actual vehicle translation speed v t and acceleration a The actual time it takes for the vehicle target to reach the lamp can be calculated as .
[0106] Step S110.6: T After the time period, several lamps that are expected to be lit are controlled to light up.
[0107] From the above, it can be seen that this embodiment provides a new method for controlling traffic lamps. Through the combination of multi-sensor fusion detection and path prediction algorithm (that is, using sensors with complementary advantages to detect at the same time), a relatively accurate vehicle trajectory prediction model is obtained within the range that both sensors can accurately detect. Then, when the two sensors cannot detect or are not accurate enough, the prediction model is used for prediction, making the prediction model more applicable. Specifically, the Kalman gain algorithm is introduced to optimize the error of the path prediction model. In a specific prediction, the time error caused by the lamp response is also eliminated. Furthermore, while achieving low-cost lighting control, it also effectively ensures that the lamp is lit in advance within the required distance, and reduces the time error when lighting up, realizing the adaptation of the current "light follows the vehicle" lighting control to actual driving.
[0108] The above program can be run in the processor, or it can be stored in the memory (or computer-readable storage medium), which includes permanent and non-permanent, removable and non-removable media. Information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined in this article, computer-readable media does not include temporary computer-readable media, such as modulated data signals and carrier waves.
[0109] These computer programs can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps of the functions specified in one or more blocks can be implemented by different modules corresponding to different steps.
[0110] Based on this, this embodiment also provides a smart lighting control system based on traffic vision. Figure 7 As shown, the control system includes:
[0111] A sensor deployment module is used to deploy a number of radars and a number of cameras on both sides of a target road; wherein the detection areas of some radars and some cameras overlap, the total detection area covered by all radars and all cameras together is smaller than the area of the target road, and both ends of the target road are within the detection area of the radar or camera.
[0112] The actual path acquisition module is used to identify and track the test vehicle in the detection intersection area of the radar and the camera based on the vehicle recognition algorithm and the target tracking algorithm to obtain several actual values of several driving parameters in time series; among which, the driving parameters include: center of gravity coordinates, turning angle and driving speed.
[0113] The prediction model building module is used to predict the path of the test vehicle using actual values as input to the vehicle path prediction algorithm, and during the prediction process, based on the Kalman gain algorithm, the time error between the test vehicle reaching the same position in the predicted path and the actual path is synchronously optimized to build a path prediction model.
[0114] A targeted optimization module is used to obtain the transfer time from the maximum grayscale value at the previous moment to the maximum grayscale value at the current moment based on the camera when the actual vehicle enters the detection area of any camera deployed on both sides of the target road, so as to obtain the total error time corresponding to the lamp lighting response process; and then in the current path prediction corresponding to the actual vehicle, the path prediction model is targetedly optimized based on the total error time.
[0115] The lighting control module is used to obtain a number of current values corresponding to a number of driving parameters when the actual vehicle enters any intersection area of the radars and cameras deployed on both sides of the target road, and input them into the targeted optimized path prediction model to obtain a number of predicted values within the detection blind spot between any intersection area and the next intersection area, and then light up a number of lamps at the corresponding positions in advance based on the current values and the predicted values.
[0116] The system is used to implement the steps of the above method, so what has been explained will not be repeated here.
[0117] For example, the actual path acquisition module previously includes:
[0118] The pixel acquisition unit is used to control all lamps to light up so as to acquire a number of images corresponding to the target road based on the camera, and to acquire all pixel data in each image based on the image processing algorithm.
[0119] The grayscale calculation unit is used to calculate the grayscale values of all pixel data in each image based on the image grayscale algorithm, and estimate the position coordinates of each lamp according to the position with the maximum grayscale value, and use the position coordinates of each lamp as the reference coordinates when tracking the vehicle target.
[0120] For another example, the lighting control module includes:
[0121] The error setting unit is used to set the total error time to include a first error time and a second error time; wherein the total error time is composed of Calculated; T The predicted time for the actual vehicle to reach the pre-lit lamp. t’ It is the actual time when the actual vehicle reaches the center of the lit street light segment identified by the camera; the first error time is caused by the path prediction model, and the second error time is caused by the mechanical and communication responses in the control process.
[0122] The model optimization unit is used to feed back the total error time to the path prediction model to adjust and optimize the position feedback matrix.
[0123] This embodiment also provides an electronic device. Figure 8 As shown, it includes at least one processor, the processor is coupled to a memory, a computer program is stored in the memory, and the computer program is configured to execute the above method when executed by the processor.
[0124] This embodiment also provides a computer-readable storage medium on which a computer program is stored. The computer program is used to execute the above method.
[0125] Although the present invention has been disclosed as above with preferred embodiments, it is not intended to limit the present invention. A person with ordinary knowledge in the technical field to which the present invention belongs may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be determined by the definition of the claims.
Claims
1. A smart lighting control method based on traffic vision, characterized in that: include: Deploy a number of radars and cameras on both sides of the target road; wherein the detection areas of some radars and some cameras overlap, the total detection area covered by all radars and all cameras together is smaller than the area of the target road, and both ends of the target road are within the detection area of the radar or camera; Control all lamps to light up to obtain a number of images corresponding to the target road based on the camera, and obtain all pixel data in each image based on the image processing algorithm; then, calculate the grayscale value of all pixel data in each image based on the image grayscale algorithm, and estimate the position coordinates of each lamp according to the position of each maximum grayscale value, and use the position coordinates of each lamp as the reference coordinates when tracking the vehicle target; In the detection intersection area of the radar and the camera, the test vehicle is identified and tracked based on the vehicle identification algorithm and the target tracking algorithm to obtain several actual values of several driving parameters in time sequence; wherein the driving parameters include: center of gravity coordinates, turning angle and driving speed; The path of the test vehicle is predicted using the actual value as the input of the vehicle path prediction algorithm, and in the prediction process, the time error between the test vehicle reaching the same position in the predicted path and the actual path is synchronously optimized based on the Kalman gain algorithm to construct a path prediction model; Among them, include: Get in t At time t, the vehicle state estimation transfer equation of the test vehicle is z t = H t s t + R+G ;in, H t is the observation matrix, s t is the estimated current state, R is the observation noise covariance matrix, G is a position error correction matrix; wherein the position error correction matrix is used for light control position feedback; in, s t = F t s t-1 + B t u t ; F t is the state transfer matrix, t -1 is recorded as the previous moment, s t-1 is the estimated state at the previous moment, u t is the velocity motion model, B t To test the internal control quantity of the vehicle on the speed motion model; u t =( v t , ω t ) T , v t is the translation speed at the current moment, ω t is the rotation speed at the current moment, T represents the transposition of the matrix; G=s t ·M ;in, M is the position feedback matrix, which is used to describe the position difference between the estimated lighting position and the actual lighting position of the lamp; At the same time, get t At time , the center of gravity coordinates of the test vehicle are: ;in, x t For testing vehicles t The X-axis coordinate value at the moment, y t For testing vehicles t The Y-axis coordinate value at the moment, θ t For testing vehicles t The rotation angle at the moment, Δ t is the time variable, v t-1 for t -1 moment translation speed, ω t-1 for t -1 moment rotation speed, For testing vehicles t -1 approximate motion trajectory at time; for t The center coordinates of the test vehicle at time -1; where, x c To test the X-axis coordinate value of the vehicle center coordinate, y c To test the Y-axis coordinate value of the vehicle center coordinate, x t-1 For testing vehicles t- The X-axis coordinate value at time 1, y t-1 For testing vehicles t- The Y-axis coordinate value at time 1, θ t-1 for t -1 The turning angle at the moment; The corrected speed Introducing the vehicle state estimation transfer equation and the vehicle center of gravity coordinates to obtain the vehicle prediction model state prediction equation is: ;in, For testing vehicles t- Correction value of translation speed at time 1, For testing vehicles t- Correction value of rotation speed at time 1, are all random errors with a mean of 0. is a random error symbol, symbol, α 1. α 2. α 3. α 4 are all error parameters; Based on the Jacobian matrix H t and the position feedback matrix M The sum of and the state transfer matrix F t Taking the derivatives, we get J Ht+M 、J Ft ,in, J Ht+M is the observation matrix H t and position feedback matrix M The first-order partial derivative of the sum, J Ft is the state transfer matrix F t The first-order partial derivative of ; and then calculate the prediction error covariance matrix as , and the optimal Kalman gain is ;in, for t The prediction error covariance matrix at the moment is an uncertain value; for t The prediction error covariance matrix at time -1 is a fixed value; for J Ft The transposed matrix of Q t is the process noise covariance matrix, for J Ht+M The transposed matrix of R is the observation noise covariance matrix; Update the vehicle prediction model state prediction equation by using the optimal Kalman gain and the prediction covariance equation The path prediction model is obtained; wherein, is the predicted state of the test vehicle before updating, which is an uncertain value; z t Estimate the transfer equation for the vehicle state of the test vehicle, for t Time observation matrix H t The observed value of I is an identity matrix; When the actual vehicle enters the detection area of any camera deployed on both sides of the target road, the transfer time from the maximum grayscale value at the previous moment to the maximum grayscale value at the current moment is obtained based on the camera, so as to obtain the total error time corresponding to the lighting response process of the lamp; and then in the current path prediction corresponding to the actual vehicle, the path prediction model is optimized based on the total error time; When an actual vehicle enters any intersection area of radars and cameras deployed on both sides of the target road, several current values corresponding to several driving parameters are obtained, and they are input into the targeted optimized path prediction model to obtain several predicted values in the detection blind spot between any intersection area and the next intersection area, and then several lamps at corresponding positions are lit in advance based on the current values and the predicted values; The method of lighting up several lamps at corresponding positions in advance based on the current value and the predicted value includes: Calculate the current time t The distance between the predicted or actual position of the actual vehicle and the position of the pre-lit lamp is: ;in, S ( x , y ) is the current time t The coordinates of the predicted or actual position of the actual vehicle, P ( x , y ) is the position coordinate of the pre-lit lamp, L is the actual vehicle path; According to the actual vehicle translation speed v t and acceleration a The actual time taken for the vehicle to reach the pre-lit lamp is calculated to be ; exist T After the time period, several lamps that are expected to be lit are controlled to light up.
2. The intelligent lighting control method based on traffic vision according to claim 1 is characterized in that: The process of constructing the path prediction model also includes: Calculate and obtain the observation matrix based on the camera observation results H t , and calculate the position feedback matrix based on the radar observation results M ; The position feedback matrix is evaluated based on whether the light and shadow effect is optimal. M make corrections; The principle for determining whether the lighting effect is optimal is as follows: whether the lighting distance is appropriate is determined by whether the light intensity at the preset distance from the test vehicle reaches the preset intensity; if it is less than the preset intensity, the current lighting distance is too far, and the position feedback matrix is determined. M If the intensity is greater than the preset value, the current lighting distance is insufficient, and the position feedback matrix is judged. M Lack of feedback.
3. The intelligent lighting control method based on traffic vision according to claim 1 is characterized in that: The method of optimizing the path prediction model based on the total error time in the current path prediction corresponding to the actual vehicle comprises: The total error time is set to include a first error time and a second error time; wherein the total error time is Calculated; T The predicted time for the actual vehicle to reach the pre-lit lamp. t’ is the actual time when the actual vehicle reaches the center of the lit street light segment identified by the camera; the first error time is caused by the path prediction model, and the second error time is caused by the mechanical and communication response in the control process; The total error time is fed back to the path prediction model to adjust and optimize the position feedback matrix.
4. A smart lighting control system based on traffic vision, characterized in that: The method for implementing any one of claims 1 to 3 comprises: A sensor deployment module is used to deploy a number of radars and a number of cameras on both sides of a target road; wherein the detection areas of some radars and some cameras overlap, the total detection area covered by all radars and all cameras together is smaller than the area of the target road, and both ends of the target road are within the detection area of the radar or camera; The actual path acquisition module is used to identify and track the test vehicle in the detection intersection area of the radar and the camera based on the vehicle recognition algorithm and the target tracking algorithm to obtain several actual values of several driving parameters in time sequence; wherein the driving parameters include: center of gravity coordinates, turning angle and driving speed; A prediction model building module, used to predict the path of the test vehicle using actual values as inputs of a vehicle path prediction algorithm, and to synchronously optimize the time error between the test vehicle reaching the same position in the predicted path and the actual path based on a Kalman gain algorithm during the prediction process to build a path prediction model; A targeted optimization module is used to obtain the transfer time from the maximum grayscale value at the previous moment to the maximum grayscale value at the current moment based on the camera when the actual vehicle enters the detection area of any camera deployed on both sides of the target road, so as to obtain the total error time corresponding to the lighting response process of the lamp; and then in the current path prediction corresponding to the actual vehicle, the path prediction model is optimized based on the total error time; The lighting control module is used to obtain a number of current values corresponding to a number of driving parameters when the actual vehicle enters any intersection area of the radars and cameras deployed on both sides of the target road, and input them into the targeted optimized path prediction model to obtain a number of predicted values within the detection blind spot between any intersection area and the next intersection area, and then light up a number of lamps at the corresponding positions in advance based on the current values and the predicted values.
5. An electronic device, characterized in that: The method comprises at least one processor, wherein the processor is coupled to a memory, wherein a computer program is stored in the memory, and wherein the computer program is configured to execute the method according to any one of claims 1 to 3 when executed by the processor.
6. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a computer, the method according to any one of claims 1 to 3 is implemented.
