Traffic signal light tracking method, device and readable storage medium

Abstract

The present application provides a method, device and readable storage medium for tracking traffic lights, which are used to solve the problem of low accuracy of electric police cameras in judging the status of traffic lights in the prior art. The method includes: determining the position parameters and first characteristic parameters of a first area where a traffic light is located in a first traffic image; wherein the first area includes the coordinates of the center point of the traffic light in the first traffic image; based on the first characteristic parameters, determining the first information of the first area in the first traffic image; based on the position parameters of the first area, determining the second area in the second traffic image, and the second information of the second area; using the first information as a template, determining the correlation between the template and the second information, and determining the location of the pixel point corresponding to the maximum value of the correlation as the center position of the traffic light in the second traffic image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device and readable storage medium for tracking traffic lights. Background Art

[0002] Traffic lights, as a signal sign, play an important role in traffic regulation. In order to build a smart city, an electric police camera is generally set up in the setting area opposite the traffic lights to capture images (frames) of vehicles running red lights and other irregular driving. After capturing the image, the electric police camera needs to first determine the state of the traffic light in the image to further determine whether the vehicle in the image is in an irregular driving state. However, the resolution of the image (frame) captured by the electric police camera greatly reduces the efficiency of judging the state of the traffic light and cannot meet the real-time requirements of the judgment of the state of the traffic light. Therefore, before judging the state of the traffic light, a smaller area of ​​interest needs to be set for the electric police camera to judge the state of the traffic light. However, since the electric police camera and the traffic light are exposed to the open air environment for a long time, they are easily affected by strong winds or other natural phenomena and offset. This causes the area of ​​interest selected by the electric police camera according to the setting rules to not include the traffic light, resulting in inaccurate or even impossible judgment of the state of the traffic light.

[0003] Therefore, there is a problem in the prior art that the accuracy of the electric police camera in judging the status of the traffic light is low. Summary of the invention

[0004] The present application provides a method, device and readable storage medium for tracking a traffic light, so as to solve the problem of low accuracy in judging the state of a traffic light in the prior art.

[0005] In a first aspect, the present application provides a method for tracking a traffic light, comprising:

[0006] Determine a position parameter and a first characteristic parameter of a first region in a first traffic image where a traffic light is located; wherein the first region includes the coordinates of a center point of the traffic light in the first traffic image, and the first characteristic parameter indicates a grayscale characteristic and a gradient characteristic of a pixel point in the first region;

[0007] Based on the first characteristic parameter, first information of a first area in the first traffic image is determined; wherein the first information indicates the grayscale characteristics and gradient characteristics of each pixel in the first area in the first traffic image, and the relationship characteristics between the grayscale of each pixel and the grayscales of other pixels;

[0008] Based on the position parameters of the first area, determine a second area in the second traffic image, and second information of the second area; wherein the second area includes the coordinates of the center point, and the second information indicates the grayscale characteristics and gradient characteristics of each pixel point in the second area, and the relationship characteristics between the grayscale of each pixel point and the grayscale of other pixels;

[0009] The first information is used as a template, the correlation between the template and the second information is determined, and the position of the pixel corresponding to the maximum correlation value is determined to be the center position of the traffic light in the second traffic image; wherein the correlation indicates the correlation between all pixel points in the first area of ​​the first traffic image and all pixel points in the second area of ​​the second traffic image.

[0010] The above method obtains the first information as a template by pre-extracting the position parameter and the first feature parameter in the first traffic image, and correlates the template with the second information of the second area in the second traffic image subsequently obtained, thereby determining the center coordinates of the second traffic light, thereby achieving the purpose of tracking the traffic light and timely discovering the deviation of the position of the traffic light; thereby avoiding the occurrence of misjudgment or inability to judge the status of the traffic light.

[0011] In a possible implementation manner, determining the correlation between the template and the second information includes:

[0012] Determining the correlation between the template and the second information based on a target tracking algorithm and a signal correlation principle;

[0013] The maximum value of the correlation is compared with a set threshold value, and when the maximum value of the correlation is greater than the set threshold value, it is determined that the traffic light tracking is successful.

[0014] In a possible implementation manner, determining the position parameter and the first characteristic parameter of the first area where the traffic light is located in the first traffic image includes:

[0015] Using the circumscribed rectangle of the traffic light as a selection frame to select the first area in the first traffic image;

[0016] Determine standard location information of the first area; wherein the standard location information includes the coordinates of the center point of the first area;

[0017] extracting a set number of the first traffic images, determining at least one first area in each of the first traffic images, and determining whether location information of the at least one first area is consistent with the standard location information;

[0018] If so, the location information of the first area in any one of the first traffic images is used as the location parameter of the first area; and the grayscale value and gradient value of all pixels in the first area in any one of the first traffic images are determined to obtain the first characteristic parameter of the first area.

[0019] The above method can ensure that the first area framed in the first traffic image is the first area at the target position by comparing the position information of the first area in the first traffic image with the standard position information, thereby improving the probability of successfully tracking the traffic light.

[0020] In a possible implementation manner, the first information M is:

[0021]

[0022] in, λ is a preset constant, σ is the preset standard deviation parameter, ☉ is the dot multiplication operation, F(·) is the Fourier transform, F -1 (·) is the inverse Fourier transform, X is the first characteristic parameter, X′ is the transpose of X, w is the length of the first region, h is the width of the first region, (i, j) is the pixel coordinates in the first region, (i c , j c ) are the coordinates of the central pixel point in the first area.

[0023] In a possible implementation manner, after determining that the position of the pixel point corresponding to the maximum correlation value is the center position of the traffic light in the second traffic image, the method includes:

[0024] Determine, in the second traffic image, a third area with a center position of a traffic light in the second traffic image as an area center by using the length and width of the first area in the first traffic image;

[0025] Determine third information of the third area; wherein the third information indicates a grayscale feature of each pixel in the third area, and a relationship feature between the grayscale of each pixel and the grayscales of other pixels;

[0026] The template and the third information are respectively multiplied by corresponding weights and summed to obtain fourth information, and the template is updated based on the fourth information for tracking the traffic light in the next frame of traffic image.

[0027] In a second aspect, the present application provides a tracking device for a traffic light, comprising:

[0028] A parameter unit: used to determine the position parameter and the first characteristic parameter of the first area where the traffic light is located in the first traffic image; wherein the first area includes the coordinates of the center point of the traffic light in the first traffic image, and the first characteristic parameter indicates the grayscale characteristics and gradient characteristics of the pixel points in the first area;

[0029] An information unit: used to determine first information of a first area in the first traffic image based on the first characteristic parameter; wherein the first information indicates the grayscale feature and gradient feature of each pixel in the first area in the first traffic image, and the relationship feature between the grayscale of each pixel and the grayscale of other pixels;

[0030] A determination unit: used to determine a second area in a second traffic image and second information of the second area based on the position parameters of the first area; wherein the second area includes the coordinates of the center point, and the second information indicates the grayscale characteristics and gradient characteristics of each pixel point in the second area, and the relationship characteristics between the grayscale of each pixel point and the grayscale of other pixels;

[0031] Central unit: used to use the first information as a template, determine the correlation between the template and the second information, and determine that the position of the pixel point corresponding to the maximum correlation value is the center position of the traffic light in the second traffic image; wherein the correlation indicates the correlation between all pixel points in the first area of ​​the first traffic image and all pixel points in the second area of ​​the second traffic image.

[0032] In a possible implementation manner, the central unit is specifically used to determine the correlation between the template and the second information based on the target tracking algorithm and the signal correlation principle; compare the maximum value in the correlation with a set threshold, and when the maximum value in the correlation is greater than the set threshold, it is determined that the traffic light tracking is successful.

[0033] In a possible implementation manner, the parameter unit is specifically used to use the circumscribed rectangle of the traffic light as a selection box to select the first area in the first traffic image; determine standard position information of the first area; wherein the standard position information includes the coordinates of the center point of the first area; extract a set number of the first traffic images, determine at least one first area in each of the first traffic images, and determine whether the position information of the at least one first area is consistent with the standard position information; if so, use the position information of the first area in any one of the first traffic images as the position parameter of the first area; and determine the grayscale value and gradient value of all pixels in the first area in any one of the first traffic images to obtain the first characteristic parameter of the first area.

[0034] In a possible implementation manner, the first information M is:

[0035]

[0036] in, λ is a preset constant, σ is the preset standard deviation parameter, ☉ is the dot multiplication operation, F(·) is the Fourier transform, F -1 (·) is the inverse Fourier transform, X is the first characteristic parameter, X′ is the transpose of X, w is the length of the first region, h is the width of the first region, (i, j) is the pixel coordinates in the first region, (i c , j c ) are the coordinates of the central pixel point in the first area.

[0037] In a possible implementation manner, the device also includes an updating unit, which is specifically used to use the length and width of the first area in the first traffic image to determine a third area in the second traffic image with the center position of the traffic light in the second traffic image as the area center; determine third information of the third area; wherein the third information indicates the grayscale characteristics and gradient characteristics of each pixel point in the third area, and the relationship characteristics of the grayscale of each pixel point with the grayscale of other pixels; respectively multiply the template and the third information with corresponding weights and sum them to obtain fourth information, and update the template based on the fourth information for tracking the traffic light in the next frame of the traffic image.

[0038] In a third aspect, the present application provides a readable storage medium, comprising:

[0039] Memory,

[0040] The memory is used to store instructions. When the instructions are executed by the processor, the device including the readable storage medium performs the method described in the first aspect and any possible implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A flow chart of a method for tracking a traffic light provided in an embodiment of the present application;

[0042] Figure 2 A schematic diagram of a first deep learning model used when selecting a first area from a first traffic image provided in an embodiment of the present application;

[0043] Figure 3 A schematic diagram of tracking traffic lights on a new traffic image using a method for tracking traffic lights provided in an embodiment of the present application;

[0044] Figure 4A schematic diagram of the structure of a traffic light tracking device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] In view of the problem of low accuracy of traffic light status judgment by electric police cameras in the prior art, the present application proposes a traffic light tracking method: based on a target tracking algorithm and the principle of signal correlation, the traffic lights are tracked by tracking the coordinates of the center of the traffic lights in the traffic image; thereby solving the problem that the electric police camera fails to detect the offset of the traffic light and still judges the status of the traffic light based on the original coordinates of the traffic light, resulting in misjudgment.

[0046] In order to better understand the above technical scheme, the technical scheme of the present application is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical scheme of the present application, rather than limitations on the technical scheme of the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0047] Please refer to Figure 1 The embodiment of the present application provides a method for tracking traffic lights, which solves the problem of low accuracy of electric police cameras in judging the status of traffic lights in the prior art. The method specifically includes the following implementation steps:

[0048] Step 101: Determine a position parameter and a first characteristic parameter of a first area where a traffic light is located in a first traffic image.

[0049] The first area includes the coordinates of the center point of the traffic light in the first traffic image, and the first characteristic parameter indicates the grayscale characteristics and gradient characteristics of the pixel points in the first area.

[0050] Specifically, in order to improve the image processing efficiency, after acquiring the first traffic image, the first area including the center point of the traffic light can be selected by the first deep learning model, and then the first area can be processed. The first deep learning model can be a single-stage target detection model, which includes a backbone network, an enhancement network, and a detection network. Figure 2As shown, in the backbone network, a CSP (Cross Stage Partial Network) residual structure can be set to extract high-dimensional features of the first traffic image. Five stages can be set in the CSP structure, each of which samples the first traffic image, and the resolution is reduced by one time compared with the previous stage, and the number of channels is doubled; until the fifth stage, the resolution of the first traffic image is reduced to 13 times, and the channels are expanded to 512 times. In the enhanced network, a PAN (Path Aggregation Network) structure can be set. In the PAN structure, two top-down and bottom-up paths are set for feature extraction and feature aggregation. The abstract high-level features and the specific low-level features are deeply integrated through the first function, the second function, and the third function, which can further improve the accuracy of the image features expressed by the backbone network. Furthermore, the first traffic image features obtained in the enhancement network can be input into the detection network, which includes a first data header, a second data header, and a third data header. The feature parameters of the first area are obtained through the detection network. The feature parameters can be the position coordinates of the first area. The position coordinates can include but are not limited to the following information: the center coordinates of the first area, the width and height of the first area, the state of the traffic light, and the confidence level. The confidence level indicates the accuracy of the determined state of the traffic light. Furthermore, the above position coordinates are used as position parameters, and the purpose of selecting the first area in the first traffic image can be achieved based on the position parameters of the first area.

[0051] Furthermore, before extracting features from the first region, the first deep learning model can be trained to avoid errors in the first deep learning model, resulting in the selection of an erroneous first region in the first traffic image. A training method is provided in an embodiment of the present application as follows: first, the circumscribed rectangle of the traffic light is used as a selection box to select the first region in the first traffic image. Then, the standard position information of the first region is determined; wherein the standard position information includes the coordinates of the center point of the first region. The standard position information can be the average value of the first region in the first traffic image for the set number of selected regions. It should be noted that the extraction of the first traffic image can be continuous extraction or extraction at set intervals until a set number of first traffic images are extracted. For example, for example, the set number is set to 200, and the first deep learning model is used. For these 200 first traffic images, the first region is selected and the average value of the width-to-length ratio of the first region is calculated to be 1.0; at the same time, the horizontal and vertical coordinates in the coordinates of the center point of each first region are averaged, and the coordinates of the center point of the first region are determined to be (50, 40). Then, the width-to-length ratio interval can be set to (0.8-1.2) in the standard position information, and the center point coordinates can be set to (50, 40). Finally, a set number of the first traffic images are continuously extracted, at least one first area is determined in the set number of first traffic images, and it is determined whether the position information of the at least one first area is consistent with the standard position information. If not, it means that the parameters of the first deep learning model need to be further adjusted and optimized. If so, the position information of the first area in any first traffic image is used as the position parameter of the first area; and the grayscale value and gradient value of all pixels in the first area in any first traffic image are determined to obtain the first feature parameter of the first area. Among them, when determining whether the position information of the at least one first area is consistent with the standard position information, the method adopted can be to directly compare whether the center point coordinates meet the error requirements; it can also be determined by finding the intersection and union ratio, and comparing the intersection and union ratio with the set threshold.

[0052] It is worth noting that the first area of ​​the traffic light framed in the first traffic image is applicable to but not limited to the first area of ​​any one of the red, green and yellow color lights in the traffic light. For the recognition of the color of the traffic light, a recognition rule can be set in the above-mentioned detection network, that is, the image features acquired in the backbone network are mapped into classification information and corresponding confidence through the recognition rule, and the classification information with the maximum confidence and the confidence exceeding the threshold is determined as the type of traffic light in the first area. Among them, the classification information refers to the color information of the traffic light. Determining the type (color type) of the traffic light in this way can improve the efficiency and accuracy of auxiliary electric police cameras in collecting photographic evidence of violations such as vehicles running red lights.

[0053] Furthermore, after the correct first area is selected, the position parameters and first characteristic parameters of the first area can be determined. Among them, the method of expressing the position parameters of the first area can be determined based on the shape of the first area. When the shape of the first area is a circle, the position parameters can be expressed based on the center point coordinates and the radius. When the shape of the first area is a square, the position parameters can be expressed based on the coordinates of the four vertices, or based on the center point coordinates and the distance from the center point to the four vertices. The first characteristic parameter indicates the grayscale characteristics and gradient characteristics of the pixel points in the first area; here the gradient characteristics can be expressed by the difference characteristics between the pixel points in the first area. The following provides a method for determining the grayscale characteristics and gradient characteristics of the pixel points in the first area:

[0054] Assuming that the coordinates of any pixel point in the first area are (i, j), the calculation method of the grayscale feature I of the pixel point with coordinates (i, j) is:

[0055]

[0056] Among them, floor() is a floor rounding function, and r, g, and b correspond to the three channel data of the image respectively.

[0057] The calculation method of the gradient feature G is:

[0058] G i (i,j)=I(i+1,j)-I(i-1,j);

[0059] G j (i,j)=I(i,j+1)-I(i,j-1);

[0060]

[0061] Among them, Gi is the gradient in the row direction, Gj is the gradient in the column direction, and G is the gradient value of the pixel with coordinates (i, j) in the first area.

[0062] It can be seen that the gradient feature can be used to represent the grayscale change feature in the first region. The grayscale feature and the gradient feature are both composed of the grayscale features and the gradient features of all the pixels in the first region, that is, the grayscale features and the gradient features of the first region are both in matrix form.

[0063] According to the grayscale features and gradient features, the first characteristic parameter X of the pixel point (i, j) can be determined as:

[0064] X(i,j)=I(i,j)+G(i,j);

[0065] Similarly, the first characteristic parameter of each pixel in the first region can be calculated by the above formula, and the matrix formed by the first characteristic parameters of all pixels in the first region can represent the first characteristic parameter of the first region.

[0066] Step 102: Determine first information of a first area in the first traffic image based on the first characteristic parameter.

[0067] The first information indicates the grayscale features and gradient features of each pixel point in the first area of ​​the first traffic image, as well as the relationship features between the grayscale of each pixel point and the grayscales of other pixels.

[0068] Specifically, in the embodiment of the present application, the first information is set as a target template, and the first information M is:

[0069]

[0070] in, λ is a preset constant, σ is the preset standard deviation parameter, ☉ is the dot multiplication operation, F(·) is the Fourier transform, F -1 (·) is the inverse Fourier transform, X is the first characteristic parameter, X′ is the transpose of X, w is the length of the first region, h is the width of the first region, (i, j) is the pixel coordinates in the first region, (i c , j c ) is the coordinate of the central pixel point in the first area. Using Fourier transform in the above formula can improve the calculation efficiency.

[0071] Step 103: Determine a second area in a second traffic image and second information of the second area based on the position parameters of the first area.

[0072] The second area includes the coordinates of the center point, and the second information indicates the grayscale characteristics and gradient characteristics of each pixel point in the second area, as well as the relationship characteristics between the grayscale of each pixel point and the grayscales of other pixels.

[0073] Specifically, according to the position parameters of the first area, the second area at the same position can be located in the second traffic image, and the characteristic parameters of the second area can be extracted by referring to the method of step 102. Based on the characteristic parameters of the second area, the second information M of the second area can be determined. 2 .

[0074] Step 104: Using the first information as a template, determining the correlation between the template and the second information, and determining the location of the pixel corresponding to the maximum correlation value as the center location of the traffic light in the second traffic image.

[0075] The correlation indicates the correlation between all pixel points in the first area of ​​the first traffic image and all pixel points in the second area of ​​the second traffic image.

[0076] Specifically, in the embodiment of the present application, the correlation between the template and the second information can be determined by the target tracking algorithm and the signal correlation principle. Based on the target tracking algorithm and the signal correlation principle, the response of the correlation can use the following formula:

[0077]

[0078]

[0079] in, is the second characteristic parameter of the second region, is the Gaussian distribution matrix corresponding to the correlation response graph.

[0080] According to the principle that the Gaussian distribution signal decays from the center of the traffic light to the surrounding areas; and the target tracking algorithm and signal correlation: use the Gaussian kernel to calculate the correlation between two adjacent frames, and take the point with the largest response, that is, the pixel point corresponding to the maximum correlation value as the predicted target center. In the embodiment of the present application, the first area in the first traffic image is used as a template, and the center of the traffic light in the template coincides with the center point of the first area. Then, determining the point with the largest response between the first area and the second area in the second traffic image is actually to determine such a pixel point in the second area: the pixel point is consistent with the pixel features and gradient features of the midpoint of the first area (that is, the midpoint of the traffic light), and the attenuation relationship between the feature intensity of the pixel point and the feature intensity of other pixels in the area is consistent with the feature intensity relationship between the pixel point located at the center position in the first area and other pixels in the area. Furthermore, in the Gaussian distribution matrix of the second area The element with the largest median value is the maximum correlation value; if the value is greater than the set threshold, it is determined that the traffic light tracking is successful. Then, the location of the pixel corresponding to the value is the center position of the traffic light in the second traffic image. If the value is not greater than the set threshold, it is determined that the traffic light tracking fails, and the second area does not include the center of the traffic light, then the first deep learning model is readjusted to obtain the second area including the center of the traffic light.

[0081] Furthermore, when the traffic light is tracked successfully and it is determined that the traffic light is offset, the above template (i.e., the first information) can be updated, thereby avoiding the problem of inaccurate judgment of the electric police camera due to the offset of the traffic light. The method of updating the template proposed in the embodiment of the present application is: first, using the length and width of the first area in the first traffic image, a third area with the center position of the traffic light in the second traffic image as the area center is determined in the second traffic image.

[0082] Then, the third information of the third area is determined; wherein the third information indicates the grayscale feature and gradient feature of each pixel in the third area, and the relationship feature between the grayscale of each pixel and the grayscale of other pixels. Then, the template and the third information are multiplied by the corresponding weights and summed to obtain fourth information, and the template is updated based on the fourth information, and the updated template can be used to track the traffic light in the next frame of traffic image.

[0083] For example, assuming that the center coordinates of the traffic light in the second traffic image are (i y , j y ), then the third information M * for:

[0084]

[0085] in, λ is a preset constant, σ is the preset standard deviation parameter, ☉ is the dot multiplication operation, F(·) is the Fourier transform, F -1 (·) is the inverse Fourier transform, X * is the first characteristic parameter, X * ′ is X * The transpose of , w is the length of the third region, and h is the width of the third region. Here, the length and width of the third region are consistent with the length and width of the first region and the second region.

[0086] Then the updated first characteristic parameter is: X = (1-β)*X + β*X * ;

[0087] The first updated information is: M = (1-β)*M + β*M * .

[0088] Further, based on the traffic light tracking method described in steps 101 to 104, the following describes an example of tracking a traffic light in a new traffic image. Figure 3 :

[0089] Assume that an electric police camera is set up directly opposite the traffic light at the valley intersection. The electric police camera is used to track the traffic light and determine the state of the tracked traffic light, so as to determine the violation / violation of the vehicle for evidence collection. The first information (template) of the first traffic image in the embodiment of the present application can be a correlation filter determined based on the first area, which is used to correlate with the new target and determine the point with the highest response as the target position, that is, the pixel corresponding to the maximum correlation value is the center point of the target. Then, after acquiring the new traffic image, it is first necessary to determine whether the detection area selected by the deep learning model is correct. The determination method includes, but is not limited to, comparing the position information, state parameters and standard position information of the detection area, and standard state information. That is, comparing whether the color of the traffic light in the detection area is consistent with the standard state information, and comparing the center coordinates of the traffic light in the detection area, the aspect ratio of the detection area and the standard position information. If they are inconsistent, the next frame image is re-extracted for determination. If they are still inconsistent, it is determined that the deep learning model and / or standard position information need to be adjusted, that is, after determining the model parameters of the deep learning model, the standard position information and standard state information are re-determined; that is, steps 101 to 103 are executed. If they are consistent, initialization is performed, and the second information of the detection area is extracted (based on the shape parameters and feature parameters of the current tracker, the correlation filter of the current tracker is determined), and the correlation is calculated with the template of the first area of ​​the previous traffic image (that is, the first information) to determine the center position of the traffic light in the detection area, and then determine whether the traffic light is offset. That is, based on the target tracking algorithm and the principle of signal correlation, the correlation filter of the first area is used as a template, and the regional information and the correlation filter of the first area are used to calculate the correlation, so that the maximum response position can be determined, that is, the pixel corresponding to the maximum correlation value is the center point of the traffic light in the new traffic image. Then, based on the maximum response position, the position of the rectangular box, that is, the position parameter of the area where the traffic light is located, can be updated. The new rectangular frame position can be used to obtain the next frame of traffic image information and accurately obtain the state change of the traffic light to avoid the traffic light overflowing the target area, resulting in misjudgment or inability to judge the state of the traffic light.

[0090] Based on the same inventive concept, a traffic light tracking device is provided in an embodiment of the present application. Figure 1 The tracking method of the traffic light shown in the figure corresponds to the specific implementation of the device. The specific implementation of the device can be found in the description of the above method embodiment part, and the repeated parts will not be repeated. Figure 4 , the device comprises:

[0091] Parameter unit 401: used to determine the position parameters and first characteristic parameters of a first area where a traffic light is located in a first traffic image; wherein the first area includes the coordinates of the center point of the traffic light in the first traffic image, and the first characteristic parameters indicate the grayscale characteristics and gradient characteristics of the pixel points in the first area.

[0092] Specifically, the circumscribed rectangle of the traffic light is used as a selection box to select the first area in the first traffic image; the standard position information of the first area is determined; wherein the standard position information includes the coordinates of the center point of the first area; a set number of the first traffic images are extracted, at least one first area is determined in each of the first traffic images, and it is determined whether the position information of the at least one first area is consistent with the standard position information; if so, the position information of the first area in any one of the first traffic images is used as the position parameter of the first area; and the grayscale value and gradient value of all pixels in the first area in any one of the first traffic images are determined to obtain the first characteristic parameter of the first area.

[0093] Information unit 402: used to determine first information of a first area in the first traffic image based on the first characteristic parameter; wherein the first information indicates the grayscale characteristics and gradient characteristics of each pixel in the first area in the first traffic image, as well as the relationship characteristics between the grayscale of each pixel and the grayscales of other pixels.

[0094] Specifically, the first information M is:

[0095]

[0096] in, λ is a preset constant, σ is the preset standard deviation parameter, ☉ is the dot multiplication operation, F(·) is the Fourier transform, F -1 (·) is the inverse Fourier transform, X is the first characteristic parameter, X′ is the transpose of X, w is the length of the first region, h is the width of the first region, (i, j) is the pixel coordinates in the first region, (i c , j c ) are the coordinates of the central pixel point in the first area.

[0097] Determination unit 403: used to determine a second area in the second traffic image and second information of the second area based on the position parameters of the first area; wherein the second area includes the coordinates of the center point, and the second information indicates the grayscale characteristics and gradient characteristics of each pixel point in the second area, and the relationship characteristics between the grayscale of each pixel point and the grayscale of other pixels.

[0098] Central unit 404: used to use the first information as a template, determine the correlation between the template and the second information, and determine that the position of the pixel point corresponding to the maximum correlation value is the center position of the traffic light in the second traffic image; wherein the correlation indicates the correlation between all pixel points in the first area of ​​the first traffic image and all pixel points in the second area of ​​the second traffic image.

[0099] The central unit is specifically used to determine the correlation between the template and the second information based on the target tracking algorithm and the signal correlation principle; compare the maximum value in the correlation with the set threshold, and when the maximum value in the correlation is greater than the set threshold, it is determined that the traffic light tracking is successful.

[0100] The traffic light tracking device also includes an updating unit, which is specifically used to use the length and width of the first area in the first traffic image to determine a third area in the second traffic image with the center position of the traffic light in the second traffic image as the area center; determine third information of the third area; wherein the third information indicates the grayscale characteristics and gradient characteristics of each pixel in the third area, and the relationship characteristics between the grayscale of each pixel and the grayscale of other pixels; respectively multiply the template and the template with the corresponding weights and sum them to obtain fourth information, and update the template based on the fourth information for tracking the traffic light in the next frame of traffic image.

[0101] Based on the same inventive concept, the embodiment of the present application further provides a readable storage medium, including:

[0102] Memory,

[0103] The memory is used to store instructions. When the instructions are executed by the processor, the device including the readable storage medium completes the traffic light tracking method as described above.

[0104] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0105] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0106] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0107] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0108] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: a universal serial bus flash disk (Universal Serial Bus flash disk), a mobile hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a disk or an optical disk, and other media that can store program codes.

[0109] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for tracking a traffic light, characterized in that: include: Determine a position parameter and a first characteristic parameter of a first region in a first traffic image where a traffic light is located; wherein the first region includes the coordinates of a center point of the traffic light in the first traffic image, and the first characteristic parameter indicates a grayscale characteristic and a gradient characteristic of a pixel point in the first region; Based on the first characteristic parameter, first information of a first area in the first traffic image is determined; wherein the first information indicates the grayscale characteristics and gradient characteristics of each pixel in the first area in the first traffic image, and the relationship characteristics between the grayscale of each pixel and the grayscales of other pixels; Based on the position parameters of the first area, determine a second area in the second traffic image and second information of the second area; wherein the second area includes the coordinates of the center point, and the second information indicates the grayscale characteristics and gradient characteristics of each pixel point in the second area, and the relationship characteristics between the grayscale of each pixel point and the grayscale of other pixels; The first information is used as a template, the correlation between the template and the second information is determined, and the position of the pixel corresponding to the maximum correlation value is determined to be the center position of the traffic light in the second traffic image; wherein the correlation indicates the correlation between all pixel points in the first area of ​​the first traffic image and all pixel points in the second area of ​​the second traffic image.

2. The method according to claim 1, characterized in that The determining the correlation between the template and the second information includes: Determining the correlation between the template and the second information based on a target tracking algorithm and a signal correlation principle; The maximum value of the correlation is compared with a set threshold value, and when the maximum value of the correlation is greater than the set threshold value, it is determined that the traffic light tracking is successful.

3. The method according to claim 1 or 2, characterized in that The determining of the position parameter and the first characteristic parameter of the first area where the traffic light is located in the first traffic image includes: Using the circumscribed rectangle of the traffic light as a selection frame to select the first area in the first traffic image; Determine standard location information of the first area; wherein the standard location information includes the coordinates of the center point of the first area; extracting a set number of the first traffic images, determining at least one first area in each of the first traffic images, and determining whether location information of the at least one first area is consistent with the standard location information; If so, the location information of the first area in any one of the first traffic images is used as the location parameter of the first area; and the grayscale value and gradient value of all pixels in the first area in any one of the first traffic images are determined to obtain the first characteristic parameter of the first area.

4. The method according to claim 3, characterized in that The first information M is: in, λ is a preset constant, σ is the preset standard deviation parameter, ☉ is the dot multiplication operation, F(·) is the Fourier transform, F -1 (·) is the inverse Fourier transform, X is the first characteristic parameter, X′ is the transpose of X, w is the length of the first region, h is the width of the first region, (i, j) is the pixel coordinates in the first region, (i c , j c ) are the coordinates of the central pixel point in the first area.

5. The method according to claim 4, characterized in that After determining that the position of the pixel point corresponding to the maximum correlation value is the center position of the traffic light in the second traffic image, the method includes: Determine, in the second traffic image, a third area with a center position of a traffic light in the second traffic image as an area center by using the length and width of the first area in the first traffic image; Determine third information of the third area; wherein the third information indicates the grayscale characteristics and gradient characteristics of each pixel in the third area, and the relationship characteristics between the grayscale of each pixel and the grayscales of other pixels; The template and the third information are respectively multiplied by corresponding weights and summed to obtain fourth information, and the template is updated based on the fourth information for tracking the traffic light in the next frame of traffic image.

6. A tracking device for a traffic light, characterized in that: include: A parameter unit: used to determine the position parameter and the first characteristic parameter of the first area where the traffic light is located in the first traffic image; wherein the first area includes the coordinates of the center point of the traffic light in the first traffic image, and the first characteristic parameter indicates the grayscale characteristics and gradient characteristics of the pixel points in the first area; An information unit: used to determine first information of a first area in the first traffic image based on the first characteristic parameter; wherein the first information indicates the grayscale feature and gradient feature of each pixel in the first area in the first traffic image, and the relationship feature between the grayscale of each pixel and the grayscale of other pixels; A determination unit: used to determine a second area in a second traffic image and second information of the second area based on the position parameters of the first area; wherein the second area includes the coordinates of the center point, and the second information indicates the grayscale characteristics and gradient characteristics of each pixel point in the second area, and the relationship characteristics between the grayscale of each pixel point and the grayscale of other pixels; Central unit: used to use the first information as a template, determine the correlation between the template and the second information, and determine that the position of the pixel point corresponding to the maximum correlation value is the center position of the traffic light in the second traffic image; wherein the correlation indicates the correlation between all pixel points in the first area of ​​the first traffic image and all pixel points in the second area of ​​the second traffic image.

7. The device according to claim 6, characterized in that The central unit is specifically used to determine the correlation between the template and the second information based on the target tracking algorithm and the signal correlation principle; compare the maximum value in the correlation with the set threshold, and when the maximum value in the correlation is greater than the set threshold, it is determined that the traffic light tracking is successful.

8. The device according to claim 6 or 7, characterized in that The parameter unit is specifically used to use the circumscribed rectangle of the traffic light as a selection box to select the first area in the first traffic image; determine the standard position information of the first area; wherein the standard position information includes the coordinates of the center point of the first area; extract a set number of the first traffic images, determine at least one first area in each of the first traffic images, and determine whether the position information of the at least one first area is consistent with the standard position information; if so, use the position information of the first area in any of the first traffic images as the position parameter of the first area; and determine the grayscale value and gradient value of all pixels in the first area in any of the first traffic images to obtain the first characteristic parameter of the first area.

9. The device according to claim 8, characterized in that The first information M is: in, λ is a preset constant, σ is the preset standard deviation parameter, ☉ is the dot multiplication operation, F(·) is the Fourier transform, F -1 (·) is the inverse Fourier transform, X is the first characteristic parameter, X′ is the transpose of X, w is the length of the first region, h is the width of the first region, (i, j) is the pixel coordinates in the first region, (i c , j c ) are the coordinates of the central pixel point in the first area.

10. The device according to claim 9, characterized in that The device also includes an updating unit, which is specifically used to determine, in the second traffic image, a third area with the center position of the traffic light in the second traffic image as the area center by using the length and width of the first area in the first traffic image; determine third information of the third area; wherein the third information indicates the grayscale feature and gradient feature of each pixel point in the third area, and the relationship feature between the grayscale of each pixel point and the grayscale of other pixels; The template and the third information are respectively multiplied by corresponding weights and summed to obtain fourth information, and the template is updated based on the fourth information for tracking the traffic light in the next frame of traffic image.

11. A readable storage medium, characterized in that: include, Memory, The memory is used to store instructions. When the instructions are executed by the processor, the device including the readable storage medium implements the method according to any one of claims 1 to 5.

Images