Traffic light-based vehicle control methods, devices, electronic equipment, and storage media

Abstract

This application provides a vehicle control method, device, electronic device, and storage medium based on traffic lights. The method, applied to a vehicle, includes: acquiring an environmental image at a current moment; the environmental image representing the environment in front of the vehicle; determining and storing the state information of traffic lights from the environmental image at the current moment; the state information includes the color of the traffic light and its actual position in the environmental image; acquiring the actual position of the traffic light in the environmental image at a previous moment, and determining a predicted position of the traffic light in the environmental image at the current moment based on the actual position of the traffic light in the environmental image at the previous moment; if the difference between the actual position and the predicted position of the traffic light in the environmental image at the current moment is less than a preset offset, then controlling the vehicle to move based on the color of the traffic light in the environmental image at the current moment. This application provides accurate traffic light identification, improving vehicle control precision.

Description

Technical Field

[0001] This application relates to autonomous driving technology, and more particularly to a vehicle control method, device, electronic device, and storage medium based on traffic lights. Background Technology

[0002] Autonomous driving technology is currently a major technological direction for the intelligent and connected development of the global automotive industry and the smart transportation sector, and has become a strategic high ground that countries are vying for. During vehicle operation, multiple sensors work together to help autonomous vehicles identify objects on the road, such as traffic lights, in real time. Traffic lights are a crucial tool for vehicle operation; therefore, accurate recognition of traffic lights is essential for vehicle control.

[0003] In existing technologies, road images are acquired during autonomous driving to identify traffic lights and determine whether the vehicle should slow down or continue driving. However, traffic lights may be obstructed by objects or affected by factors such as ambient light, leading to incorrect recognition results, affecting the vehicle's control accuracy, and consequently impacting driving safety. Summary of the Invention

[0004] This application provides a vehicle control method, device, electronic device, and storage medium based on traffic lights to improve the control accuracy of vehicle driving.

[0005] In a first aspect, this application provides a vehicle control method based on traffic lights, which is applied to a vehicle and includes:

[0006] Acquire an environmental image captured at the current moment; wherein the environmental image is used to represent the environment in front of the vehicle;

[0007] The status information of the traffic lights is determined from the environmental image acquired at the current moment and stored; wherein, the status information includes the color of the traffic lights and the actual position of the traffic lights in the environmental image;

[0008] Obtain the actual position of the traffic light in the environmental image captured at the previous moment, and determine the predicted position of the traffic light in the environmental image captured at the current moment based on the actual position of the traffic light in the environmental image captured at the previous moment.

[0009] If the difference between the actual position of the traffic light in the environmental image captured at the current moment and the predicted position is less than a preset offset, then the vehicle is controlled to move according to the color of the traffic light in the environmental image captured at the current moment.

[0010] Secondly, this application provides a vehicle control device based on traffic lights, which is applied to a vehicle and includes:

[0011] An environmental image acquisition module is used to acquire an environmental image collected at the current moment; wherein, the environmental image is used to represent the environment in front of the vehicle;

[0012] The status information determination module is used to determine and store the status information of the traffic lights from the environmental image acquired at the current moment; wherein, the status information includes the color of the traffic lights and the actual position of the traffic lights in the environmental image;

[0013] The predicted position determination module is used to obtain the actual position of the traffic light in the environmental image acquired at the previous moment, and determine the predicted position of the traffic light in the environmental image acquired at the current moment based on the actual position of the traffic light in the environmental image acquired at the previous moment.

[0014] The vehicle control module is used to control the vehicle to drive based on the color of the traffic light in the environmental image acquired at the current time if the difference between the actual position of the traffic light in the environmental image acquired at the current time and the predicted position is less than a preset offset.

[0015] Thirdly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0016] The memory stores computer-executed instructions;

[0017] The processor executes computer execution instructions stored in the memory to implement the traffic light-based vehicle control method as described in the first aspect of this application.

[0018] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the traffic light-based vehicle control method as described in the first aspect of this application.

[0019] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the traffic light-based vehicle control method as described in the first aspect of this application.

[0020] This application provides a vehicle control method, device, electronic device, and storage medium based on traffic lights. It acquires an environmental image at the current moment and identifies the color and actual position of the traffic lights within the image. It also acquires the actual position of the traffic lights in the previous moment's environmental image and predicts their trajectory. If the predicted position of the traffic lights at the current moment matches their actual position, then the traffic lights in the current environmental image are the basis for vehicle driving control, and the vehicle can be controlled based on the color of the signal. This solves the problem in existing technologies where a single environmental image is affected by factors such as ambient light, leading to recognition errors or failures. It improves the recognition accuracy of traffic lights, thereby enhancing the control accuracy and safety of vehicle driving. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0022] Figure 1 A schematic flowchart illustrating a vehicle control method based on traffic lights, provided as an embodiment of this application;

[0023] Figure 2 A schematic flowchart illustrating a vehicle control method based on traffic lights, provided as an embodiment of this application;

[0024] Figure 3 This is a schematic diagram of the backbone network of the traffic light recognition model provided in the embodiments of this application;

[0025] Figure 4 A schematic flowchart illustrating a vehicle control method based on traffic lights, provided as an embodiment of this application;

[0026] Figure 5 A structural block diagram of a vehicle control device based on traffic lights provided in an embodiment of this application;

[0027] Figure 6 A structural block diagram of a vehicle control device based on traffic lights provided in an embodiment of this application;

[0028] Figure 7 A structural block diagram of an electronic device provided in an embodiment of this application;

[0029] Figure 8 This is a structural block diagram of an electronic device provided in an embodiment of this application.

[0030] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0032] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0033] In the following description, when referring to the accompanying drawings, the same numbers in different drawings denote the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0034] In the description of this application, it should be understood that the terms "first," "second," "third," etc., are used only to distinguish similar objects and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances. Furthermore, in the description of this application, unless otherwise stated, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0035] It should be noted that, due to space limitations, this application specification does not exhaustively list all possible implementation methods. Those skilled in the art, after reading this application specification, should be able to deduce that, as long as the technical features do not contradict each other, any combination of technical features can constitute an optional implementation method. The following provides a detailed description of each embodiment.

[0036] Autonomous driving technology is a product of the deep integration of the traditional automotive industry with advanced artificial intelligence, the Internet of Things, high-performance computing, and other next-generation information technologies. It represents the main direction for the intelligent and connected development of global automotive companies and the smart transportation sector, and has become a strategic high ground for competition among nations. The autonomous driving perception system is essentially the "eyes" of an autonomous vehicle, using multiple sensors to help it identify drivable areas, other vehicles, pedestrians, traffic lights, and other obstacles in real time. Traffic lights have become a crucial tool for traffic safety and transportation hubs; therefore, accurate traffic light recognition is vital for the safe driving of autonomous vehicles.

[0037] Currently, traffic light recognition methods extract the color features of the lights. However, the color of traffic lights is easily affected by factors such as ambient light, leading to incorrect recognition results. Furthermore, traffic lights may be obscured by objects such as leaves, further increasing the error rate.

[0038] This application provides a vehicle control method, device, electronic device, and storage medium based on traffic lights, which aims to solve the above-mentioned technical problems of the prior art.

[0039] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0040] Figure 1 This is a flowchart illustrating a traffic light-based vehicle control method according to an embodiment of this application. The method is applied to vehicles and can be executed by a traffic light-based vehicle control device. Figure 1 As shown, the method includes the following steps:

[0041] S101. Acquire the environmental image collected at the current moment; wherein, the environmental image is used to represent the environment in front of the vehicle.

[0042] For example, an image acquisition device, such as a camera, can be installed on the vehicle. The image acquisition device can be installed at the front of the vehicle, for example, at the hood. While the vehicle is in motion, the image acquisition device can acquire images in real-time or periodically to obtain environmental images. That is, the environmental images can represent the environment in front of the vehicle, and may include, for example, roads, trees, pedestrians, other vehicles, and traffic lights.

[0043] The vehicle can acquire environmental images captured by the camera in real time or at set intervals. For example, if the camera captures an environmental image at a given moment, the vehicle can acquire that image in real time as the current environmental image. The vehicle can also preset the image acquisition period and acquire environmental images at set intervals, for example, the vehicle can acquire one frame of environmental image every 0.1 seconds.

[0044] S102. Determine and store the status information of the traffic lights from the environmental image acquired at the current moment; wherein, the status information includes the color of the traffic lights and the actual position of the traffic lights in the environmental image.

[0045] For example, after acquiring the current environmental image, the vehicle identifies traffic lights within that image. Specifically, it determines whether traffic lights exist in the current environmental image and determines their status information. This status information may include the color of the traffic light and its actual position in the environmental image. The color category may include red, green, yellow, black, and unknown. The actual position of the traffic light in the environmental image refers to its coordinates. A black traffic light means it is not lit. An unknown color can mean the traffic light is a color other than red, green, yellow, or black. For example, if the traffic light is obscured by an object and cannot be identified, its color may be unknown; or if the color is unclear due to factors such as lighting conditions, its color may also be unknown.

[0046] A neural network model can be pre-set, which can be used to identify traffic lights in environmental images. For example, based on a combination of color segmentation and feature matching, the colors of the environmental image can be segmented to extract the basic geometric features of the traffic lights, thereby locating the traffic lights and classifying and recognizing their colors.

[0047] After obtaining the status information of the traffic lights in the current environmental image, the current time can be associated with and stored with the traffic light status information. For example, the color of the traffic lights in the environmental image and their actual coordinates in the environmental image at 12 noon can be stored.

[0048] If the environmental image includes multiple traffic lights, the status information of each traffic light can be identified, a unique identifier can be assigned to each traffic light, and the status information of each traffic light at the current moment can be stored.

[0049] S103. Obtain the actual position of the traffic light in the environmental image acquired at the previous moment, and determine the predicted position of the traffic light in the environmental image acquired at the current moment based on the actual position of the traffic light in the environmental image acquired at the previous moment.

[0050] For example, the previous moment is the moment preceding the current moment, and it can be a moment consecutive to the current moment. The previous moment can be determined based on the image acquisition period. For instance, if the vehicle acquires an environmental image every 0.1 seconds, and the current time is 12:00, then the previous moment is 0.1 seconds before 12:00. That is, the environmental image acquired at the previous moment is the last environmental image acquired by the vehicle before acquiring the environmental image at the current moment.

[0051] Since the vehicle stores the status information of the traffic lights in each environmental image it acquires, it can directly obtain the status information of the traffic lights in the environmental image acquired at the previous moment. For example, it can obtain the actual position of the traffic lights in the environmental image acquired at the previous moment.

[0052] Traffic lights in environmental images can be tracked. That is, based on the actual position of the traffic light in the environmental image acquired at the previous moment, its position in the current environmental image can be inferred, and this inferred position is determined as the predicted position. In other words, the predicted position of the traffic light in the current environmental image can be determined based on its actual position in the environmental image acquired at the previous moment. For example, if a vehicle is moving forward on a road, and the traffic light was positioned lower in the previous environmental image, in the next environmental image, because the vehicle is moving forward and the road is uphill, the traffic light will be positioned higher, and the area where the traffic light is located will be larger.

[0053] In this embodiment, obtaining the actual position of the traffic light in the environmental image acquired at the previous moment includes: if the traffic light is not present in the environmental image acquired at the previous moment, then obtaining the actual position of the traffic light in at least one environmental image prior to the previous moment.

[0054] Specifically, traffic lights may not be present in the environmental images acquired by the vehicle, for example, because the traffic lights are obstructed by objects or there are no traffic lights on the road. After acquiring the environmental image collected at the previous moment, if it is determined that the actual location of the traffic light is not present in the stored traffic light status information corresponding to that environmental image, i.e., there is no traffic light, then environmental images from before the previous moment can be acquired. For example, the environmental image collected at the moment before the previous moment can be acquired.

[0055] When acquiring environmental images from the previous moment, multiple environmental images prior to the current moment can be acquired; for example, five consecutive environmental images prior to the current moment can be acquired. Alternatively, multiple environmental images can be acquired from previous moments. The actual positions of each traffic light in each environmental image can then be obtained.

[0056] The advantage of this setup is that if the traffic light is not present in the environmental image of the previous moment, it can continue to acquire images from previous moments to track and calculate the trajectory of the traffic light, thereby improving the accuracy of determining the position of the traffic light and thus enhancing driving safety.

[0057] S104. If it is determined that the difference between the actual position and the predicted position of the traffic light in the environmental image collected at the current time is less than a preset offset, then the vehicle is controlled to drive according to the color of the traffic light in the environmental image collected at the current time.

[0058] For example, after obtaining the predicted position of the traffic light at the current moment, it is determined whether the actual position of the traffic light in the environmental image acquired at the current moment is the predicted position. If so, the trajectory tracking of the traffic light is considered successful. That is, the traffic light is the one in front of the vehicle, and the vehicle's movement is controlled based on the color of the traffic light in the environmental image acquired at the current moment. Vehicle driving control rules can be preset, and the vehicle's movement is controlled based on the color of the traffic light and the vehicle driving control rules. For example, the vehicle driving control rules could be acceleration at green lights and deceleration at red lights. If there is no actual position in the environmental image acquired at the current moment that matches the predicted position, the vehicle's movement is not controlled based on the color of the traffic light in the environmental image acquired at the current moment. A new environmental image can be acquired to re-identify the traffic light.

[0059] Vehicle control can also be based on the color of the traffic light in the current environmental image and the color in the previous environmental image. If the color of the traffic light in the current environmental image and the color in the previous environmental image are the same, the vehicle can be controlled according to the color of the traffic light in the current environmental image; if they are not the same, the environmental image of the next moment can be acquired, the current environmental image can be used as the new previous environmental image, and the next environmental image can be used as the new current environmental image, and the traffic light can be identified again.

[0060] The system can also compare the predicted position of the traffic light with its actual position in the currently acquired environmental image to determine the coordinate difference between the predicted and actual positions. A pre-set offset is used, and the coordinate difference is compared to this offset. If the difference is less than the offset, the traffic light's trajectory is considered successfully tracked. That is, the traffic light is identified as the one in front of the vehicle, and the vehicle is controlled based on its color in the currently acquired environmental image. If the difference is equal to or greater than the offset, the vehicle is not controlled based on the traffic light's color in the currently acquired environmental image. The system then acquires a new environmental image for re-identification of the traffic light. Alternatively, vehicle control can be based on the traffic light's color in the current environmental image and its color in the previous environmental image.

[0061] This application provides a vehicle control method based on traffic lights. It acquires an environmental image at the current moment and identifies the color and actual position of the traffic lights within the image. It also acquires the actual position of the traffic lights in the previous moment's environmental image and predicts their trajectory. If the predicted position of the traffic lights at the current moment matches their actual position, then the traffic lights in the current environmental image are the basis for vehicle control, and the vehicle can be controlled based on their color. This solves the problem in existing technologies where a single environmental image is affected by factors such as ambient light, leading to recognition errors or failures. It improves the accuracy of traffic light recognition, thereby enhancing vehicle control accuracy and safety.

[0062] Figure 2 This is a flowchart illustrating a vehicle control method based on traffic lights, which is an optional embodiment based on the above embodiments.

[0063] In this embodiment, determining the status information of the traffic lights from the environmental image collected at the current moment can be further refined as follows: inputting the environmental image collected at the current moment into a preset traffic light recognition model; identifying the traffic lights from the environmental image at the current moment according to the traffic light recognition model, and outputting the status information of the traffic lights.

[0064] like Figure 2 The method includes the following steps:

[0065] S201. Acquire the environmental image collected at the current moment; wherein, the environmental image is used to represent the environment in front of the vehicle.

[0066] For example, this step can refer to step S101 above, and will not be repeated here.

[0067] S202. Input the environmental image collected at the current moment into the preset traffic light recognition model.

[0068] For example, a traffic light recognition model can be pre-trained. This model can be a deep learning neural network model that outputs state information such as the color of the traffic lights in the environmental image and their actual position within the image. After obtaining the environmental image captured at the current moment, the image can be input into the traffic light recognition model.

[0069] In this embodiment, before inputting the environmental image acquired at the current moment into the preset traffic light recognition model, the method further includes: acquiring a pre-acquired traffic light image; wherein the traffic light image is marked with the actual target bounding box of the traffic light, the actual center coordinates of the traffic light, and the actual color of the traffic light; inputting the traffic light image into the pre-built traffic light recognition model, and outputting the predicted color of the traffic light, the predicted center coordinates, and four distance parameters corresponding to the predicted center coordinates; wherein the distance parameters are used to represent the distance between the center coordinates and the four borders of the target bounding box; determining the predicted target bounding box of the traffic light based on the predicted center coordinates and the four distance parameters corresponding to the predicted center coordinates; if the predicted target bounding box is in the same position as the actual target bounding box, and the predicted color is the same as the actual color, then the training of the traffic light recognition model is considered complete.

[0070] Specifically, before using the traffic light recognition model, it needs to be trained. Multiple traffic light images are pre-collected as training data; the size, color, and position of the traffic lights in each image can differ. The traffic light images are pre-labeled with color categories, which can include five labels: red, green, yellow, black, and unknown. The pre-labeled color category represents the actual color of each traffic light in the image.

[0071] Beforehand, target bounding boxes are drawn on each traffic light image. Each target bounding box contains one traffic light, and the target bounding box can be a rectangle, with its four sides tangent to the circle of the traffic light. These pre-drawn target bounding boxes on the traffic light images are the actual target bounding boxes for the traffic lights. The center coordinates of the traffic lights also need to be marked on the traffic light images, serving as the actual center coordinates. In other words, the traffic light images need to be processed beforehand to determine the actual target bounding box, the actual center coordinates, and the actual color of the traffic light for each image.

[0072] The processed traffic light images are input into a pre-built traffic light recognition model. The model can identify the color of the traffic lights in each image, using this as the predicted color. It can also obtain the coordinates of the center of each traffic light in the image, which are used to predict the center coordinates. Furthermore, it can obtain four distance parameters for the predicted center coordinates. These distance parameters refer to the vertical distances between the center coordinates and the four borders of the target bounding box. For example, the distance parameters may include the distances between the center coordinates and the top border, bottom border, left border, and right border of the target bounding box.

[0073] After obtaining the four distance parameters corresponding to the predicted center coordinates, a bounding box can be drawn on the traffic light image based on the position of the predicted center coordinates and the four distance parameters. The drawn box is the predicted target box. If the position of the predicted target box matches the position of the actual target box, and the predicted color matches the actual color, then the traffic light recognition model training is complete. A loss function can be pre-set, and the completion of the traffic light recognition model training can be determined based on the loss function. For example, the loss function can be a classification loss function, a localization loss function, and a center-ness loss function. The role of center-ness is to suppress low-quality target box bounding boxes and improve recognition performance. When determining the center coordinates during training, a center-ness branch can be used. The center-ness branch predicts one parameter at each location point in the traffic light image, reflecting the distance of that point from the actual center coordinates. The value range of this parameter is [0, 1]. The closer to the actual center coordinates, the closer the parameter value is to 1, and vice versa.

[0074] When performing feature extraction, the traffic light recognition model can design its own backbone network. In this embodiment, the backbone network includes five layers: C1, C2, C3, C4, and C5. C1 has one convolutional layer, C2 has two convolutional layers, C3 has two convolutional layers, C4 has three convolutional layers, and C5 has three convolutional layers, for a total of 11 layers. Figure 3This is a schematic diagram of the backbone network of the traffic light recognition model provided in this embodiment. The image input to C1 is 640×640×3, where 640×640 represents the pixel size and 3 represents the RGB channels. C1 convolves to obtain a 320×320×64 image and outputs it to C2, where 320×320 represents the size of the output image from C1 and 64 represents the number of RGB channels. C2 convolves to obtain a 160×160×256 image and outputs it to C3, where 160×160 represents the size of the output image from C2 and 256 represents the number of RGB channels. C3... A convolution process yields an 80×80×512 image, which is then output to C4. Here, 80×80 represents the size of the image output by C3, and 256 represents the number of RGB channels in the C3 output image. C4 then performs convolution to obtain a 40×40×1024 image, which is output to C5. Here, 40×40 represents the size of the image output by C4, and 1024 represents the number of RGB channels in the C4 output image. Finally, C5 performs convolution to obtain a 20×20×2048 image, where 20×20 represents the size of the image output by C5, and 2048 represents the number of RGB channels in the C5 output image. This multi-layer feature extraction from the image improves the accuracy of feature extraction, thereby enhancing the recognition accuracy of traffic lights.

[0075] The advantage of this setup is that by pre-training the traffic light recognition model, the recognition efficiency of traffic lights can be improved. Furthermore, the traffic light recognition model can employ an anchor-free network structure, increasing computational speed.

[0076] Before training on traffic light images, the SAHI (slice-assisted reasoning) method can be used to enhance small target objects on distant traffic lights, thereby improving the recognition accuracy of small targets.

[0077] S203. Based on the traffic light recognition model, identify the traffic lights from the current environmental image and output the status information of the traffic lights.

[0078] For example, a trained traffic light recognition model can identify traffic lights in an environmental image and obtain state information such as the color and position of the traffic lights. That is, the output of the traffic light recognition model can be state information such as the color and position of the traffic lights.

[0079] In this embodiment, based on the traffic light recognition model, traffic lights are identified from the current environmental image, and the status information of the traffic lights is output. This includes: extracting features from the current environmental image based on the feature pyramid network structure in the traffic light recognition model, and determining the target box in the current environmental image; wherein, a target box includes a traffic light; determining the color of the traffic light in the target box, and determining the position of the target box in the current environmental image, which is the actual position of the traffic light in the environmental image.

[0080] Specifically, the traffic light recognition model can employ an FPN (Feature Pyramid Networks) structure. FPN is used to fuse the feature vectors output from each layer of the backbone network. The FPN structure can fuse feature maps of different sizes, thereby achieving accurate recognition of small targets such as distant traffic lights. Through feature fusion and extraction using the FPN structure, the final feature vector is obtained. Based on the final feature vector, the bounding box in the environmental image is determined, i.e., the traffic light in the environmental image is identified.

[0081] For example, performing a 1×1 convolution on the image output from C5 yields image P5. The 1×1 convolution reduces the number of channels, parameters, and computational cost. Downsampling P5 yields P6. Performing a 1×1 convolution on the image output from C4, upsampling P5, and then fusing the upsampled P5 with the 1×1 convolutioned C4 yields P4. Alternatively, performing a 1×1 convolution on the image output from C3, upsampling P4, and then fusing the upsampled P4 with the 1×1 convolutioned C3, can also be used. In this embodiment, fusion can refer to the addition of feature vector matrices.

[0082] The color of the traffic light within the bounding box is determined based on the pixel size. The coordinates of the bounding box within the current environmental image can also be determined, serving as the actual position of the traffic light within the environmental image.

[0083] The advantage of this setup is that by using an FPN structure for feature fusion, it avoids missing feature information and improves the accuracy of traffic light recognition.

[0084] S204. Obtain the actual position of the traffic light in the environmental image acquired at the previous moment, and determine the predicted position of the traffic light in the environmental image acquired at the current moment based on the actual position of the traffic light in the environmental image acquired at the previous moment.

[0085] For example, this step can refer to step S103 above, and will not be repeated here.

[0086] S205. If the difference between the actual position and the predicted position of the traffic light in the environmental image collected at the current moment is less than the preset offset, then the vehicle is controlled to move according to the color of the traffic light in the environmental image collected at the current moment.

[0087] For example, this step can refer to step S104 above, and will not be repeated here.

[0088] This application provides a vehicle control method based on traffic lights. It acquires an environmental image at the current moment and identifies the color and actual position of the traffic lights within the image. It also acquires the actual position of the traffic lights in the previous moment's environmental image and predicts their trajectory. If the predicted position of the traffic lights at the current moment matches their actual position, then the traffic lights in the current environmental image are the basis for vehicle control, and the vehicle can be controlled based on their color. This solves the problem in existing technologies where a single environmental image is affected by factors such as ambient light, leading to recognition errors or failures. It improves the accuracy of traffic light recognition, thereby enhancing vehicle control accuracy and safety.

[0089] Figure 4 This is a flowchart illustrating a vehicle control method based on traffic lights, which is an optional embodiment based on the above embodiments.

[0090] In this embodiment, the predicted position of the traffic light in the environmental image acquired at the current moment is determined based on the actual position of the traffic light in the environmental image acquired at the previous moment. This can be further refined as follows: obtaining the current driving information of the vehicle; wherein, the current driving information includes driving speed, driving acceleration, and driving trajectory; predicting the trajectory of the traffic light based on the actual position of the traffic light in the environmental image acquired at the previous moment and the current driving information of the vehicle, thereby obtaining the predicted position of the traffic light in the environmental image acquired at the current moment.

[0091] like Figure 4 The method includes the following steps:

[0092] S401. Acquire the environmental image collected at the current moment; wherein, the environmental image is used to represent the environment in front of the vehicle.

[0093] For example, this step can refer to step S101 above, and will not be repeated here.

[0094] S402. Determine and store the status information of the traffic lights from the environmental image acquired at the current moment; wherein, the status information includes the color of the traffic lights and the actual position of the traffic lights in the environmental image.

[0095] For example, this step can refer to step S102 above, and will not be repeated here.

[0096] S403. Obtain the actual position of the traffic light in the environmental image collected at the previous moment, and obtain the current driving information of the vehicle; wherein, the current driving information includes driving speed, driving acceleration and driving trajectory.

[0097] For example, the system stores the environmental images acquired at each moment, as well as the status information of the traffic lights in each environmental image. After obtaining the status information of the traffic lights in the environmental image at the current moment, the system obtains the status information of the traffic lights in the environmental image at the previous moment. For example, the system can obtain the actual position of the traffic lights in the environmental image acquired at the previous moment.

[0098] The vehicle is equipped with various sensors, such as speed sensors and acceleration sensors, which can monitor the vehicle's current driving information in real time. The vehicle can acquire current driving information in real time, such as the vehicle's current speed, acceleration, and driving trajectory.

[0099] In this embodiment, the environmental image includes at least two traffic lights; obtaining the actual position of the traffic lights in the environmental image acquired at the previous moment includes: obtaining the actual position of each traffic light in the environmental image acquired at the previous moment, and the tracking identifier assigned to each traffic light.

[0100] Specifically, an environmental image can contain multiple traffic lights. If multiple traffic lights existed in the previous environmental image, the actual positions of each traffic light in that image, along with its tracking identifier, are retrieved. The tracking identifiers for the traffic lights are assigned when the environmental images are stored, and the same traffic light has the same tracking identifier. For example, if the same traffic light appears in two environmental images, its tracking identifier will be the same in both images.

[0101] When the first environmental image is captured, a tracking identifier can be randomly assigned to the traffic lights in that image. In subsequent environmental images, if the same traffic light appears as in the first image, the tracking identifier from the first image is assigned to that traffic light in the subsequent image. If a new traffic light appears in a subsequent image, a random tracking identifier is assigned to the new traffic light. Different traffic lights have different tracking identifiers. In other words, if multiple traffic lights exist in the same environmental image, each traffic light will have a different tracking identifier.

[0102] The advantage of this setup is that, since there may be multiple traffic lights on the road, determining the position and tracking markers of each traffic light in an environmental image makes it easier to distinguish between them, avoids confusing different traffic lights, and improves the accuracy of vehicle control.

[0103] S404. Based on the actual position of the traffic light in the environmental image collected at the previous moment and the current driving information of the vehicle, predict the trajectory of the traffic light to obtain the predicted position of the traffic light in the environmental image collected at the current moment.

[0104] For example, after obtaining the actual position of the traffic light in the environmental image captured at the previous moment, and the current driving information of the vehicle, the trajectory of the traffic light can be predicted based on a preset trajectory tracking algorithm. That is, the position of the traffic light in the environmental image captured at the current moment can be predicted as the predicted position. The trajectory tracking algorithm can be based on Kalman filtering, Hungarian algorithm, and Mahalanobis distance to track the trajectory of the traffic light. In this embodiment, the trajectory tracking algorithm is not specifically limited.

[0105] For example, based on the vehicle's current driving information, it can be determined that the vehicle is accelerating forward at a certain speed. If the previous time was 12:34 and the current time is 12:34:02, then based on the actual position of the traffic light in the environmental image at 12:34, the position of the traffic light in the environmental image two seconds later can be predicted.

[0106] If there are multiple traffic lights in the environmental image of the previous moment, the trajectory of each traffic light is predicted based on the actual position of each traffic light in the environmental image of the previous moment and the current driving information of the vehicle, so as to obtain the predicted position of each traffic light in the environmental image of the current moment.

[0107] S405. If it is determined that the difference between the actual position of the traffic light in the environmental image collected at the current moment and the predicted position is less than a preset offset, then the vehicle is controlled to drive according to the color of the traffic light in the environmental image collected at the current moment.

[0108] For example, after obtaining the predicted position of the traffic light in the environmental image captured at the current moment, the predicted position is compared with the actual position of the traffic light in the environmental image at the current moment. It is then determined whether the difference between the actual position and the predicted position in the environmental image at the current moment is less than an offset. If so, the vehicle is controlled to move according to the color of the traffic light in the environmental image captured at the current moment. If not, the environmental image at the next moment is acquired, and the predicted position of the traffic light in the environmental image at the next moment is predicted based on the actual position of the traffic light in the environmental image at the current moment.

[0109] In this embodiment, if it is determined that the difference between the actual position and the predicted position of the traffic light in the environmental image acquired at the current moment is less than a preset offset, the method includes: comparing the predicted position of each traffic light in the environmental image acquired at the previous moment with the actual position of each traffic light in the environmental image acquired at the current moment; if the deviation between the actual position of a traffic light in the environmental image acquired at the current moment and the predicted position of any traffic light in the environmental image acquired at the previous moment is less than the preset offset, then the traffic light is determined to be the target light, and the tracking identifier of the traffic light corresponding to the target light in the environmental image acquired at the previous moment is assigned to the target light.

[0110] Specifically, if multiple traffic lights exist in the environmental image of the previous moment or the current moment, after obtaining the predicted positions of each traffic light, the predicted positions of each traffic light are compared with their actual positions in the current moment's environmental image. It is then determined whether there is a traffic light in the current moment's environmental image whose actual position deviates from the predicted position of any traffic light in the previous moment's environmental image by a preset offset. If so, that traffic light is identified as the target light. The tracking identifier of the target light in the previous moment's environmental image is determined, and this tracking identifier is assigned to the target light in the current moment's environmental image.

[0111] If there is no matching traffic light in the environmental images of the previous and current times, the next environmental image is acquired, and the predicted position of the traffic light in the next environmental image is predicted based on the actual position of the traffic light in the current environmental image.

[0112] The beneficial effect of this facility is that it tracks traffic lights, identifies the same traffic lights in different environmental images, and thus confirms the status of the traffic lights. It does not use single-frame images to determine the type of traffic lights, improves the recognition accuracy of traffic lights, and thus improves the accuracy of vehicle control.

[0113] If the actual locations of multiple traffic lights in the current environmental image match their predicted locations, the next environmental image can be acquired, and the traffic lights can be identified again.

[0114] In this embodiment, controlling vehicle movement based on the color of traffic lights in the environmental image acquired at the current moment includes: controlling vehicle movement based on the color of target lights in the environmental image acquired at the current moment and a preset vehicle movement control rule.

[0115] Specifically, pre-set vehicle driving control rules can be used, such as slowing down at red lights and accelerating at green lights. Vehicle driving control rules can also be set based on countdowns and arrows displayed on the traffic lights. In this embodiment, the vehicle driving control rules can be determined according to actual needs.

[0116] After determining the color of the target light in the environmental image acquired at the current moment, the vehicle's movement is controlled according to the target light's color and vehicle driving control rules. For example, if the target light is red, the vehicle can be controlled to slow down and stop before the red light.

[0117] The advantage of this setup is that it controls vehicle movement by the color of the target light, avoiding interference from other traffic lights in the current environmental image and improving vehicle control accuracy.

[0118] This application provides a vehicle control method based on traffic lights. It acquires an environmental image at the current moment and identifies the color and actual position of the traffic lights within the image. It also acquires the actual position of the traffic lights in the previous moment's environmental image and predicts their trajectory. If the predicted position of the traffic lights at the current moment matches their actual position, then the traffic lights in the current environmental image are the basis for vehicle control, and the vehicle can be controlled based on their color. This solves the problem in existing technologies where a single environmental image is affected by factors such as ambient light, leading to recognition errors or failures. It improves the accuracy of traffic light recognition, thereby enhancing vehicle control accuracy and safety.

[0119] Figure 5 This is a structural block diagram of a vehicle control device based on traffic lights, provided as an embodiment of this application. For ease of explanation, only the parts relevant to the embodiments of this disclosure are shown. This device is applied to a vehicle, see reference... Figure 5 The device includes: an environmental image acquisition module 501, a status information determination module 502, a predicted position determination module 503, and a vehicle control module 504.

[0120] The environmental image acquisition module 501 is used to acquire an environmental image collected at the current moment; wherein, the environmental image is used to represent the environment in front of the vehicle;

[0121] The status information determination module 502 is used to determine and store the status information of the traffic lights from the environmental image acquired at the current time; wherein, the status information includes the color of the traffic lights and the actual position of the traffic lights in the environmental image;

[0122] The predicted position determination module 503 is used to obtain the actual position of the traffic light in the environmental image collected at the previous moment, and determine the predicted position of the traffic light in the environmental image collected at the current moment based on the actual position of the traffic light in the environmental image collected at the previous moment.

[0123] The vehicle control module 504 is used to control the vehicle to drive based on the color of the traffic light in the environmental image acquired at the current time if the difference between the actual position of the traffic light in the environmental image acquired at the current time and the predicted position is less than a preset offset.

[0124] Figure 6 This application provides a structural block diagram of a vehicle control device based on traffic lights, in accordance with an embodiment of the present application. Figure 5 Based on the illustrated embodiments, as Figure 6 As shown, the state information determination module 502 includes a model input unit 5021 and an information output unit 5022.

[0125] The model input unit 5021 is used to input the environmental image collected at the current moment into a preset traffic light recognition model;

[0126] The information output unit 5022 is used to identify traffic lights from the current environmental image according to the traffic light recognition model and output the status information of the traffic lights.

[0127] In one example, the information output unit 5022 is specifically used for:

[0128] Based on the feature pyramid network structure in the traffic light recognition model, features are extracted from the environmental image at the current moment to determine the target bounding box in the environmental image at the current moment; wherein, a target bounding box includes a traffic light;

[0129] The color of the traffic light in the target frame is determined, and the position of the target frame in the environmental image at the current moment is determined, which is the actual position of the traffic light in the environmental image.

[0130] In one example, the device also includes:

[0131] The model training module is used to obtain pre-collected traffic light images before inputting the environmental image collected at the current moment into the preset traffic light recognition model; wherein, the traffic light images are marked with the actual target box of the traffic light, the actual center coordinates of the traffic light, and the actual color of the traffic light;

[0132] The traffic light image is input into a pre-built traffic light recognition model, and the predicted color of the traffic light, the predicted center coordinates, and four distance parameters corresponding to the predicted center coordinates are output; wherein, the distance parameters are used to represent the distance between the center coordinates and the four borders of the target box;

[0133] The predicted target box of the traffic light is determined based on the predicted center coordinates and the four distance parameters corresponding to the predicted center coordinates.

[0134] If the predicted target box and the actual target box are in the same position, and the predicted color and the actual color are the same, then the traffic light recognition model training is complete.

[0135] In one example, the predicted location determination module 503 includes:

[0136] The actual location acquisition unit is used to acquire the actual location of the traffic light in at least one environmental image before the previous moment if the traffic light is not present in the environmental image acquired at the previous moment.

[0137] In one example, the predicted location determination module 503 includes:

[0138] A driving information acquisition unit is used to acquire the current driving information of the vehicle; wherein, the current driving information includes driving speed, driving acceleration and driving trajectory;

[0139] The position prediction unit is used to predict the trajectory of the traffic light based on the actual position of the traffic light in the environmental image acquired at the previous moment and the current driving information of the vehicle, so as to obtain the predicted position of the traffic light in the environmental image acquired at the current moment.

[0140] In one example, the environmental image includes at least two traffic lights;

[0141] The predicted location determination module 503 includes:

[0142] The identification allocation unit is used to obtain the actual position of each traffic light in the environmental image acquired at the previous moment, and to assign a tracking identification to each traffic light.

[0143] In one example, vehicle control module 504 is specifically used for:

[0144] Compare the predicted positions of each traffic light in the environmental image acquired at the previous moment with the actual positions of each traffic light in the environmental image acquired at the current moment.

[0145] If the deviation between the actual position of a traffic light in the environmental image acquired at the current moment and the predicted position of any traffic light in the environmental image acquired at the previous moment is less than a preset offset, then the traffic light is determined to be the target light, and the tracking identifier of the traffic light corresponding to the target light in the environmental image acquired at the previous moment is assigned to the target light.

[0146] In one example, vehicle control module 504 is specifically used for:

[0147] Based on the color of the target light in the environmental image acquired at the current moment, and based on preset vehicle driving control rules, the vehicle is controlled to drive.

[0148] Figure 7 A structural block diagram of an electronic device provided in an embodiment of this application, such as... Figure 7 As shown, the electronic device includes: a memory 71 and a processor 72; the memory 71 is a memory for storing executable instructions of the processor 72.

[0149] The processor 72 is configured to perform the method provided in the above embodiments.

[0150] The electronic device also includes a receiver 73 and a transmitter 74. The receiver 73 is used to receive instructions and data sent by other devices, and the transmitter 74 is used to send instructions and data to external devices.

[0151] Figure 8 This is a block diagram illustrating an electronic device according to an exemplary embodiment. The device may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness device, personal digital assistant, etc.

[0152] Device 800 may include one or more of the following components: processing component 802, memory 804, power supply component 806, multimedia component 808, audio component 810, input / output (I / O) interface 812, sensor component 814, and communication component 816.

[0153] Processing component 802 typically controls the overall operation of device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the methods described above. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0154] Memory 804 is configured to store various types of data to support the operation of device 800. Examples of this data include instructions for any application or method operating on device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0155] Power supply component 806 provides power to various components of device 800. Power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 800.

[0156] Multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0157] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0158] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0159] Sensor assembly 814 includes one or more sensors for providing status assessments of various aspects of device 800. For example, sensor assembly 814 may detect the on / off state of device 800, the relative positioning of components such as the display and keypad of device 800, changes in the position of device 800 or a component of device 800, the presence or absence of user contact with device 800, the orientation or acceleration / deceleration of device 800, and temperature changes of device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0160] Communication component 816 is configured to facilitate wired or wireless communication between device 800 and other devices. Device 800 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0161] In an exemplary embodiment, device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0162] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, which can be executed by a processor 820 of device 800 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0163] A non-transitory computer-readable storage medium, wherein instructions in the storage medium, when executed by a processor of a terminal device, enable the terminal device to perform the aforementioned traffic light-based vehicle control method of the terminal device.

[0164] This application also discloses a computer program product, including a computer program that, when executed by a processor, implements the method described in this embodiment.

[0165] Various embodiments of the systems and technologies described above in this application can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0166] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or electronic device.

[0167] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0168] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0169] The systems and technologies described herein can be implemented in computing systems that include back-end components (e.g., as data electronic devices), or computing systems that include middleware components (e.g., application electronic devices), or computing systems that include front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0170] Computer systems can include client and electronic devices. Clients and electronic devices are generally geographically separated and typically interact via communication networks. The client-electronic device relationship is created by computer programs running on the respective computers and having a client-electronic device relationship with each other. The electronic device can be a cloud electronic device, also known as a cloud computing electronic device or cloud host, a host product within the cloud computing service system, addressing the shortcomings of traditional physical hosts and VPS services ("Virtual Private Server," or simply "VPS") in terms of management difficulty and weak business scalability. The electronic device can also be an electronic device in a distributed system or an electronic device incorporating blockchain technology. It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application is achieved, and this is not limited herein.

[0171] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0172] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A vehicle control method based on traffic lights, characterized in that, The method is applied to a vehicle, and the method includes: Acquire an environmental image captured at the current moment; wherein the environmental image is used to represent the environment in front of the vehicle, and the environmental image includes at least two traffic lights; The status information of the traffic lights is determined from the environmental image acquired at the current moment and stored; wherein, the status information includes the color of the traffic lights and the actual position of the traffic lights in the environmental image, and the actual position is obtained by segmenting the colors in the environmental image and extracting the basic geometric features of the traffic lights; The system acquires the actual positions of each traffic light in the environmental image captured at the previous moment, as well as the tracking identifiers assigned to each traffic light, and obtains the vehicle's current driving information. This current driving information includes driving speed, driving acceleration, and driving trajectory. Based on the actual positions of the traffic lights in the environmental image captured at the previous moment and the vehicle's current driving information, the trajectory of the traffic lights is predicted to obtain the predicted position of the traffic lights in the environmental image captured at the current moment. The predicted positions of each traffic light in the environmental image acquired at the previous moment are compared with the actual positions of each traffic light in the environmental image acquired at the current moment. If the deviation between the actual position of a traffic light in the environmental image acquired at the current moment and the predicted position of any traffic light in the environmental image acquired at the previous moment is less than a preset offset, then the traffic light is determined to be the target light, and the tracking identifier of the traffic light corresponding to the target light in the environmental image acquired at the previous moment is assigned to the target light. The vehicle is controlled to move according to the color of the target light in the environmental image acquired at the current moment.

2. The method according to claim 1, characterized in that, Determining the status information of the traffic lights from the environmental image acquired at the current moment includes: The environmental image acquired at the current moment is input into a preset traffic light recognition model; Based on the traffic light recognition model, traffic lights are identified from the environmental image at the current moment, and the status information of the traffic lights is output.

3. The method according to claim 2, characterized in that, Based on the traffic light recognition model, traffic lights are identified from the current environmental image, and the status information of the traffic lights is output, including: Based on the feature pyramid network structure in the traffic light recognition model, features are extracted from the environmental image at the current moment to determine the target bounding box in the environmental image at the current moment; wherein, a target bounding box includes a traffic light; The color of the traffic light in the target frame is determined, and the position of the target frame in the environmental image at the current moment is determined, which is the actual position of the traffic light in the environmental image.

4. The method according to claim 2, characterized in that, Before inputting the environmental image acquired at the current moment into the preset traffic light recognition model, the method further includes: Acquire pre-collected traffic light images; wherein the traffic light images are marked with the actual target frame of the traffic light, the actual center coordinates of the traffic light, and the actual color of the traffic light; The traffic light image is input into a pre-built traffic light recognition model, and the predicted color of the traffic light, the predicted center coordinates, and four distance parameters corresponding to the predicted center coordinates are output; wherein, the distance parameters are used to represent the distance between the center coordinates and the four borders of the target box; The predicted target box of the traffic light is determined based on the predicted center coordinates and the four distance parameters corresponding to the predicted center coordinates. If the predicted target box and the actual target box are in the same position, and the predicted color and the actual color are the same, then the traffic light recognition model training is complete.

5. The method according to claim 1, characterized in that, Obtain the actual position of the traffic light in the environmental image captured in the previous moment, including: If the traffic light is not present in the environmental image acquired in the previous moment, then obtain the actual location of the traffic light in at least one environmental image prior to the previous moment.

6. A vehicle control device based on traffic lights, characterized in that, The device is applied to a vehicle, and the device includes: An environmental image acquisition module is used to acquire an environmental image collected at the current moment; wherein, the environmental image is used to represent the environment in front of the vehicle, and the environmental image includes at least two traffic lights; The status information determination module is used to determine and store the status information of the traffic light from the environmental image acquired at the current time; wherein, the status information includes the color of the traffic light and the actual position of the traffic light in the environmental image, and the actual position is obtained by segmenting the color in the environmental image and extracting the basic geometric features of the traffic light; The predicted position determination module is used to acquire the actual position of each traffic light in the environmental image acquired at the previous moment, as well as the tracking identifier assigned to each traffic light, and to acquire the current driving information of the vehicle; wherein, the current driving information includes driving speed, driving acceleration, and driving trajectory; based on the actual position of the traffic light in the environmental image acquired at the previous moment and the current driving information of the vehicle, the trajectory of the traffic light is predicted to obtain the predicted position of the traffic light in the environmental image acquired at the current moment; The vehicle control module compares the predicted positions of each traffic light in the environmental image acquired at the previous moment with the actual positions of each traffic light in the environmental image acquired at the current moment. If the deviation between the actual position of a traffic light in the environmental image acquired at the current moment and the predicted position of any traffic light in the environmental image acquired at the previous moment is less than a preset offset, then the traffic light is determined to be a target light, and the tracking identifier of the traffic light corresponding to the target light in the environmental image acquired at the previous moment is assigned to the target light. The vehicle is controlled to move according to the color of the target light in the environmental image acquired at the current moment.

7. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the traffic light-based vehicle control method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the traffic light-based vehicle control method as described in any one of claims 1-5.

9. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the traffic light-based vehicle control method as described in any one of claims 1-5.

Images