Lane and traffic light association method and device, storage medium and electronic device

Abstract

The invention provides a lane and traffic light association method and device, a storage medium and an electronic device, and the method comprises the steps: recognizing the traffic light information and lane information of a vehicle at a current traffic intersection; filtering traffic lights not at the current traffic intersection according to the traffic light information and the lane information to obtain a target traffic light group matched with the current traffic intersection; and associating each lane of the current traffic intersection to the matched traffic light in the target traffic light group. According to the embodiment of the invention, traffic lights of non-current vehicle driving roads and non-current intersections can be effectively filtered, and the technical problem of inaccurate association between traffic lights and lanes in the related technology is solved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a method and device for associating lanes and traffic lights, a storage medium, and an electronic device. Background Art

[0002] Among the related technologies, urban autonomous driving refers to the application of autonomous driving technology in urban environments. The importance and significance of urban autonomous driving are reflected in many aspects, from technological innovation to social impact to economic benefits, and its application prospects are very broad. However, compared with autonomous driving on highways and urban expressways, urban autonomous driving faces more and more complex challenges. There are usually more complex and dynamic traffic conditions in urban environments, including complex road types, numerous and different types of traffic participants, complex and frequent traffic lights and traffic signs, traffic congestion and building obstructions, as well as other uncertain factors, which result in the vehicle being unable to effectively identify and filter traffic lights that are not at the current traffic intersection, and thus cannot ensure that the vehicle can comply with traffic rules and safely drive through the intersection.

[0003] With respect to the above-mentioned problems existing in the related technologies, no efficient and accurate solutions have been found yet. Summary of the invention

[0004] The present invention provides a lane and traffic light association method and device, a storage medium, and an electronic device to solve the technical problems in the related art.

[0005] According to one embodiment of the present invention, a method for associating lanes and traffic lights is provided, comprising: identifying traffic light information and lane information of a vehicle at a current traffic intersection; filtering traffic lights at non-current traffic intersections according to the traffic light information and the lane information to obtain a target traffic light group that matches the current traffic intersection; and associating each lane of the current traffic intersection to a matching traffic light in the target traffic light group.

[0006] Optionally, identifying traffic light information and lane information of a vehicle at a current traffic intersection includes: acquiring a visual image collected by the vehicle at the current traffic intersection; establishing a vehicle coordinate system with the center point of the vehicle as the origin in the visual image; abstracting each traffic light as a point and each lane as a lane centerline in the visual image, and acquiring the point coordinates of each traffic light and the point cloud coordinates of each lane centerline in the vehicle coordinate system; configuring first attribute information of each traffic light, and configuring second attribute information of each lane; determining the first attribute information and point coordinates of all traffic lights as traffic light information, and determining the point cloud coordinates and second attribute information of all lanes as lane information.

[0007] Optionally, configuring the first attribute information of each traffic light includes: for each traffic light, configuring the following first attribute information: traffic light ID, control type, wherein the control type is used to indicate the lane type and travel direction controlled by the traffic light.

[0008] Optionally, configuring the second attribute information of each lane includes: for each lane, configuring the following second attribute information: lane ID, traffic type, wherein the traffic type is used to indicate the lane type and the traffic direction of lane guidance.

[0009] Optionally, traffic lights at non-current traffic intersections are filtered according to the traffic light information and the lane information to obtain a target traffic light group that matches the current traffic intersection, including: filtering traffic lights for non-motorized vehicle lanes and sidewalks from all traffic light sets to obtain a first traffic light set for controlling motor vehicle lanes; clustering the first traffic light set according to spatial position to obtain a plurality of traffic light groups, wherein each traffic light group corresponds to a road segment, and a road segment includes a plurality of lanes in the same direction; calculating the centroid position of each traffic light group; filtering traffic light groups at non-current traffic intersections from the plurality of traffic light groups according to the centroid position and the lane information to obtain a target traffic light group that matches the current traffic intersection.

[0010] Optionally, traffic light groups that are not at the current traffic intersection are filtered out from the several traffic light groups according to the centroid position and the lane information to obtain a target traffic light group that matches the current traffic intersection, including: for each traffic light group in the several traffic light groups, calculating the average distance from the centroid position to the center lines of all lanes, wherein the lane information includes the lane center lines; selecting a set of traffic light groups whose average distance is less than a first preset distance; for each traffic light group in the set of traffic light groups, calculating the distance from the centroid position to the vehicle; and selecting a target traffic light group with the smallest distance from the set of traffic light groups.

[0011] Optionally, calculating the average distance from the centroid position to the center lines of all lanes includes: calculating the lateral distances from all lanes to the vehicle, and calculating the longitudinal distances from all lanes to the vehicle; selecting several adjacent lanes from all lanes whose lateral distances are less than a first threshold and whose longitudinal distances are less than a second threshold; for each adjacent lane, selecting two lane points close to the stop line in the center line of the lane; calculating the minimum distance from the centroid position to the straight line where the two lane points are located; and calculating the average of the minimum distances of all adjacent lanes to obtain an average distance.

[0012] Optionally, binding each traffic light of the target traffic light group to a matching lane includes: classifying all lanes according to traffic type, and sorting the classified lane classes in the longitudinal direction to obtain a first sequence, classifying the target traffic light group according to control type, and sorting the grouped traffic light classes in the longitudinal direction to obtain a second sequence; and using the first sequence and the second sequence to associate each lane of the current traffic intersection with a matching traffic light in the target traffic light group.

[0013] Optionally, using the first sequence and the second sequence to associate each lane of the current traffic intersection to a matching traffic light in the target traffic light group includes: for each lane class in the first sequence, selecting a traffic light class with the same element position in the second sequence; determining whether the number of elements in the second sequence is equal to the number of elements in the first sequence; if the number of elements in the second sequence is equal to the number of elements in the first sequence, binding the lane ID of each lane class in the first sequence to the traffic light ID of the traffic light class with the same element position.

[0014] Optionally, after determining whether the number of elements in the second sequence is equal to the number of elements in the first sequence, the method further includes: if the number of elements in the second sequence is not equal to the number of elements in the first sequence, calculating the distance between each lane in the first sequence and each traffic light in the second sequence; and binding the lane ID of each lane in the first sequence to the traffic light ID of the target traffic light with the smallest distance.

[0015] Optionally, after associating each lane of the current traffic intersection to a matching traffic light in the target traffic light group, the method further includes: determining the target lane in which the vehicle is currently located; searching for a target traffic light corresponding to the target lane; and generating a passage instruction for the vehicle at the current traffic intersection according to the color of the target traffic light, wherein the passage instruction is used to indicate whether the vehicle is allowed to pass through the current traffic intersection.

[0016] According to another embodiment of the present invention, a lane and traffic light association device is provided, comprising: an identification module for identifying traffic light information and lane information of a vehicle at a current traffic intersection; a filtering module for filtering traffic lights at non-current traffic intersections according to the traffic light information and the lane information to obtain a target traffic light group that matches the current traffic intersection; and an association module for associating each lane of the current traffic intersection to a matching traffic light in the target traffic light group.

[0017] Optionally, the recognition module includes: a first acquisition unit, used to acquire a visual image collected by the vehicle at the current traffic intersection; a construction unit, used to establish a vehicle body coordinate system with the center point of the vehicle as the origin in the visual image; a first acquisition unit, used to abstract each traffic light as a point and each lane as a lane centerline in the visual image, and acquire the point coordinates of each traffic light and the point cloud coordinates of each lane centerline in the vehicle body coordinate system; a configuration unit, used to configure the first attribute information of each traffic light and the second attribute information of each lane; a determination unit, used to determine the first attribute information and point coordinates of all traffic lights as traffic light information, and determine the point cloud coordinates and second attribute information of all lanes as lane information.

[0018] Optionally, the configuration unit includes: a first configuration subunit, used to configure the following first attribute information for each traffic light: traffic light ID, control type, wherein the control type is used to indicate the lane type and travel direction controlled by the traffic light.

[0019] Optionally, the configuration unit includes: a second configuration subunit, used to configure the following second attribute information for each lane: lane ID, traffic type, wherein the traffic type is used to indicate the lane type and the traffic direction of lane guidance.

[0020] Optionally, the association module includes: a first filtering unit, used to filter the traffic lights for non-motorized lanes and sidewalks from all traffic light sets to obtain a first traffic light set for controlling motor vehicle lanes; a clustering unit, used to cluster the first traffic light set according to spatial position to obtain a plurality of traffic light groups, wherein each traffic light group corresponds to a road segment, and a road segment includes a plurality of lanes in the same direction; a calculation unit, used to calculate the centroid position of each traffic light group; a second filtering unit, used to filter the traffic light groups for non-current traffic intersections from the plurality of traffic light groups according to the centroid position and the lane information to obtain a target traffic light group that matches the current traffic intersection.

[0021] Optionally, the second filtering unit includes: a first calculation subunit, used to calculate, for each traffic light group in the several traffic light groups, an average distance from the center of mass position to all lane center lines, wherein the lane information includes the lane center lines; a first selection subunit, used to select a set of traffic light groups whose average distance is less than a first preset distance; a second calculation subunit, used to calculate, for each traffic light group in the set of traffic light groups, the distance from the center of mass position to the vehicle; and a second selection subunit, used to select a target traffic light group with the smallest distance in the set of traffic light groups.

[0022] Optionally, the first calculation subunit is also used to: calculate the lateral distance from all lanes to the vehicle, and calculate the longitudinal distance from all lanes to the vehicle; select several adjacent lanes from all lanes whose lateral distance is less than a first threshold and whose longitudinal distance is less than a second threshold; for each adjacent lane, select two lane points close to the stop line in the center line of the lane; calculate the minimum distance from the center of mass position to the straight line where the two lane points are located; calculate the average of the minimum distances of all adjacent lanes to obtain an average distance.

[0023] Optionally, the association module includes: a classification unit, used to classify all lanes according to traffic type, and sort the classified lane classes in the longitudinal direction to obtain a first sequence, classify the target traffic light group according to control type, and sort the grouped traffic light classes in the longitudinal direction to obtain a second sequence; an association unit, used to associate each lane of the current traffic intersection with a matching traffic light in the target traffic light group using the first sequence and the second sequence.

[0024] Optionally, the association unit includes: a selection subunit, used to select a traffic light class with the same element position in the second sequence for each lane class in the first sequence; a judgment subunit, used to judge whether the number of elements in the second sequence is equal to the number of elements in the first sequence; and a first association subunit, used to bind the lane ID of each lane class in the first sequence to the traffic light ID of the traffic light class with the same element position if the number of elements in the second sequence is equal to the number of elements in the first sequence.

[0025] Optionally, the association unit also includes: a calculation subunit, which is used to calculate the distance between each lane in the first sequence and each traffic light in the second sequence after the judgment subunit determines whether the number of elements in the second sequence is equal to the number of elements in the first sequence. If the number of elements in the second sequence is not equal to the number of elements in the first sequence, a second association subunit is used to bind the lane ID of each lane in the first sequence to the traffic light ID of the target traffic light with the smallest distance.

[0026] Optionally, the device also includes: an association module, used to determine the target lane where the vehicle is currently located after the association module associates each lane of the current traffic intersection with a matching traffic light in the target traffic light group; a search module, used to search for a target traffic light corresponding to the target lane; and a generation module, used to generate a passage instruction for the vehicle at the current traffic intersection according to the color of the target traffic light, wherein the passage instruction is used to indicate whether the vehicle is allowed to pass through the current traffic intersection.

[0027] According to another aspect of an embodiment of the present application, a storage medium is further provided, which includes a stored program, and the above steps are executed when the program is run.

[0028] According to another aspect of an embodiment of the present application, there is also provided an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; wherein: the memory is used to store computer programs; and the processor is used to execute the steps in the above method by running the program stored in the memory.

[0029] The embodiment of the present application also provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute the steps in the above method.

[0030] Beneficial effects of the present invention:

[0031] 1. It does not rely on IoT devices in the environment and can effectively filter traffic lights on roads that are not the current vehicle driving on and at intersections, effectively solving the problem of the association between traffic lights and lanes at intersections;

[0032] 2. It provides the necessary prerequisite for autonomous vehicles to pass through intersections safely and efficiently while complying with traffic rules, thereby ensuring that vehicles can comply with traffic rules and pass through intersections safely and autonomously. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0034] Figure 1 is a hardware structure block diagram of a car according to an embodiment of the present invention;

[0035] Figure 2 is a flow chart of a method for associating lanes and traffic lights according to an embodiment of the present invention;

[0036] Figure 3 is a schematic diagram of a traffic intersection in an embodiment of the present invention;

[0037] Figure 4 is a schematic diagram of a vehicle body coordinate system in an embodiment of the present invention;

[0038] Figure 5 is a flow chart of traffic light filtering in an embodiment of the present invention;

[0039] Figure 6 A schematic diagram of a lane and a traffic light provided by an embodiment of the present invention;

[0040] Figure 7 A schematic diagram of another lane and traffic light provided by an embodiment of the present invention;

[0041] Figure 8 A schematic diagram of another lane and traffic light provided by an embodiment of the present invention;

[0042] Fig. 9 A flow chart of the association between traffic lights and lanes in an embodiment of the present invention;

[0043] Fig.10 4 is a structural block diagram of a lane and traffic light association device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only embodiments of a part of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present application. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0045] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0046] Example 1

[0047] The method embodiment provided in the first embodiment of the present application can be executed in a car, a server, a processor, an automatic driving / assisted driving / intelligent driving controller or a similar processing device. Taking running on a car as an example, Figure 1 1 is a hardware structure diagram of a car according to an embodiment of the present invention. Figure 1 As shown, a car may include one or more ( Figure 1Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the above-mentioned automobile may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the above-mentioned automobile. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0048] The memory 104 can be used to store automobile programs, for example, software programs and modules of application software, such as an automobile program corresponding to a method for associating lanes and traffic lights of an automobile in an embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the automobile program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the automobile via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0049] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the car. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0050] In this embodiment, a method for associating lanes and traffic lights is provided. Figure 2 is a flow chart of a method for associating lanes and traffic lights according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0051] Step S202, identifying traffic light information and lane information of the vehicle at the current traffic intersection;

[0052] The traffic light of this embodiment is also called a traffic signal light, which indicates the traffic signal for vehicles at the intersection through color (red, yellow, and green) signals.

[0053] Figure 3 It is a schematic diagram of a traffic intersection in an embodiment of the present invention. The traffic intersection is a crossroads. Each intersection is painted with zebra crossings, stop lines, lane arrows, lane lines, and each intersection is equipped with traffic lights.

[0054] Step S204, filtering traffic lights other than the current traffic intersection according to the traffic light information and the lane information to obtain a target traffic light group matching the current traffic intersection;

[0055] In this embodiment, the target traffic light group consists of one or more traffic lights.

[0056] Step S206: Associating each lane of the current traffic intersection with a matching traffic light in the target traffic light group.

[0057] Optionally, the lane and the traffic light are associated before the stop line of the lane. For each lane with a clear traffic type that the current vehicle can see, the correct traffic light association needs to be performed to ensure that the vehicle can change lanes and pass through intersections with traffic lights.

[0058] In the present invention, a lane can be associated with multiple traffic lights. For example, if the lane traffic type is straight and left turn, the lane can be associated with a left turn light and a straight light. The specific traffic light to be used needs to be determined by the downstream module (such as the automatic driving module) according to the target direction of the vehicle. Multiple lanes can be associated with the same traffic light. For example, when there is only one traffic light controlling straight driving, both the straight driving lane and the straight left turn lane can be associated with the traffic light.

[0059] Through the above steps, the traffic light information and lane information of the vehicle at the current traffic intersection are identified; the traffic lights at non-current traffic intersections are filtered according to the traffic light information and the lane information to obtain a target traffic light group matching the current traffic intersection; each lane at the current traffic intersection is associated with a matching traffic light in the target traffic light group. By identifying the traffic light information and lane information of the current traffic intersection, traffic lights at non-current vehicle driving roads and non-current intersections can be effectively filtered, thereby solving the technical problem of inaccurate association between traffic lights and lanes in related technologies.

[0060] In one implementation of the present embodiment, identifying traffic light information and lane information of a vehicle at a current traffic intersection includes: acquiring a visual image collected by the vehicle at the current traffic intersection; establishing a vehicle coordinate system with the center point of the vehicle as the origin in the visual image; abstracting each traffic light as a point and each lane as a lane centerline in the visual image, and acquiring the point coordinates of each traffic light and the point cloud coordinates of the center line of each lane in the vehicle coordinate system; configuring the first attribute information of each traffic light, and configuring the second attribute information of each lane; determining the first attribute information and point coordinates of all traffic lights as traffic light information, and determining the point cloud coordinates and second attribute information of all lanes as lane information.

[0061] In this embodiment, in addition to determining the traffic light information based on the visual image captured by the vehicle's camera, the vehicle can be further combined with traffic information transmitted by roadside equipment (such as traffic light poles) or the city traffic police system to fuse the final traffic light information. The lane information can also be based on the lane information in the map information of the current location, and fused with the visual image captured by the vehicle's camera to obtain the final accurate lane information.

[0062] In this embodiment, the positions of traffic lights and lanes are all represented by a three-dimensional Cartesian coordinate system. Each traffic light position is represented as a point in three-dimensional space. Each lane is abstracted as a lane centerline. A lane centerline is represented by continuous points. The origin of the coordinate system (0, 0, 0) is the center of the rear axle of the vehicle, forming a vehicle body coordinate system. The X direction is the vehicle's forward direction, the Y direction is perpendicular to the X direction and points to the left, that is, the left is positive and the right is negative, and the Z direction is perpendicular to the ground and points to the sky. Figure 4 Schematic diagram of the vehicle body coordinate system in an embodiment of the present invention.

[0063] The perception information acquired by the vehicle in this embodiment is collected by visual sensors. The visual sensors may include two forward-looking cameras, four surround-looking cameras, and one rear-looking camera. After the input image is processed by the model and other algorithm modules, the traffic light input (the point coordinates of the traffic light) and the lane input (the point cloud coordinates of the center line of the lane) in the vehicle coordinate system are obtained.

[0064] In one example, configuring the first attribute information of each traffic light includes: for each traffic light, configuring the following first attribute information: traffic light ID, control type, wherein the control type is used to indicate the lane type and travel direction controlled by the traffic light.

[0065] In this embodiment, the traffic light has attribute information such as ID, position, direction, control type and color. Among them, ID is the unique digital identifier of a traffic light in the current frame input, because there can be multiple traffic light groups in one frame input; the three-dimensional position is the spatial coordinate in the vehicle body coordinate system, marked with XYZ; the attribute of the control type refers to the direction or shape of traffic controlled by the traffic light. For example, in reality, traffic lights have round lights, which generally control traffic behaviors such as straight driving, left turn, and U-turn at simple intersections. At complex intersections, each light in a group of traffic lights independently controls straight driving, left turn, U-turn, or right turn. In the present invention, the control type of traffic lights is defined as follows: round lights, straight driving lights, left turn lights, right turn lights, U-turn lights, pedestrian lights, non-motor vehicle lights, timing lights, left turn and U-turn lights, straight driving left turn lights, straight driving right turn lights, no passing lights, and allowed passing lights. Color attributes: green, red, and yellow.

[0066] In one example, configuring the second attribute information of each lane includes: for each lane, configuring the following second attribute information: lane ID, traffic type, wherein the traffic type is used to indicate the lane type and the traffic direction of lane guidance.

[0067] The lane in this embodiment is represented by the lane centerline, which is composed of a point cloud. The lane centerline has attributes such as ID, position, and traffic type. Its ID is a unique digital identifier for each lane in a group of lanes. The position is the coordinate of the lane centerline point cloud. The lane types indicated by the traffic type include motor vehicle lanes, pedestrian lanes, and non-motor vehicle lanes. The lane guidance directions include the following: straight, right turn, right turn and straight, left turn, left turn and straight, left turn and right turn, U-turn, straight U-turn, left turn and U-turn, straight left turn and right turn, straight left turn and U-turn, U-turn and right turn, left merging, right merging, no left turn, no right turn, no U-turn, no straight, no merging to the left, and no merging to the right.

[0068] In one implementation of the present embodiment, traffic lights at non-current traffic intersections are filtered according to the traffic light information and the lane information to obtain a target traffic light group that matches the current traffic intersection, including: filtering traffic lights for non-motorized vehicle lanes and sidewalks from all traffic light sets to obtain a first traffic light set for controlling motor vehicle lanes; clustering the first traffic light set according to spatial position to obtain a plurality of traffic light groups, wherein each traffic light group corresponds to a road segment, and a road segment includes a plurality of lanes in the same direction; calculating the centroid position of each traffic light group; filtering traffic light groups at non-current traffic intersections from the plurality of traffic light groups according to the centroid position and the lane information to obtain a target traffic light group that matches the current traffic intersection.

[0069] The center of mass position of this embodiment is the center position of the mass distribution of the traffic light group, and the center of mass position is a three-dimensional coordinate point in the vehicle body coordinate system. Optionally, the center of mass position can also be the center position of the geometric shape of the traffic light group, or the feature point position of the key feature of the traffic light group (such as the round light in the traffic light group).

[0070] Since the main body vehicle of the autonomous driving has a wide field of view, the detected traffic lights may include traffic lights on roads in other directions controlled by the intersection, and traffic lights at the next intersection may be detected. These traffic lights should not be associated with the road in the direction of the vehicle's travel and need to be checked and filtered. The filtering of this embodiment includes: pre-filtering the lights that control non-motor vehicles or sidewalks. For those that control vehicles, firstly cluster all traffic lights based on spatial position, group the traffic lights, and each group of lights controls the lanes in a certain road direction; for the clustering grouping results, calculate the centroid coordinates of each group of lights. Then filter the lights in the Y direction (vehicle travel direction) of the intersection, that is, the lights on the transverse road, calculate the average distance from the centroid coordinates to the center lines of multiple lanes, that is, the distance from the point to the straight line, and obtain a group of lights with the smallest distance. For other light groups, if the distance is more than five meters greater than the minimum distance, filter them out. For light groups within five meters greater than the minimum distance, retain them, and it can be considered that it is the light of the next intersection in the X direction. Next, the lights in the X direction (the direction perpendicular to the vehicle's travel direction) are filtered, and the light group with the smallest distance from the light group to the vehicle can be calculated and selected, and finally a set of target traffic light groups is obtained.

[0071] In one example, traffic light groups that are not at a current traffic intersection are filtered out from the several traffic light groups according to the centroid position and the lane information to obtain a target traffic light group that matches the current traffic intersection, including: for each traffic light group in the several traffic light groups, calculating an average distance from the centroid position to center lines of all lanes, wherein the lane information includes the lane center lines; selecting a set of traffic light groups whose average distance is less than a first preset distance; for each traffic light group in the set of traffic light groups, calculating a distance from the centroid position to the vehicle; and selecting a target traffic light group with a smallest distance from the set of traffic light groups.

[0072] Optionally, when filtering horizontal traffic light groups, in addition to calculating the average distance from the center of mass position to the center lines of all lanes, you can also filter by the direction of the traffic light groups, retaining the set of traffic light groups opposite to the front of the vehicle, and filtering the traffic light groups that are in the same direction and perpendicular to the front of the vehicle.

[0073] Figure 5 : is a flow chart of traffic light filtering in an embodiment of the present invention, including the following steps:

[0074] If there are multiple traffic lights, perform clustering based on the spatial position according to the coordinates of the traffic lights to obtain different groups; calculate the centroid coordinates of each group of traffic lights, which can represent the average position of the group of traffic lights; select valid lanes on the road where the vehicle is traveling, and select multiple lanes to represent the road; the distance in the Y direction is less than 20 meters, and the distance in the X direction is less than 60 meters; calculate the straight line of each lane, and select two points in the center line of the lane that are closer to the stop line to represent the straight line of the lane; calculate the distance from the centroid coordinate of the traffic light to each lane, and finally calculate the average distance; according to the average distance from the traffic light group to the lane, filter the lights in the Y direction to obtain the traffic lights of the current road section; according to the obtained group of lights with the smallest distance, for other light groups, if the distance is more than five meters greater than the minimum distance, filter them out, and for the light groups within five meters greater than the minimum distance, retain them, and it can be considered that it is the light of the next intersection in the X direction; filter the lights in the X direction to obtain the traffic lights of the current traffic intersection. At present, the light group with the smallest distance from the centroid of the light group to the vehicle can be directly selected; obtain the best group of traffic lights (target traffic light group).

[0075] Optionally, calculating the average distance from the centroid position to the center lines of all lanes includes: calculating the lateral distances from all lanes to the vehicle, and calculating the longitudinal distances from all lanes to the vehicle; selecting several adjacent lanes from all lanes whose lateral distances are less than a first threshold and whose longitudinal distances are less than a second threshold; for each adjacent lane, selecting two lane points close to the stop line in the center line of the lane; calculating the minimum distance from the centroid position to the straight line where the two lane points are located; and calculating the average of the minimum distances of all adjacent lanes to obtain an average distance.

[0076] In the above-mentioned process of filtering traffic lights, it involves selecting multiple lanes for calculating the average distance from traffic lights to lanes. Based on the vehicle body coordinate system, the vehicle travel direction is the X direction, and the direction perpendicular to the vehicle travel direction is the Y direction. According to the road width, lanes with a lateral distance of less than 20 meters (the first threshold) in the Y direction can be selected, and lanes with a longitudinal distance of less than 60 meters (the second threshold) in the X direction can be selected, thereby obtaining multiple lanes. For each lane, a straight line equation needs to be constructed. We select two lane points in the center line of the lane that are closer to the stop line, and use the two lane points to fit and generate a straight line to represent the straight line of the lane. The minimum projection distance (vertical distance) from the centroid position of each traffic light group to the straight line is calculated, thereby supporting the calculation of the distance from the light to the center line of the lane. The reason for selecting two points closer to the stop line is that the road may be curved within a certain distance before the intersection.

[0077] In one implementation of the present embodiment, binding each traffic light of the target traffic light group to a matching lane includes: classifying all lanes according to traffic type, and sorting the classified lane classes in the longitudinal direction to obtain a first sequence, classifying the target traffic light group according to control type, and sorting the grouped traffic light classes in the longitudinal direction to obtain a second sequence; and using the first sequence and the second sequence to associate each lane of the current traffic intersection to a matching traffic light in the target traffic light group.

[0078] After obtaining a set of target traffic light groups, the control range and function of the light group are determined based on the traffic light information, and then the control type of each traffic light in the target traffic light group is classified, and the lane traffic type is classified according to the guide line on the lane. According to traffic laws and management regulations, in the case of only one round light, the light is set to control all lanes in the intersection except the right turn, that is, the control range of the round light is expanded to U-turn, left turn, and straight ahead; if there is no U-turn light in a set of lights, the U-turn light is expressed by the left turn light to control the U-turn, that is, the control range of the left turn light is expanded to left turn U-turn, if there is no left turn light, the straight-ahead light is used to express the control of the U-turn and also express the control of the left turn, that is, the straight-ahead light expands the control range to U-turn, left turn, and straight ahead. Figure 6 A schematic diagram of a lane and a traffic light provided by an embodiment of the present invention, wherein the target traffic light group has only circular lights; Figure 7 A schematic diagram of another lane and traffic lights provided by an embodiment of the present invention, wherein the target traffic light group includes traffic lights for U-turn, left turn, right turn, and straight ahead; Figure 8 A schematic diagram of another lane and traffic lights provided in an embodiment of the present invention, wherein the target traffic light group includes left turn lights and straight ahead lights.

[0079] At some intersections, there is no right-turn traffic light, so a traffic light for controlling right turns is added to the target traffic light group, and the signal color is always green.

[0080] In one example, using the first sequence and the second sequence to associate each lane of the current traffic intersection to a matching traffic light in the target traffic light group includes: for each lane class in the first sequence, selecting a traffic light class with the same element position in the second sequence; determining whether the number of elements in the second sequence is equal to the number of elements in the first sequence; if the number of elements in the second sequence is equal to the number of elements in the first sequence, binding the lane ID of each lane class in the first sequence to the traffic light ID of the traffic light class with the same element position.

[0081] Lanes of the same type are grouped together and sorted by the Y value to ensure that the spatial position is consistent with the actual situation. The same process is applied to traffic lights, that is, traffic lights of the same type are grouped together and sorted by the Y value to ensure that the spatial position is consistent with the actual situation.

[0082] In one implementation scenario of this example, the first sequence includes three elements: left-turn lane 1; through lane 2, through lane 3; right-turn lane 4. Lane 2 and lane 3 are of the same lane class. The second sequence also includes three elements: left-turn traffic light 1; through traffic light 2; right-turn traffic light 3. When matching and binding, left-turn lane 1 is bound to left-turn traffic light 1, through lane 2 and through lane 3 are both bound to through traffic light 2, and right-turn lane 4 is bound to right-turn traffic light 3.

[0083] In another example, after determining whether the number of elements in the second sequence is equal to the number of elements in the first sequence, the method further includes: if the number of elements in the second sequence is not equal to the number of elements in the first sequence, calculating the distance between each lane in the first sequence and each traffic light in the second sequence; and binding the lane ID of each lane in the first sequence to the traffic light ID of the target traffic light with the smallest distance.

[0084] In an implementation scenario of this example, the number of elements in the second sequence is not equal to the number of elements in the first sequence. For example, the first sequence includes three elements: left turn lane 1; straight lane 2, straight lane 3; right turn lane 4. Lane 2 and lane 3 are of the same lane class. The second sequence includes four elements: left turn traffic light 1; straight left turn traffic light 2; straight traffic light 3; right turn traffic light 4. Then, association is performed based on distance, and the target traffic light with the smallest distance from the current lane is bound to the current lane.

[0085] Fig. 9 The flowchart of the association between traffic lights and lanes in an embodiment of the present invention includes: setting the control range of each light according to the number and type of traffic signal groups; grouping lane types and sorting them by spatial position; grouping traffic light types and sorting them by spatial position; traversing all lane groups and associating them with traffic lights: judging whether the types match; after the type matching is successful; judging whether the number of lanes is equal to the number of traffic lights; if the numbers are equal, associating them one by one according to the position relationship; if the numbers are not equal, associating each lane with the nearest light; and obtaining the association relationship between all lanes and traffic lights.

[0086] In an application scenario of the present embodiment, after associating each lane of the current traffic intersection to a matching traffic light in the target traffic light group, it also includes: determining the target lane where the vehicle is currently located; searching for a target traffic light corresponding to the target lane; and generating a passage instruction for the vehicle at the current traffic intersection according to the color of the target traffic light, wherein the passage instruction is used to indicate whether the vehicle is allowed to pass through the current traffic intersection.

[0087] The solution of this embodiment is adopted to meet the needs of urban autonomous driving and the shortcomings of the existing technology. The present invention provides a method for lane-associated traffic lights for autonomous driving, which uses real-time perception data collected by vehicle visual sensors, including traffic lights, lanes and lane attributes, to achieve automatic, efficient and correct association between lanes and traffic lights. The present invention can effectively cope with various unstructured intersection scenarios, including simple intersections, complex intersections, roundabouts, etc., and can effectively filter traffic lights on non-current vehicle driving roads and non-current traffic intersections, thereby ensuring that vehicles can comply with traffic rules and safely drive through intersections.

[0088] The solution of this embodiment does not rely on the equipment in the environment, effectively solves the problem of the association between traffic lights and lanes at the intersection, and provides the necessary prerequisite for the automatic driving of vehicles to pass through the intersection safely and efficiently while complying with traffic regulations.

[0089] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0090] Example 2

[0091] In this embodiment, a lane and traffic light association device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0092] Fig.10is a structural block diagram of a lane and traffic light association device according to an embodiment of the present invention, such as Fig.10 As shown, the device comprises:

[0093] The recognition module 100 is used to recognize the traffic light information and lane information of the vehicle at the current traffic intersection;

[0094] A filtering module 102, configured to filter traffic lights other than the current traffic intersection according to the traffic light information and the lane information, to obtain a target traffic light group matching the current traffic intersection;

[0095] The associating module 104 is configured to associate each lane of the current traffic intersection with a matching traffic light in the target traffic light group.

[0096] Optionally, the recognition module includes: a first acquisition unit, used to acquire a visual image collected by the vehicle at the current traffic intersection; a construction unit, used to establish a vehicle body coordinate system with the center point of the vehicle as the origin in the visual image; a first acquisition unit, used to abstract each traffic light as a point and each lane as a lane centerline in the visual image, and acquire the point coordinates of each traffic light and the point cloud coordinates of each lane centerline in the vehicle body coordinate system; a configuration unit, used to configure the first attribute information of each traffic light and the second attribute information of each lane; a determination unit, used to determine the first attribute information and point coordinates of all traffic lights as traffic light information, and determine the point cloud coordinates and second attribute information of all lanes as lane information.

[0097] Optionally, the configuration unit includes: a first configuration subunit, used to configure the following first attribute information for each traffic light: traffic light ID, control type, wherein the control type is used to indicate the lane type and travel direction controlled by the traffic light.

[0098] Optionally, the configuration unit includes: a second configuration subunit, used to configure the following second attribute information for each lane: lane ID, traffic type, wherein the traffic type is used to indicate the lane type and the traffic direction of lane guidance.

[0099] Optionally, the association module includes: a first filtering unit, used to filter the traffic lights for non-motorized lanes and sidewalks from all traffic light sets to obtain a first traffic light set for controlling motor vehicle lanes; a clustering unit, used to cluster the first traffic light set according to spatial position to obtain a plurality of traffic light groups, wherein each traffic light group corresponds to a road segment, and a road segment includes a plurality of lanes in the same direction; a calculation unit, used to calculate the centroid position of each traffic light group; a second filtering unit, used to filter the traffic light groups for non-current traffic intersections from the plurality of traffic light groups according to the centroid position and the lane information to obtain a target traffic light group that matches the current traffic intersection.

[0100] Optionally, the second filtering unit includes: a first calculation subunit, used to calculate, for each traffic light group in the several traffic light groups, an average distance from the center of mass position to all lane center lines, wherein the lane information includes the lane center lines; a first selection subunit, used to select a set of traffic light groups whose average distance is less than a first preset distance; a second calculation subunit, used to calculate, for each traffic light group in the set of traffic light groups, the distance from the center of mass position to the vehicle; and a second selection subunit, used to select a target traffic light group with the smallest distance in the set of traffic light groups.

[0101] Optionally, the first calculation subunit is also used to: calculate the lateral distance from all lanes to the vehicle, and calculate the longitudinal distance from all lanes to the vehicle; select several adjacent lanes from all lanes whose lateral distance is less than a first threshold and whose longitudinal distance is less than a second threshold; for each adjacent lane, select two lane points close to the stop line in the center line of the lane; calculate the minimum distance from the center of mass position to the straight line where the two lane points are located; calculate the average of the minimum distances of all adjacent lanes to obtain an average distance.

[0102] Optionally, the association module includes: a classification unit, used to classify all lanes according to traffic type, and sort the classified lane classes in the longitudinal direction to obtain a first sequence, classify the target traffic light group according to control type, and sort the grouped traffic light classes in the longitudinal direction to obtain a second sequence; an association unit, used to associate each lane of the current traffic intersection with a matching traffic light in the target traffic light group using the first sequence and the second sequence.

[0103] Optionally, the association unit includes: a selection subunit, used to select a traffic light class with the same element position in the second sequence for each lane class in the first sequence; a judgment subunit, used to judge whether the number of elements in the second sequence is equal to the number of elements in the first sequence; and a first association subunit, used to bind the lane ID of each lane class in the first sequence to the traffic light ID of the traffic light class with the same element position if the number of elements in the second sequence is equal to the number of elements in the first sequence.

[0104] Optionally, the association unit also includes: a calculation subunit, which is used to calculate the distance between each lane in the first sequence and each traffic light in the second sequence after the judgment subunit determines whether the number of elements in the second sequence is equal to the number of elements in the first sequence. If the number of elements in the second sequence is not equal to the number of elements in the first sequence, a second association subunit is used to bind the lane ID of each lane in the first sequence to the traffic light ID of the target traffic light with the smallest distance.

[0105] Optionally, the device also includes: an association module, used to determine the target lane where the vehicle is currently located after the association module associates each lane of the current traffic intersection with a matching traffic light in the target traffic light group; a search module, used to search for a target traffic light corresponding to the target lane; and a generation module, used to generate a passage instruction for the vehicle at the current traffic intersection according to the color of the target traffic light, wherein the passage instruction is used to indicate whether the vehicle is allowed to pass through the current traffic intersection.

[0106] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0107] Example 3

[0108] An embodiment of the present invention further provides a storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0109] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0110] S1, identifying the traffic light information and lane information of the vehicle at the current traffic intersection;

[0111] S2, filtering traffic lights other than the current traffic intersection according to the traffic light information and the lane information to obtain a target traffic light group matching the current traffic intersection;

[0112] S3, associating each lane of the current traffic intersection with a matching traffic light in the target traffic light group.

[0113] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

[0114] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0115] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0116] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0117] S1, identifying the traffic light information and lane information of the vehicle at the current traffic intersection;

[0118] S2, filtering traffic lights other than the current traffic intersection according to the traffic light information and the lane information to obtain a target traffic light group matching the current traffic intersection;

[0119] S3, associating each lane of the current traffic intersection with a matching traffic light in the target traffic light group.

[0120] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0121] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0122] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0123] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0124] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for associating lanes and traffic lights, characterized in that: include: Identify traffic light information and lane information of the vehicle at the current traffic intersection; Filtering traffic lights other than the current traffic intersection according to the traffic light information and the lane information to obtain a target traffic light group matching the current traffic intersection; Each lane of the current traffic intersection is associated with a matching traffic light in the target traffic light group.

2. The method according to claim 1, characterized in that Identify the traffic light information and lane information of the vehicle at the current traffic intersection including: Acquire a visual image collected by the vehicle at a current traffic intersection; Establishing a vehicle body coordinate system in the visual image with the center point of the vehicle as the origin; Abstracting each traffic light as a point in the visual image and each lane as a lane centerline, and obtaining the point coordinates of each traffic light and the point cloud coordinates of each lane centerline in the vehicle body coordinate system; Configure first attribute information of each traffic light, and configure second attribute information of each lane; The first attribute information and point coordinates of all traffic lights are determined as traffic light information, and the point cloud coordinates and second attribute information of all lanes are determined as lane information.

3. The method according to claim 2, characterized in that The first attribute information for configuring each traffic light includes: For each traffic light, the following first attribute information is configured: traffic light ID, control type, wherein the control type is used to indicate the lane type and travel direction controlled by the traffic light.

4. The method according to claim 2, characterized in that: The second attribute information for configuring each lane includes: For each lane, the following second attribute information is configured: lane ID, traffic type, wherein the traffic type is used to indicate the lane type and the traffic direction of lane guidance.

5. The method according to claim 1, characterized in that Filtering traffic lights other than the current traffic intersection according to the traffic light information and the lane information to obtain a target traffic light group matching the current traffic intersection includes: Filtering the traffic lights of non-motorized vehicle lanes and sidewalks from all traffic light sets to obtain a first traffic light set for controlling motor vehicle lanes; Clustering the first traffic light set according to spatial positions to obtain a plurality of traffic light groups, wherein each traffic light group corresponds to a road segment, and a road segment includes a plurality of lanes in the same direction; Calculate the centroid position of each traffic light group; Traffic light groups other than the current traffic intersection are filtered out from the plurality of traffic light groups according to the centroid position and the lane information to obtain a target traffic light group matching the current traffic intersection.

6. The method according to claim 5, characterized in that Filtering traffic light groups other than the current traffic intersection from the plurality of traffic light groups according to the centroid position and the lane information to obtain a target traffic light group matching the current traffic intersection includes: For each of the plurality of traffic light groups, calculating an average distance from the centroid position to the center lines of all lanes, wherein the lane information includes the center lines of the lanes; Selecting a set of traffic light groups whose average distance is less than a first preset distance; For each traffic light group in the traffic light group set, calculating the distance from the centroid position to the vehicle; A target traffic light group with the smallest distance is selected from the traffic light group set.

7. The method according to claim 6, characterized in that Calculating the average distance from the centroid position to the center lines of all lanes includes: Calculating the lateral distances from all lanes to the vehicle, and calculating the longitudinal distances from all lanes to the vehicle; Selecting a plurality of adjacent lanes from among all lanes whose lateral distance is less than a first threshold and whose longitudinal distance is less than a second threshold; For each adjacent lane, select two lane points on the lane centerline close to the stop line; Calculate the minimum distance from the centroid position to the straight line where the two lane points are located; Calculate the average of the minimum distances of all adjacent lanes to get the average distance.

8. The method according to claim 1, characterized in that Binding each traffic light of the target traffic light group to a matching lane includes: Classifying all lanes according to traffic types, and sorting the classified lane types in a longitudinal direction to obtain a first sequence, and classifying the target traffic light group according to control types, and sorting the grouped traffic light types in a longitudinal direction to obtain a second sequence; Each lane of the current traffic intersection is associated with a matching traffic light in the target traffic light group using the first sequence and the second sequence.

9. The method according to claim 8, characterized in that Using the first sequence and the second sequence to associate each lane of the current traffic intersection with a matching traffic light in the target traffic light group includes: For each lane class in the first sequence, selecting a traffic light class with the same element position in the second sequence; Determining whether the number of elements in the second sequence is equal to the number of elements in the first sequence; If the number of elements in the second sequence is equal to the number of elements in the first sequence, the lane ID of each lane class in the first sequence is bound to the traffic light ID of the traffic light class with the same element position.

10. The method according to claim 9, characterized in that After determining whether the number of elements in the second sequence is equal to the number of elements in the first sequence, the method further includes: If the number of elements in the second sequence is not equal to the number of elements in the first sequence, calculating the distance between each lane in the first sequence and each traffic light in the second sequence; The lane ID of each lane in the first sequence is bound to the traffic light ID of the target traffic light with the shortest distance.

11. The method according to claim 8, characterized in that After associating each lane of the current traffic intersection with a matching traffic light in the target traffic light group, the method further includes: Determine a target lane where the vehicle is currently located; Finding a target traffic light corresponding to the target lane; A passing instruction for the vehicle at the current traffic intersection is generated according to the color of the target traffic light, wherein the passing instruction is used to indicate whether the vehicle is allowed to pass through the current traffic intersection.

12. A lane and traffic light association device, characterized in that: include: A recognition module, used to recognize the traffic light information and lane information of the vehicle at the current traffic intersection; A filtering module, used for filtering traffic lights other than the current traffic intersection according to the traffic light information and the lane information, to obtain a target traffic light group matching the current traffic intersection; An associating module is used to associate each lane of the current traffic intersection with a matching traffic light in the target traffic light group.

13. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 11 when executed.

14. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 11.

Images