U-shaped turning scene recognition method and related equipment thereof

Abstract

The invention discloses a U-shaped turning scene recognition method and related equipment thereof, and relates to the technical field of vehicles, and the U-shaped turning scene recognition method comprises the steps: obtaining historical scene fragments; when an intersection scene exists in the historical scene segment, a scene recognition result is determined based on each frame of data in the historical scene segment, and it is avoided that data of a non-key road section is mistakenly regarded as a U-shaped turning scene (such as a vehicle reversing scene); the scene recognition result can be that a left U-shaped turning scene exists in the historical scene segment, and / or a right U-shaped turning scene exists in the historical scene segment, and / or no U-shaped turning scene exists in the historical scene segment; according to the method, only the historical scene fragment needs to be acquired, and the scene recognition result about the U-shaped turning of the vehicle in the historical scene fragment is accurately obtained in the mode, so that the data about the U-shaped turning of the vehicle is simply acquired.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a U-turn scene recognition method and related equipment. Background Art

[0002] With the development of intelligent driving technology, the demand for data from real traffic scenarios is increasing. These data from real traffic scenarios can not only be used to train and verify autonomous driving algorithms, but also to improve vehicle perception, decision-making and control algorithms.

[0003] High-quality data from real-world traffic scenarios can significantly improve the accuracy of autonomous driving algorithms, vehicle perception, decision-making, and control algorithms. However, due to technical limitations involved in data collection and the relatively rare and complex nature of U-turns, obtaining data on U-turn scenarios is difficult. Summary of the Invention

[0004] The main purpose of this application is to provide a U-turn scene recognition method and related equipment, aiming to solve the technical problem of high difficulty in obtaining data on vehicle turning scenes.

[0005] To achieve the above objectives, the present application proposes a U-turn scene recognition method, which includes: Get historical scene clips; When there is a road intersection scene in the historical scene segment, determining a scene recognition result based on each frame data in the historical scene segment; The scene recognition result includes whether a left U-turn scene, a right U-turn scene, and / or no U-turn scene exists in the historical scene segment.

[0006] In one embodiment, the step of determining a scene recognition result based on each frame of data in the historical scene segment includes: Determine whether the vehicle corresponding to each frame between a target frame and a preset frame has a possibility of making a U-turn, wherein the target frame includes a start frame and the preset frame is separated from the target frame by a preset frame length; If so, determining a change in the vehicle's driving direction based on the vehicle heading angles corresponding to the target frame and the preset frame respectively; A scene recognition result is determined based on the change in the vehicle's driving direction.

[0007] In one embodiment, the step of determining whether the vehicle corresponding to each frame between the target frame and the preset frame has a possibility of making a U-turn includes: Determine whether the vehicle speed corresponding to the target frame data is less than a preset speed threshold; If the vehicle speed is greater than or equal to a preset speed threshold, then based on the vehicle position information corresponding to each frame, the vehicle travel distances corresponding to each two consecutive frames from the target frame to the preset frame are calculated in sequence; Sum up the travel distances of each vehicle to get the total travel distance; If the total driving distance is greater than or equal to the preset driving distance, it is determined that there is a possibility that the vehicle corresponding to each frame between the target frame and the preset frame has a U-turn.

[0008] In one embodiment, the step of determining a change in the vehicle's driving direction based on the vehicle heading angles corresponding to the target frame and the preset frame respectively includes: Calculating the angle difference between the vehicle heading angles corresponding to the target frame and the preset frame respectively; Based on the angle difference, a change in the vehicle's driving direction in the scene sub-segment corresponding to the target frame and the preset frame is determined.

[0009] In one embodiment, after the step of determining the change in the vehicle driving direction in the scene sub-segment corresponding to the target frame and the preset frame based on the angle difference, the method further includes: The target frame is updated to be a frame that slides backward by a preset number of frames, and the process returns to the step of determining whether the vehicle corresponding to each frame between the target frame and the preset frame has the possibility of making a U-turn, until the preset frame becomes the end frame, thereby obtaining the change in the vehicle's driving direction in the multiple scene sub-segments corresponding to the start frame and the end frame.

[0010] In one embodiment, the step of determining a change in the vehicle's driving direction in a scene sub-segment corresponding to a target frame and a preset frame based on the angle difference includes: performing normalization processing on the angle difference; If the normalized angle difference is within the first angle interval, determining that the change in the vehicle's driving direction is a left U-turn; If the normalized angle difference is within the second angle interval, determining that the change in the vehicle's driving direction is a right U-turn; If the normalized angle difference is within the third angle range, determining that the change in the vehicle's driving direction does not result in a U-turn; There is no intersection among the first angle interval, the second angle interval and the third angle interval.

[0011] In one embodiment, the step of determining a scene recognition result based on a change in the vehicle's driving direction includes: determining scene recognition results for the multiple scene sub-segments based on changes in the vehicle driving directions in the multiple scene sub-segments corresponding to the start frame and the end frame; After the step of determining the scene recognition result based on the change in the vehicle's driving direction, the method further includes: The U-turn moment of the vehicle is determined based on the target scene sub-segment where the vehicle U-turn scene exists.

[0012] In one embodiment, after the step of obtaining the historical scene fragments, the method further includes: Determine whether there is intersection identification information in each frame of data; If so, the target distance between the intersection and the vehicle is calculated based on the intersection identification information; When the target distance is less than a preset distance threshold, it is determined that an intersection scene exists in the historical scene segment.

[0013] In one embodiment, the intersection identification information includes traffic light information, the traffic light information including first coordinates of at least one traffic light in a global coordinate system, and the step of calculating the target distance between the intersection and the vehicle based on the intersection identification information includes: Converting the first coordinate into a second coordinate in a vehicle body coordinate system; Calculating a first longitudinal distance between at least one traffic light and the vehicle based on the second coordinate; determining, based on the first longitudinal distance, whether there is a target traffic light in front of the vehicle; If so, the minimum longitudinal distance among the first longitudinal distances corresponding to the target traffic light is used as the target distance between the intersection and the vehicle; The step of determining whether there is a target traffic light in front of the vehicle based on the first longitudinal distance includes: When the first longitudinal distance is greater than zero, it is determined that the corresponding target traffic light is in front of the vehicle.

[0014] In one embodiment, the intersection identification information further includes zebra crossing information, where the zebra crossing information includes third coordinates of all corner points of a rectangular box corresponding to at least one zebra crossing in a global coordinate system. The step of calculating the target distance between the intersection and the vehicle based on the intersection identification information includes: Converting the third coordinate into a fourth coordinate in the vehicle body coordinate system; Based on the fourth coordinate, calculating a second longitudinal distance between each corner point and the vehicle; Based on the second longitudinal distance, determining whether there is a target zebra crossing in front of or below the vehicle; If so, the minimum longitudinal distance among the second longitudinal distances corresponding to the target zebra crossing is used as the target distance between the intersection and the vehicle; The step of determining whether there is a target zebra crossing in front of or below the vehicle based on the second longitudinal distance includes: If a second longitudinal distance between at least one corner point of the target zebra crossing and the vehicle is greater than or equal to zero, determining that the target zebra crossing is in front of the vehicle or below the vehicle; If the second longitudinal distances between all corner points of the target zebra crossing and the vehicle are less than zero, it is determined that the target zebra crossing is behind the vehicle.

[0015] In addition, to achieve the above-mentioned purpose, the present application also proposes a U-turn scene recognition device, which includes: Acquisition module, used to obtain historical scene fragments; An identification module is used to determine a scene recognition result based on each frame data in the historical scene segment when there is an intersection scene in the historical scene segment, wherein the scene recognition result includes the presence of a left U-turn scene, a right U-turn scene, and / or the absence of a U-turn scene in the historical scene segment.

[0016] In addition, to achieve the above-mentioned purpose, the present application also proposes a U-turn scene recognition device, which includes: a memory, a processor, and a computer program stored on the memory and runnable on the processor, and the computer program is configured to implement the steps of the U-turn scene recognition method described above.

[0017] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the U-turn scene recognition method described above are implemented.

[0018] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the steps of the U-turn scene recognition method as described above.

[0019] One or more technical solutions proposed in this application have at least the following technical effects: The present application obtains historical scene fragments; when there is an intersection scene in the historical scene fragments, the scene recognition result is determined based on the data of each frame in the historical scene fragments to avoid mistaking the data of non-critical sections for U-turn scenes (for example, vehicle reversing scenes); the scene recognition result can be that there is a left U-turn scene in the historical scene fragment, and / or there is a right U-turn scene in the historical scene fragment, and / or there is no U-turn scene in the historical scene fragment; the present application only needs to obtain historical scene fragments, and accurately obtain the scene recognition results about the vehicle U-turn in the historical scene fragments through the above method, thereby simply obtaining data about the vehicle U-turn. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0022] Figure 1 A flowchart of the first embodiment of the U-turn scene recognition method of the present application is provided; Figure 2 A schematic diagram of a simplified flow chart of a U-turn scene recognition method provided in Example 1 of the present application; Figure 3 A schematic diagram of a scene provided in Example 1 of the U-turn scene recognition method of this application; Figure 4 A flowchart of the second embodiment of the U-turn scene recognition method provided in this application; Figure 5 A flowchart of the third embodiment of the U-turn scene recognition method provided in this application; Figure 6 This is a schematic diagram of the module structure of the U-turn scene recognition device according to an embodiment of the present application; Figure 7 This is a schematic diagram of the device structure of the hardware operating environment involved in the U-turn scene recognition method in the embodiment of the present application.

[0023] The purpose, features and advantages of this application will be further explained with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0024] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0025] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0026] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, mobile phone, etc., or an electronic device capable of implementing the above functions, such as a U-turn scene recognition device. The following uses the U-turn scene recognition device as an example to illustrate this embodiment and the following embodiments.

[0027] Based on this, the embodiment of the present application provides a U-turn scene recognition method, referring to Figure 1 , Figure 1 This is a flowchart of the first embodiment of the U-turn scene recognition method of the present application.

[0028] In this embodiment, the U-turn scene recognition method includes steps S10 to S20: Step S10, obtaining historical scene fragments; It should be noted that with the development of intelligent driving technology, the demand for data on real traffic scenarios is increasing; these real traffic scenario data can not only be used to train and verify autonomous driving algorithms, but also to improve vehicle perception, decision-making and control algorithms.

[0029] High-quality data from real-world traffic scenarios can significantly improve the accuracy of autonomous driving algorithms, vehicle perception, decision-making, and control algorithms. However, due to technical limitations involved in data collection and the relatively rare and complex nature of U-turns, obtaining data on U-turn scenarios is difficult.

[0030] In order to solve the above problems, this embodiment obtains historical scene fragments, aiming to offline identify whether there are U-turn scenes in the historical scene fragments, so as to use the historical scene fragments with scene recognition results as effective sample data for training and verifying autonomous driving algorithms, vehicle perception algorithms, driving decision algorithms and vehicle control algorithms.

[0031] Specifically, the historical scene fragment can be a historically collected vehicle driving record, or it can be vehicle driving information generated by simulation; the historical scene fragment includes multiple frames of data, and the scene information corresponding to each frame of data may include dynamic perception information collected by sensors such as cameras and radars, or static map information obtained by matching the historically collected vehicle latitude and longitude with a high-precision map; wherein, the dynamic perception information includes historically collected road images, timestamps, vehicle location information (for example, the vehicle's coordinates in the global coordinate system), vehicle speed and acceleration, vehicle direction and U-turn angle, vehicle's direction of travel, vehicle steering wheel angle, driver operation (for example, braking, throttle force, gear shifting, etc.), etc., and the static map information includes traffic sign information on the road, the location and behavior of other vehicles, pedestrians, obstacles, and other information.

[0032] Step S20, when there is an intersection scene in the historical scene segment, determine the scene recognition result based on each frame data in the historical scene segment, wherein the scene recognition result includes the presence of a left U-turn scene, a right U-turn scene, and / or the absence of a U-turn scene in the historical scene segment.

[0033] Since U-turn operations usually occur in intersection scenes, in order to improve the recognition efficiency of U-turn scenes for historical scene fragments, this embodiment will only continue to recognize U-turn scenes for historical scene fragments when it is determined that there is an intersection scene in the historical scene fragment; if there is no intersection scene in the historical scene fragment, the U-turn scene will not be recognized for the historical scene fragment, and the historical scene fragment will be abandoned as training data.

[0034] Specifically, the specific implementation method of this embodiment for determining the scene recognition result of the historical scene fragment based on each frame data in the historical scene fragment may be: determining the scene recognition result of the historical scene fragment based on the dynamic perception information and static map information corresponding to each frame data in the historical scene fragment.

[0035] Among them, the scene recognition results of the historical scene fragments include the presence of a left U-turn scene, a right U-turn scene, and / or the absence of a U-turn scene in the historical scene fragments. Therefore, the historical scene fragments with scene recognition results can be used as training data.

[0036] Since the historical scene fragment includes multiple frames of data, the scene recognition results of the historical scene fragment can also be the vehicle left U-turn scene, the vehicle right U-turn scene, and / or the non-U-turn scene in each frame of data; in order to increase the amount of data, each frame of data with scene recognition results can be used as training data.

[0037] For details, please refer to Figure 2 , the step S20 includes steps A1 to A3: Step A1, determining whether the vehicle corresponding to each frame between a target frame and a preset frame has a possibility of making a U-turn, wherein the target frame includes a start frame, and the preset frame is separated from the target frame by a preset frame length; Since the historical scene segment includes multiple frames of data, in order to improve data processing efficiency, it is possible to first determine whether there is a possibility of a U-turn in the corresponding vehicle of each frame between the target frame and the preset frame. If there is no possibility of a U-turn, further recognition of the scene sub-segment corresponding to the target frame to the preset frame can be skipped, thereby saving computing resources and time; if there is a possibility of a U-turn, scene recognition can be further performed on the scene sub-segment corresponding to the target frame to the preset frame.

[0038] Here, it is assumed that the total number of frames of multi-frame data in the historical scene segment is recorded as M, and the preset frame length (frame_size) is determined based on the time required for the vehicle to complete a U-turn operation. Specifically, the preset frame length can be 50 frames or 100 frames, etc. The target frame (frame_i frame) includes the starting frame and other frames, and the preset frame is separated from the target frame by a preset frame length; it can be understood that when the target frame is the 0th frame, the preset frame can be the 100th frame, and when the target frame is the 20th frame, the preset frame can be the 120th frame.

[0039] That is, it is determined whether the vehicle corresponding to each frame between the target frame and the preset frame has the possibility of making a U-turn.

[0040] The specific implementation of determining whether the vehicle corresponding to each frame between the target frame and the preset frame has a possibility of a U-turn may be: Determine whether the vehicle speed corresponding to the target frame data is less than a preset speed threshold; if the vehicle speed is greater than or equal to the preset speed threshold, then calculate the vehicle travel distances corresponding to each consecutive frame from the target frame to the preset frame based on the vehicle position information corresponding to each frame; sum the travel distances of each vehicle to obtain a total travel distance; if the total travel distance is greater than or equal to the preset travel distance, determine the possibility that the vehicle corresponding to each frame between the target frame and the preset frame has made a U-turn.

[0041] It should be noted that since a vehicle can only perform a U-turn operation when its speed meets certain conditions, this embodiment determines whether the vehicle speed corresponding to the target frame is less than a preset speed threshold (vel_static), thereby excluding the situation where the vehicle is stationary, avoiding further U-turn judgments, and reducing the waste of computing resources.

[0042] Among them, the preset speed threshold (distance_static) can be 0.5m / s or 0.7m / s, etc. Since the difficulty of a vehicle U-turn is greater than that of a vehicle steering, usually during driving, if a U-turn is desired, the vehicle speed will usually be reduced, and the speed at which the vehicle may make a U-turn is lower than the speed at which the vehicle may turn left or right. For example, if the speed corresponding to the 20th frame (target frame) is less than 0.5m / s, it is considered that the vehicle corresponding to the 20th frame is in a stationary condition, and no further scene recognition is performed on the frame, that is, the scene recognition of the 20th to 120th frames is skipped; if the speed corresponding to the 20th frame (target frame) is greater than or equal to 0.5m / s, it is considered that the vehicle has the possibility of making a U-turn or turning left or right.

[0043] Furthermore, if the vehicle speed corresponding to the target frame is greater than or equal to a preset speed threshold, based on the vehicle position information corresponding to each frame, the vehicle travel distances corresponding to each consecutive frame from the target frame to the preset frame are calculated in sequence, and the travel distances of each vehicle are summed to obtain the total travel distance (dis_sum); it can be understood that since the vehicle's travel direction may also change during the reversing process, in order to avoid misjudging the reversing scene as a U-turn scene during the scene recognition process, this embodiment needs to calculate the vehicle travel distances corresponding to each consecutive frame from the target frame to the preset frame, and then calculate the total travel distance, rather than calculating the difference in vehicle position between the target frame and the preset frame.

[0044] Specifically, if the scene from the target frame to the preset frame is a reversing scene (short distance movement (such as slightly adjusting the position backward)), the calculated total driving distance of the vehicle from the target frame to the preset frame is usually less than the total driving distance of the vehicle from the target frame to the preset frame in the U-turn scene. Figure 3 ; Therefore, in order to avoid the situation where the autonomous driving algorithm obtained through training and verification cannot accurately distinguish between reversing scenarios and U-turn scenarios, this embodiment needs to base its identification of U-turn scenarios on the total driving distance from the target frame to the preset frame.

[0045] In order to complete an effective U-turn, the vehicle not only travels a certain distance but also takes a certain amount of time. This embodiment calculates the vehicle travel distances between two consecutive frames from the target frame to the preset frame in sequence, while taking into account time and space factors to ensure that a coherent action is detected. For example, when the vehicle is reversing forward or backward, there may be parts of the driving route that overlap, that is, the vehicle's reversing process is a discontinuous action.

[0046] Furthermore, if the total driving distance is less than the preset driving distance, it is determined that there is a possibility that the vehicle corresponding to each frame between the target frame and the preset frame has made a U-turn; in order to avoid misjudging the vehicle reversing as a U-turn, no further scene recognition is performed on the scene sub-segment corresponding to the target frame to the preset frame, thereby reducing the waste of computing resources; if the total driving distance is greater than or equal to the preset driving distance, it is determined that there is a possibility that the vehicle corresponding to each frame between the target frame and the preset frame has made a U-turn, and further scene recognition can be performed on the scene sub-segment corresponding to the target frame to the preset frame.

[0047] The preset driving distance (distance_static) can be 15m or 17m, etc., because the distance required for the vehicle to make a U-turn is longer than that for the vehicle to turn. For example, if the vehicle driving distance from the 0th frame to the 100th frame is less than 15m, it is considered that the vehicle is in a low-speed driving condition or a vehicle turning condition from the 0th frame to the 100th frame, and further scene recognition is not performed on this frame. For example, if the vehicle driving distance from the 0th frame to the 100th frame is greater than or equal to 15m, it is considered that the vehicle may have made a U-turn operation from the 0th frame to the 100th frame, and further scene recognition can be performed on this frame.

[0048] Specifically, the vehicle position information and vehicle coordinate information may be the vehicle coordinates coordinate_start (start_x, start_y) under the target frame (frame_i frame), and the vehicle coordinates coordinate_i (i_x, i_y) under the preset frame (frame_i+1 frame); based on the vehicle position information corresponding to each frame, a specific implementation method of calculating the vehicle travel distance from the target frame to the preset frame may be: calculating the straight-line distance dist between coordinate_start and coordinate_i. If dis_sum is less than distance_static, it is considered that the distance from the target frame to the preset frame corresponding to the vehicle is insufficient to complete the U-turn operation.

[0049] It can be understood that this embodiment can effectively filter out frames in which U-turns are impossible by setting a preset speed threshold and a preset driving distance, ensuring that only frames in which U-turn scenes are actually likely to exist are identified; this embodiment only needs to obtain historical scene fragments and accurately obtain scene recognition results about vehicle U-turns in the historical scene fragments through the above method, thereby easily obtaining data about vehicle U-turns.

[0050] Step A2: If yes, determining the change of the vehicle's driving direction based on the vehicle heading angles corresponding to the target frame and the preset frame respectively; It should be noted that the vehicle heading angle can be obtained through the vehicle box or the vehicle speed. Specifically, the vehicle bounding box is usually detected by a visual or distance sensor such as a camera or radar, so the vehicle heading angle can be calculated based on the vehicle bounding box. The vehicle heading angle can also be derived by analyzing the vehicle's velocity vector (including speed magnitude and direction).

[0051] During the reversing process, the vehicle's driving direction may also change in a short period of time, for example, the direction changes when forward and reverse alternately occur. In this case, even if the total driving distance is greater than or equal to the preset driving distance, it is still possible to misjudge this reversing scene as a vehicle U-turn scene. At the same time, since the vehicle heading angle can reflect the change in the vehicle's driving direction, this embodiment determines the change in the vehicle's driving direction in the scene sub-segment corresponding to the target frame to the preset frame based on the vehicle heading angles corresponding to the target frame and the preset frame, when determining the possibility that the vehicle corresponding to each frame between the target frame and the preset frame has made a U-turn.

[0052] Step A3: determining a scene recognition result based on the change in the vehicle's driving direction.

[0053] Furthermore, by analyzing changes in the vehicle's driving direction, it is possible to determine whether the vehicle has a left U-turn scenario, a right U-turn scenario, and / or no U-turn scenario, that is, to obtain a scene recognition result.

[0054] In this embodiment, by acquiring historical scene segments, when intersection scenes are present in the historical scene segments, scene recognition results are determined based on the data of each frame in the historical scene segments, thereby avoiding misidentification of data on non-critical road sections as U-turn scenes, thereby accurately obtaining scene recognition results for vehicle U-turns in the historical scene segments. Furthermore, by setting a preset speed threshold and a preset travel distance, frames where U-turns are unlikely can be effectively filtered out, ensuring that only frames where U-turns are truly possible are recognized. This embodiment only requires acquiring historical scene segments and accurately obtaining scene recognition results for vehicle U-turns in the historical scene segments through the above-mentioned method, thereby easily obtaining data related to vehicle U-turns.

[0055] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above embodiment 1 can be referred to the above introduction and will not be described in detail later. Figure 4 After step S10, steps B1 to B3 are also included: Step B1, determining whether there is intersection identification information in each frame of data; In order to improve the efficiency and accuracy of identifying intersection scenes, this embodiment first determines whether there is intersection identification information in each frame data; if so, it is more likely that there is an intersection in the scene corresponding to the frame; if not, it is less likely that there is an intersection in the scene corresponding to the frame.

[0056] The specific implementation method for determining whether there is intersection identification information in each frame of data can be: based on the road image in the dynamic perception information, identify whether there is intersection identification information in each frame of data through a preset target recognition model; it can also be determined whether there is intersection identification information in the traffic sign information in the static map information corresponding to each frame of data.

[0057] Among them, intersection sign information can be speed bumps, intersection warning signs, traffic lights (traffic lights), zebra crossings, etc.

[0058] Step B2: If yes, calculate the target distance between the intersection and the vehicle based on the intersection identification information; It can be understood that since vehicles usually make U-turns when they are about to reach an intersection, in order to ensure effective filtering of frames where U-turns are impossible, this embodiment calculates the target distance between the intersection and the vehicle based on the intersection identification information; the intersection identification information includes the coordinate information of the intersection identification, and this embodiment can calculate the target distance between the intersection and the vehicle based on the coordinate information of the intersection identification and the coordinate information of the vehicle.

[0059] Specifically, for each frame of data, there may be traffic light information and / or zebra crossing information. If a frame contains traffic light information, the specific implementation method of calculating the target distance between the intersection and the vehicle based on the intersection identification information may be: The first coordinate is converted into a second coordinate in the vehicle's coordinate system; based on the second coordinate, a first longitudinal distance between at least one traffic light and the vehicle is calculated; based on the first longitudinal distance, a determination is made as to whether there is a target traffic light in front of the vehicle; if so, the minimum longitudinal distance among the first longitudinal distances corresponding to the target traffic light is used as the target distance between the intersection and the vehicle.

[0060] It can be understood that for each frame of data, when traffic light information exists in the road perception information in this embodiment, since multiple sets of traffic light information may exist at the same time, this embodiment can use the minimum distance from the vehicle to each traffic light as the target distance; if there is only one set of traffic light information, the distance from the traffic light to the vehicle is directly used as the target distance.

[0061] It should be noted that the traffic light information includes the first coordinates of at least one traffic light in the global coordinate system.

[0062] Since the position and direction of the vehicle change continuously as it moves in the global coordinate system, this may lead to cumulative errors in the distance between the traffic light and the vehicle calculated based on the global coordinate system. The vehicle coordinate system always uses the vehicle's own position as the reference point, which can more accurately reflect the vehicle's immediate motion state, thereby reducing the cumulative errors caused by dynamic driving.

[0063] Since a vehicle's U-turn operation usually occurs at a specific distance from an intersection, the error in the distance between the traffic light and the vehicle caused by the above-mentioned cumulative error may affect the accuracy and efficiency of judging the U-turn scenario. In this embodiment, the first coordinate of at least one traffic light in the global coordinate system is converted into a second coordinate in the vehicle's body coordinate system.

[0064] Specifically, a specific implementation of converting the first coordinate into the second coordinate in the vehicle body coordinate system may be: translating a point in the global coordinate system to a corresponding position in the vehicle body coordinate system.

[0065] In addition, a specific implementation of converting the first coordinates into second coordinates in the vehicle body coordinate system may also be: converting the first coordinates based on a rotation matrix to obtain the second coordinates in the vehicle body coordinate system.

[0066] Specifically, assuming that the first coordinate is the coordinate (global_x, global_y) corresponding to point P, the coordinates of the vehicle in the global coordinate system are (base_x, base_y), and the vehicle heading angle is base_theta; the specific implementation method of converting the first coordinate based on the rotation matrix to obtain the second coordinate in the vehicle body coordinate system can be: First, calculate the relative position difference between point P and the vehicle: x_diff=global_x-base_x; y_diff=global_y-base_y; Next, calculate the cosine value angle_cos and sine value angle_sin of the vehicle's heading angle: angle_cos=math.cos(base_theta); angle_sin=math.sin(base_theta); Finally, the first coordinate is transformed based on the rotation matrix to obtain the second coordinate (local_x, local_y) of point P in the vehicle body coordinate system: local_x=x_diff×angle_cos-y_diff×angle_sin; local_y=x_diff×angle_sin+y_diff×angle_cos; Furthermore, the above-mentioned local_x represents the longitudinal distance of the traffic light relative to the vehicle, and local_y represents the lateral distance of the traffic light relative to the vehicle; therefore, based on the second coordinate of at least one traffic light in the vehicle body coordinate system, the first longitudinal distance between at least one traffic light and the vehicle can be accurately calculated.

[0067] Furthermore, this embodiment determines whether there is a target traffic light in front of the vehicle based on the first longitudinal distance. Specifically, when the first longitudinal distance is greater than zero, it is determined that the corresponding target traffic light is in front of the vehicle; when the first longitudinal distance is equal to zero, it is determined that the corresponding target traffic light is side by side with the vehicle; and when the first longitudinal distance is greater than zero, it is determined that the corresponding target traffic light is behind the vehicle.

[0068] Because when a vehicle accurately makes a U-turn, it usually does so before entering an intersection, that is, the U-turn usually occurs when a traffic light is in front of the vehicle.

[0069] To improve determination efficiency, this embodiment uses the minimum distance (traffic_local_x_min) in the first longitudinal distance corresponding to the target traffic light in front of the vehicle as the target distance between the intersection and the vehicle.

[0070] If the traffic light information traffic_lights_infos does not exist in a frame, the minimum longitudinal distance (traffic_local_x_min) and the minimum lateral distance (traffic_local_y_min) from the traffic light to the vehicle are both assigned large positive values (such as 10000) to indicate that there is no intersection ahead.

[0071] In addition, for each frame of data, when zebra crossing information (crosswalks_infos) exists in the road perception information, since multiple sets of zebra crossing information may exist simultaneously, this embodiment may use the minimum distance from the vehicle to each zebra crossing as the target distance.

[0072] Specifically, the implementation method of calculating the target distance between the intersection and the vehicle based on the intersection identification information may be: converting the third coordinate into a fourth coordinate in the vehicle body coordinate system; based on the fourth coordinate, calculating the second longitudinal distance between each corner point and the vehicle; based on the second longitudinal distance, judging whether there is a target zebra crossing in front of or under the vehicle; if so, taking the minimum longitudinal distance in the second longitudinal distance corresponding to the target zebra crossing as the target distance between the intersection and the vehicle.

[0073] It should be noted that the zebra crossing information includes the third coordinates of all corner points of the rectangular frame corresponding to at least one zebra crossing in the global coordinate system.

[0074] Since a vehicle's U-turn operation usually occurs when it is at a specific distance from the intersection, the accuracy of calculating the target distance between the vehicle and the intersection is crucial for identifying U-turn scenarios. In addition, since the vehicle may pass through a zebra crossing during the U-turn process, or may start the U-turn on the zebra crossing, this embodiment needs to calculate the distance between all corner points of the rectangular box corresponding to each zebra crossing and the vehicle, rather than just calculating one corner point of the rectangular box corresponding to each zebra crossing, so as to avoid misjudging the zebra crossing as being behind the vehicle when there is a corner point behind the vehicle, when in fact other corner points may be in front of the vehicle (i.e., misjudging the zebra crossing as being behind the vehicle when it is below the vehicle).

[0075] At the same time, in order to avoid the error in the distance between the traffic light and the vehicle caused by cumulative errors, which may affect the judgment accuracy and efficiency of the U-turn scene, this embodiment converts the third coordinates of all corner points of the rectangular box corresponding to at least one zebra crossing in the global coordinate system into the fourth coordinates in the vehicle body coordinate system.

[0076] Assume that all corner points of the rectangular box corresponding to each zebra crossing are point_1, point_2, point_3, and point_4, respectively. Using the above coordinate conversion method, the four corner points point_1, point_2, point_3, and point_4 are converted into fourth coordinates local_point_1(local_x, local_y), local_point_2(local_x, local_y), local_point_3(local_x, local_y), and local_point_4(local_x, local_y) in the vehicle coordinate system, respectively. Local_x represents the longitudinal distance of the corner point from the vehicle, and local_y represents the lateral distance of the corner point from the vehicle. Therefore, based on the fourth coordinates, the second longitudinal distance of each corner point from the vehicle can be accurately calculated.

[0077] Furthermore, this embodiment determines whether there is a target zebra crossing in front of or below the vehicle based on the second longitudinal distance. Specifically, if the second longitudinal distance between at least one corner point of the target zebra crossing and the vehicle is greater than or equal to zero, it is determined that the target zebra crossing is in front of or below the vehicle; if the second longitudinal distance between all corner points of the target zebra crossing and the vehicle is less than zero, it is determined that the target zebra crossing is behind the vehicle.

[0078] If the second longitudinal distances between all corner points of the target zebra crossing and the vehicle are greater than zero, it is determined that the target zebra crossing is in front of the vehicle.

[0079] Because when a vehicle accurately makes a U-turn, it usually does so before entering an intersection, that is, the U-turn usually occurs when a traffic light is in front of the vehicle.

[0080] In order to improve the determination efficiency, this embodiment uses the minimum longitudinal distance (crosswalks_local_x_min) among the second longitudinal distances corresponding to the target zebra crossing as the target distance between the intersection and the vehicle.

[0081] If the traffic light information traffic_lights_infos does not exist in a certain frame, the minimum longitudinal distance (crosswalks_local_x_min) and the minimum lateral distance (crosswalks_local_y_min) from the traffic light to the vehicle are both assigned large positive values (such as 10000) to indicate that there is no intersection ahead.

[0082] If a frame contains both traffic light information and zebra crossing information, the minimum distance in the first longitudinal distance corresponding to the target zebra crossing can be used as the target distance between the intersection and the vehicle, or the minimum distance in the second longitudinal distance corresponding to the target traffic light can be used as the target distance between the intersection and the vehicle, or the smaller distance between the minimum distance in the first longitudinal distance and the minimum distance in the second longitudinal distance can be used as the target distance.

[0083] Step B3: When the target distance is less than a preset distance threshold, it is determined that an intersection scene exists in the historical scene segment.

[0084] It can be understood that if the minimum distance in the first longitudinal distance corresponding to the target zebra crossing is less than the preset distance threshold, or the minimum distance in the second longitudinal distance corresponding to the target traffic light is less than the preset distance threshold (valid_dist_threshold), then the vehicle is considered to be close enough to the intersection and the intersection is a valid intersection.

[0085] That is, if traffic_local_x_min is less than valid_dist_threshold, or crosswalks_local_x_min is less than valid_dist_threshold, it is considered that there is a valid intersection in the frame, otherwise there is no valid intersection in the frame; if there is no valid intersection in all frames in the historical scene segment, no further vehicle U-turn scene recognition is performed on the historical scene segment.

[0086] This embodiment uses the above method to accurately identify whether there is a valid intersection in the historical scene segment, thereby improving the accuracy of obtaining data about vehicle turning scenes.

[0087] Based on the first and second embodiments of the present application, in the third embodiment of the present application, the same or similar contents as those in the first embodiment can be referred to the above introduction and will not be described in detail later. Figure 5 , step A2 also includes steps C1~C2: Step C1, calculating the angle difference between the vehicle heading angles corresponding to the target frame and the preset frame respectively; It should be noted that when it is determined that there is a possibility that the vehicle corresponding to each frame between the target frame and the preset frame has a U-turn, it is still impossible to determine whether the U-turn actually occurred or not. This embodiment accurately captures the change in the vehicle's driving direction from the target frame to the preset frame by calculating the angle difference between the vehicle heading angles corresponding to the target frame and the preset frame respectively.

[0088] For example, calculate the angular difference (theta_end - theta_start) between the vehicle heading angle (theta_start) corresponding to the target frame (frame_i) and the vehicle heading angle (theta_end) corresponding to the preset frame (frame_i+100).

[0089] Step C2, determining a change in the vehicle's driving direction in the scene sub-segment corresponding to the target frame and the preset frame based on the angle difference; Since the angle difference can reflect the change in the vehicle's driving direction from the target frame to the preset frame, this embodiment can determine the change in the vehicle's driving direction in the scene sub-segment corresponding to the target frame to the preset frame based on the angle difference; for example, when the angle difference is within a certain range, it can be considered that the vehicle has made a U-turn, and if it is not within this range, it is considered that the vehicle has not made a U-turn.

[0090] In addition, a specific embodiment of determining the change in the vehicle driving direction in the scene sub-segment corresponding to the target frame and the preset frame based on the angle difference may also be: The angle difference is normalized; if the normalized angle difference is within a first angle interval, it is determined that the change in the vehicle's driving direction is a left U-turn; if the normalized angle difference is within a second angle interval, it is determined that the change in the vehicle's driving direction is a right U-turn; if the normalized angle difference is within a third angle interval, it is determined that the change in the vehicle's driving direction is that no U-turn occurs; wherein, there is no intersection between the first angle interval, the second angle interval, and the third angle interval.

[0091] It should be noted that, in the vehicle left turn scene and the vehicle right turn scene, the angle difference between the vehicle heading angle corresponding to the target frame and the preset frame may be - , 2 , 3 , 6 In order to facilitate the subsequent judgment of the driving operation type corresponding to the angle difference, this embodiment normalizes the angle difference, thereby judging the change of the vehicle's driving direction more intuitively, simply and accurately.

[0092] The normalization of the angle difference may be normalizing the angle difference to the same quantization standard, for example, normalizing the angle difference to [0, 2 )or(- , ) etc.; Specifically, the angle difference is normalized to [0, 2 ) can be calculated as follows: Normalized angle difference (theta_diff) = (theta_end - theta_start + 2 )% (2 ).

[0093] Correspondingly, the first angle interval, the second angle interval and the third angle interval can be set to represent the intervals where the corresponding angle differences of the corresponding scenes are located, and there is no intersection between the first angle interval, the second angle interval and the third angle interval; it can be understood that the first angle interval can be , the second angle interval can be , the third angle interval can be and , that is, there is no intersection between the first angle interval, the second angle interval and the third angle interval.

[0094] It can be understood that since the angle difference between the vehicle heading angles corresponding to the target frame and the preset frame is normalized to the same quantitative standard, the change in the vehicle's driving direction can be determined simply by comparing the angles without causing any errors. , 6 Improve data processing efficiency when the same standard cannot be used for comparison of equal angles.

[0095] Specifically, if the normalized angle difference is within the first angle interval, it is determined that the change in the vehicle's driving direction is a left U-turn; if the normalized angle difference is within the second angle interval, it is determined that the change in the vehicle's driving direction is a right U-turn; if the normalized angle difference is within the third angle interval, it is determined that the change in the vehicle's driving direction is that no U-turn occurs. It can be understood that if the normalized angle difference is within the third angle interval, the change in the vehicle's driving direction may be straight going, turning left, turning right or stationary, etc.

[0096] Furthermore, after the step of determining the change in the vehicle's driving direction in the scene sub-segment corresponding to the target frame and the preset frame based on the angle difference, the following steps can be performed: updating the target frame to a frame that slides backward by a preset number of frames, returning to the step of determining whether there is a possibility of a U-turn in the vehicle corresponding to each frame between the target frame and the preset frame, until the preset frame is the end frame, to obtain the change in the vehicle's driving direction in multiple scene sub-segments corresponding to the start frame and the end frame.

[0097] Since the above only determines whether there is a possibility of a U-turn in the vehicle corresponding to each frame between the target frame and the preset frame, and the change in the vehicle's driving direction in the scene sub-segment corresponding to the target frame and the preset frame, in order to determine the change in the vehicle's driving direction in multiple scene sub-segments corresponding to the start frame and the end frame, the target frame can be updated to a frame that slides backward by a preset number of frames, and the process returns to the step of determining whether there is a possibility of a U-turn in the vehicle corresponding to each frame between the target frame and the preset frame until the preset frame is the end frame.

[0098] Among them, the preset number of frames can be less than the number of frames corresponding to the preset frame length, thereby increasing the number of identified scene sub-segments without missing the identification of each frame data in the historical scene segment; the preset number of frames can also be equal to the number of frames corresponding to the preset frame length, reducing repeated scenes and improving scene recognition efficiency; the preset number of frames can also be greater than the number of frames corresponding to the preset frame length and less than (N-100 frames), thereby reducing the number of scene sub-segments that need to be identified and improving data processing efficiency.

[0099] In order to increase the number of identified scene sub-segments without missing the identification of each frame of data in the historical scene segment, this embodiment can return to the step of determining whether there is a possibility of a U-turn in the corresponding vehicle in each frame between the target frame and the preset frame every 20 frames or every 50 frames until the preset frame is the end frame (that is, until all frames are traversed).

[0100] Furthermore, the specific implementation method of determining the scene recognition result based on the change of the vehicle's driving direction may be: determining the scene recognition results of the multiple scene sub-segments based on the change of the vehicle's driving direction in the multiple scene sub-segments corresponding to the start frame to the end frame.

[0101] It can be understood that if the change in the vehicle's driving direction in the scene sub-segment from frame 0 to frame 100 is a left U-turn, the change in the vehicle's driving direction in the scene sub-segment from frame 20 to frame 120 is no U-turn, ..., the change in the vehicle's driving direction in the scene sub-segment from frame (N-100) to frame N is a right U-turn; then the corresponding scene recognition results from frame 0 to frame 100 are a scene in which the vehicle makes a left U-turn, the scene recognition results from frame 20 to frame 120 are a scene in which the vehicle does not make a U-turn, ..., the scene recognition results from frame (N-100) to frame N are a scene in which the vehicle makes a right U-turn.

[0102] Furthermore, after the step of determining the scene recognition result based on the change in the vehicle's driving direction, the method may further include: determining the moment when the vehicle makes a U-turn based on a target scene sub-segment in which the vehicle makes a U-turn.

[0103] Since the change of the vehicle driving direction in the multiple scene sub-segments corresponding to the start frame and the end frame is obtained in the above manner, the U-turn moment corresponding to the target scene sub-segment where the vehicle U-turn scene occurs can be determined.

[0104] For example, the target scene sub-segment with a vehicle U-turn scene includes the scene sub-segment corresponding to the 0th frame to the 100th frame, the scene sub-segment corresponding to the 80th frame to the 180th frame, ..., the scene sub-segment corresponding to the (N-100)th frame to the Nth frame; in order to obtain higher-quality training data, the vehicle U-turn moment (including the vehicle left U-turn moment and the vehicle right U-turn moment) can be determined based on the above-mentioned target scene sub-segment with the vehicle U-turn scene, for example, the vehicle left U-turn moment is from the 0th frame to the 100th frame, and the vehicle right U-turn moment is from the (N-100)th frame to the Nth frame.

[0105] This embodiment can quickly locate the vehicle left U-turn scene and the vehicle right U-turn scene in the historical scene fragment through the above method, and can accurately locate the vehicle left U-turn moment and the vehicle right U-turn moment; thereby obtaining more detailed data about the vehicle turning scene.

[0106] This application also provides a U-turn scene recognition device, please refer to Figure 6 , the U-turn scene recognition device includes: An acquisition module 10 is used to acquire historical scene fragments; The recognition module 20 is used to determine the scene recognition result based on each frame data in the historical scene segment when there is an intersection scene in the historical scene segment, wherein the scene recognition result includes the presence of a left U-turn scene, a right U-turn scene, and / or the absence of a U-turn scene in the historical scene segment.

[0107] In one embodiment, the identification module 20 includes: The first judgment submodule is used to judge whether the vehicle corresponding to each frame between the target frame and the preset frame has a possibility of making a U-turn, wherein the target frame includes the start frame, and the preset frame is separated from the target frame by a preset frame length; a determination submodule, configured to determine, if any, a change in the vehicle's driving direction based on the vehicle heading angles corresponding to the target frame and the preset frame respectively; The recognition submodule is used to determine a scene recognition result based on changes in the vehicle's driving direction.

[0108] In one embodiment, the first determination submodule includes: a judgment unit, configured to judge whether the vehicle speed corresponding to the target frame data is less than a preset speed threshold; A first calculation unit is configured to calculate, if the vehicle speed is greater than or equal to a preset speed threshold, the vehicle travel distances corresponding to each consecutive frame from the target frame to the preset frame based on the vehicle position information corresponding to each frame; A summing unit, used for summing the travel distances of each vehicle to obtain a total travel distance; The second determining unit is configured to determine the possibility that the vehicle has a U-turn in each frame between the target frame and the preset frame if the total driving distance is greater than or equal to the preset driving distance.

[0109] In one embodiment, the determining submodule includes: A second calculation unit is used to calculate the angle difference between the vehicle heading angles corresponding to the target frame and the preset frame respectively; The second determining unit is configured to determine, based on the angle difference, a change in the vehicle's driving direction in a scene sub-segment corresponding to a target frame and a preset frame.

[0110] After the step of determining a change in the vehicle's driving direction in a scene sub-segment corresponding to a target frame to a preset frame based on the angle difference, the determining submodule further includes: The return unit is used to update the target frame to a frame that slides backward a preset number of frames, return to the step of determining whether there is a possibility of a U-turn corresponding to each frame between the target frame and the preset frame, until the preset frame is the end frame, and obtain the change in the vehicle's driving direction in multiple scene sub-segments corresponding to the start frame and the end frame.

[0111] In one embodiment, the second determining unit includes: a normalization subunit, configured to perform normalization processing on the angle difference; a first determining subunit, configured to determine that the change in the vehicle's driving direction is a left U-turn if the normalized angle difference is within a first angle interval; a second determining subunit, configured to determine that the change in the vehicle's driving direction is a right U-turn if the normalized angle difference is within a second angle interval; a third determining subunit, configured to determine that the change in the vehicle's driving direction is that no U-turn occurs if the normalized angle difference is within a third angle range; There is no intersection among the first angle interval, the second angle interval and the third angle interval.

[0112] In one embodiment, the identification submodule includes: a third determining unit, configured to determine scene recognition results of the multiple scene sub-segments based on changes in the vehicle driving directions in the multiple scene sub-segments corresponding to the starting frame and the ending frame; Wherein, after the step of determining the scene recognition result based on the change in the vehicle's driving direction, the U-turn scene recognition device further includes: The first determining module is configured to determine a moment when the vehicle makes a U-turn based on a target scene sub-segment in which the vehicle makes a U-turn.

[0113] In one embodiment, after the step of obtaining historical scene segments, the U-turn scene recognition device further includes: A judgment module, used to judge whether there is intersection identification information in each frame of data; a calculation module, configured to calculate a target distance between the intersection and the vehicle based on the intersection identification information, if any; The second determining module is configured to determine that an intersection scene exists in the historical scene segment when the target distance is less than a preset distance threshold.

[0114] In one embodiment, the intersection identification information includes traffic light information, and the traffic light information includes a first coordinate of at least one traffic light in a global coordinate system. The calculation module includes: A first conversion submodule, configured to convert the first coordinate into a second coordinate in a vehicle body coordinate system; a first calculation submodule, configured to calculate a first longitudinal distance between at least one traffic light and the vehicle based on the second coordinate; a second judgment submodule, configured to judge whether there is a target traffic light in front of the vehicle based on the first longitudinal distance; a first setting submodule, configured to, if present, use the minimum longitudinal distance among the first longitudinal distances corresponding to the target traffic light as the target distance between the intersection and the vehicle; The second judgment submodule includes: The fourth determining unit is configured to determine that the corresponding target traffic light is in front of the vehicle when the first longitudinal distance is greater than zero.

[0115] In one embodiment, the intersection identification information further includes zebra crossing information, where the zebra crossing information includes third coordinates of all corner points of a rectangular box corresponding to at least one zebra crossing in a global coordinate system, and the calculation module includes: A second conversion submodule, configured to convert the third coordinate into a fourth coordinate in a vehicle body coordinate system; a second calculation submodule, configured to calculate a second longitudinal distance between each corner point and the vehicle based on the fourth coordinate; a third judgment submodule, configured to judge whether there is a target zebra crossing in front of or below the vehicle based on the second longitudinal distance; A second setting submodule is configured to, if present, use the minimum longitudinal distance among the second longitudinal distances corresponding to the target zebra crossing as the target distance between the intersection and the vehicle; Wherein, the third judgment submodule includes: a sixth determining unit, configured to determine that the target zebra crossing is in front of the vehicle or below the vehicle if a second longitudinal distance between at least one corner point of the target zebra crossing and the vehicle is greater than or equal to zero; The seventh determining unit is configured to determine that the target zebra crossing is behind the vehicle if the second longitudinal distances between all corner points of the target zebra crossing and the vehicle are less than zero.

[0116] The U-turn scene recognition device provided in this application, utilizing the U-turn scene recognition method described in the aforementioned embodiments, can address the technical issue of the difficulty in acquiring data related to vehicle turning scenarios. Compared to the prior art, the U-turn scene recognition device provided in this application achieves the same beneficial effects as the U-turn scene recognition method described in the aforementioned embodiments. Other technical features of the U-turn scene recognition device are the same as those disclosed in the aforementioned embodiments and are not further elaborated here.

[0117] The present application provides a U-turn scene recognition device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the U-turn scene recognition method in the above-mentioned embodiment one.

[0118] Reference below Figure 7 , which shows a schematic diagram of the structure of a U-turn scene recognition device suitable for implementing embodiments of the present application. The U-turn scene recognition device in the embodiments of the present application can include, but is not limited to, mobile terminals such as mobile phones, tablet computers, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PMPs (Portable Media Players), in-vehicle terminals (such as in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 7 The U-turn scene recognition device shown is only an example and should not bring any limitation to the functions and usage range of the embodiments of the present application.

[0119] like Figure 7 As shown, the U-turn scene recognition device may include a processing device 1001 (e.g., a central processing unit, graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage device 1003 into a random access memory (RAM) 1004. RAM 1004 also stores various programs and data required for the operation of the U-turn scene recognition device. Processing device 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input device 1007, such as a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output device 1008, such as a liquid crystal display (LCD), speaker, vibrator, etc.; storage device 1003, such as a magnetic tape or hard disk; and communication device 1009. The communication device 1009 can allow the U-turn scene recognition device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a U-turn scene recognition device with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented or have alternatively.

[0120] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0121] The U-turn scene recognition device provided in this application, utilizing the U-turn scene recognition method described in the aforementioned embodiment, can address the technical issue of the difficulty in acquiring data related to vehicle turning scenarios. Compared to the prior art, the U-turn scene recognition device provided in this application achieves the same beneficial effects as the U-turn scene recognition method described in the aforementioned embodiment. Other technical features of the U-turn scene recognition device are the same as those disclosed in the aforementioned embodiment and are not further elaborated here.

[0122] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0123] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any variation or substitution that a person skilled in the art can easily conceive within the technical scope disclosed in the present application should be included within the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

[0124] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, and the computer-readable program instructions are used to execute the U-turn scene recognition method in the above embodiment.

[0125] The computer-readable storage medium provided herein may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including, but not limited to, wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0126] The computer-readable storage medium may be included in the U-turn scene recognition device; or may exist independently without being assembled into the U-turn scene recognition device.

[0127] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the U-turn scene recognition device, the U-turn scene recognition device executes the U-turn scene recognition method.

[0128] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0129] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to the various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the prescribed logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the prescribed function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0130] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0131] The computer-readable storage medium provided in this application stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned U-turn scene recognition method. This computer-readable storage medium can address the technical issue of the difficulty in acquiring data related to vehicle turning scenarios. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are similar to those of the U-turn scene recognition method provided in the aforementioned embodiments, and are not further elaborated here.

[0132] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned U-turn scene recognition method when executed by a processor.

[0133] The computer program product provided in this application can solve the technical problem of the high difficulty in obtaining data related to vehicle turning scenarios. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the U-turn scene recognition method provided in the above-mentioned embodiment, and will not be elaborated here.

[0134] The above description is only part of the embodiments of the present application and does not limit the patent range of the present application. All equivalent structural transformations made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection range of the present application.

Claims

1. A U-turn scene recognition method, characterized in that: The U-turn scene recognition method includes: Get historical scene clips; When there is a road intersection scene in the historical scene segment, determining a scene recognition result based on each frame data in the historical scene segment; The scene recognition result includes whether a left U-turn scene, a right U-turn scene, and / or no U-turn scene exists in the historical scene segment.

2. The U-turn scene recognition method according to claim 1, characterized in that: The step of determining the scene recognition result based on each frame data in the historical scene fragment includes: Determine whether the vehicle corresponding to each frame between a target frame and a preset frame has a possibility of making a U-turn, wherein the target frame includes a start frame and the preset frame is separated from the target frame by a preset frame length; If so, determining a change in the vehicle's driving direction based on the vehicle heading angles corresponding to the target frame and the preset frame respectively; A scene recognition result is determined based on the change in the vehicle's driving direction.

3. The U-turn scene recognition method according to claim 2, characterized in that: The step of determining whether the vehicle corresponding to each frame between the target frame and the preset frame has a possibility of making a U-turn includes: Determine whether the vehicle speed corresponding to the target frame data is less than a preset speed threshold; If the vehicle speed is greater than or equal to a preset speed threshold, then based on the vehicle position information corresponding to each frame, the vehicle travel distances corresponding to each two consecutive frames from the target frame to the preset frame are calculated in sequence; Sum up the travel distances of each vehicle to get the total travel distance; If the total driving distance is greater than or equal to the preset driving distance, it is determined that there is a possibility that the vehicle corresponding to each frame between the target frame and the preset frame has a U-turn.

4. The U-turn scene recognition method according to claim 2, characterized in that: The step of determining a change in the vehicle's driving direction based on the vehicle heading angles corresponding to the target frame and the preset frame respectively includes: Calculating the angle difference between the vehicle heading angles corresponding to the target frame and the preset frame respectively; Based on the angle difference, a change in the vehicle's driving direction in the scene sub-segment corresponding to the target frame and the preset frame is determined.

5. The U-turn scene recognition method according to claim 4, characterized in that: After the step of determining a change in the vehicle's driving direction in a scene sub-segment corresponding to a target frame to a preset frame based on the angle difference, the method further includes: The target frame is updated to be the frame that slides backward by the second number of frames, and the process returns to the step of determining whether the vehicle corresponding to each frame between the target frame and the preset frame has the possibility of making a U-turn, until the preset frame is the end frame, thereby obtaining the change in the vehicle driving direction in the multiple scene sub-segments corresponding to the start frame and the end frame.

6. The U-turn scene recognition method according to claim 4, characterized in that: The step of determining a change in the vehicle driving direction in a scene sub-segment corresponding to a target frame and a preset frame based on the angle difference includes: performing normalization processing on the angle difference; If the normalized angle difference is within the first angle interval, determining that the change in the vehicle's driving direction is a left U-turn; If the normalized angle difference is within the second angle interval, determining that the change in the vehicle's driving direction is a right U-turn; If the normalized angle difference is within the third angle range, determining that the change in the vehicle's driving direction does not result in a U-turn; There is no intersection among the first angle interval, the second angle interval and the third angle interval.

7. The U-turn scene recognition method according to claim 4, characterized in that: The step of determining the scene recognition result based on the change in the vehicle's driving direction includes: determining scene recognition results for the multiple scene sub-segments based on changes in the vehicle driving directions in the multiple scene sub-segments corresponding to the start frame and the end frame; After the step of determining the scene recognition result based on the change in the vehicle's driving direction, the method further includes: The U-turn moment of the vehicle is determined based on the target scene sub-segment where the vehicle U-turn scene exists.

8. The U-turn scene recognition method according to claim 1, characterized in that: After the step of obtaining the historical scene fragments, the method further includes: Determine whether there is intersection identification information in each frame of data; If so, the target distance between the intersection and the vehicle is calculated based on the intersection identification information; When the target distance is less than a preset distance threshold, it is determined that an intersection scene exists in the historical scene segment.

9. The U-turn scene recognition method according to claim 8, characterized in that: The intersection identification information includes traffic light information, and the traffic light information includes a first coordinate of at least one traffic light in a global coordinate system. The step of calculating the target distance between the intersection and the vehicle based on the intersection identification information includes: Converting the first coordinate into a second coordinate in a vehicle body coordinate system; Calculating a first longitudinal distance between at least one traffic light and the vehicle based on the second coordinate; determining, based on the first longitudinal distance, whether there is a target traffic light in front of the vehicle; If so, the minimum longitudinal distance among the first longitudinal distances corresponding to the target traffic light is used as the target distance between the intersection and the vehicle; The step of determining whether there is a target traffic light in front of the vehicle based on the first longitudinal distance includes: When the first longitudinal distance is greater than zero, it is determined that the corresponding target traffic light is in front of the vehicle.

10. The U-turn scene recognition method according to claim 8, characterized in that: The intersection identification information further includes zebra crossing information, wherein the zebra crossing information includes third coordinates of all corner points of a rectangular box corresponding to at least one zebra crossing in a global coordinate system. The step of calculating the target distance between the intersection and the vehicle based on the intersection identification information includes: Converting the third coordinate into a fourth coordinate in the vehicle body coordinate system; Based on the fourth coordinate, calculating a second longitudinal distance between each corner point and the vehicle; Based on the second longitudinal distance, determining whether there is a target zebra crossing in front of or below the vehicle; If so, the minimum longitudinal distance among the second longitudinal distances corresponding to the target zebra crossing is used as the target distance between the intersection and the vehicle; The step of determining whether there is a target zebra crossing in front of or below the vehicle based on the second longitudinal distance includes: If a second longitudinal distance between at least one corner point of the target zebra crossing and the vehicle is greater than or equal to zero, determining that the target zebra crossing is in front of the vehicle or below the vehicle; If the second longitudinal distances between all corner points of the target zebra crossing and the vehicle are less than zero, it is determined that the target zebra crossing is behind the vehicle.

11. A U-turn scene recognition device, characterized in that: The U-turn scene recognition device includes: Acquisition module, used to obtain historical scene fragments; An identification module is used to determine a scene recognition result based on each frame data in the historical scene segment when there is an intersection scene in the historical scene segment, wherein the scene recognition result includes the presence of a left U-turn scene, a right U-turn scene, and / or the absence of a U-turn scene in the historical scene segment.

12. A U-turn scene recognition device, characterized in that: The U-turn scene recognition device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the U-turn scene recognition method according to any one of claims 1 to 10.

13. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the U-turn scene recognition method according to any one of claims 1 to 10 are implemented.

14. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the U-turn scene recognition method according to any one of claims 1 to 10 are implemented.

Images