Category determination method, apparatus, vehicle terminal, and computer program product

Abstract

This application relates to the field of automotive technology and provides a category determination method, apparatus, vehicle-mounted terminal, and computer program product. The method includes: determining obstacle information of an obstacle to be matched using data frames acquired by radar in a vehicle and a target detection model; determining a target object matching the obstacle information from a set of tracked objects corresponding to the vehicle; updating the category of the target object based on a category information table of the target object and the obstacle information to obtain the target category of the target object; the category information table includes multiple set categories and the number of occurrences corresponding to each set category. Compared with the prior art, which requires information from multiple different devices, this application only needs to acquire data acquired by radar to achieve category determination, eliminating the need for multiple devices, reducing costs, and avoiding the complexity caused by category determination based on information from multiple different devices. Therefore, category determination can be achieved without high computing power, thereby improving work efficiency.

Description

Technical Field

[0001] This application belongs to the field of computer technology, and in particular relates to a method, apparatus, vehicle terminal and computer program product for class determination. Background Technology

[0002] With the increasing prevalence of autonomous driving, self-driving vehicles are already able to replace drivers in certain scenarios. For the planning and decision-making processes and human-machine interaction of autonomous vehicles, obstacle category information affects the realism and comfort of human-machine interaction, and to some extent influences the planning and decision-making processes of autonomous vehicles.

[0003] Existing methods for determining obstacle categories typically require acquiring information from multiple different devices to detect and update obstacle categories. This process is complex and requires significant computing power. Therefore, existing technologies suffer from high costs and low efficiency. Summary of the Invention

[0004] This application provides a category determination method, apparatus, vehicle terminal, and computer program product to solve the problems of high cost and low efficiency in the prior art.

[0005] In a first aspect, embodiments of this application provide a method for determining a category, including:

[0006] The obstacle information of the obstacle to be matched is determined by using data frames collected by the radar in the vehicle and the target detection model; the target detection model is trained by a pre-built deep learning model.

[0007] A target object matching the obstacle information is determined from the set of tracked objects corresponding to the vehicle; the set of tracked objects includes multiple historical obstacles, each of which carries historical obstacle information;

[0008] The category of the target object is updated based on the category information table of the target object and the obstacle information to obtain the target category of the target object; the category information table includes multiple set categories and the number of occurrences of each set category.

[0009] Optionally, determining the target object matching the obstacle information from the set of tracked objects corresponding to the vehicle includes:

[0010] Based on the obstacle information, the first position of the obstacle to be matched is determined;

[0011] Based on the information of each historical obstacle, the second position of each historical obstacle at the time of acquisition is predicted; the time of acquisition refers to the moment when the radar acquires the data frame.

[0012] The target object is determined based on the first position and each of the second positions.

[0013] Optionally, determining the target object based on the first position and each of the second positions includes:

[0014] Perform coordinate system transformation on the first position to obtain the third position in the vehicle coordinate system corresponding to the second position;

[0015] The differences between the third position and each of the second positions are calculated.

[0016] Generate a cost matrix based on each of the aforementioned difference values;

[0017] The target object is obtained by matching the cost matrix according to the Hungarian algorithm.

[0018] Optionally, updating the category of the target object based on the category information table of the target object and the obstacle information to obtain the target category of the target object includes:

[0019] Based on the obstacle information, the category of the obstacle to be matched is determined;

[0020] The category information table is updated according to the category of the obstacle to be matched to obtain the target information table;

[0021] The target category is determined based on the number of occurrences of each of the defined categories in the target information table; each defined category includes the category of the obstacle to be matched.

[0022] Optionally, determining the target category based on the frequency of occurrence of each of the defined categories in the target information table includes:

[0023] If there are multiple maximum values ​​among all occurrences, then the target category of the target object is determined as the historical category; the historical category refers to the target object category at the moment before the acquisition time; the acquisition time refers to the moment when the radar acquires the data frame.

[0024] Optionally, determining the target category based on the frequency of occurrence of each of the defined categories in the target information table includes:

[0025] If there is only one maximum value, and the set category corresponding to the maximum value is not a historical category, then the occurrence status of the set category corresponding to the maximum value is determined.

[0026] If the occurrence state is a continuous occurrence state, then the set category corresponding to the maximum value is determined as the target category.

[0027] Optionally, after determining the occurrence state of the set category corresponding to the maximum value, the method further includes:

[0028] If the occurrence state is a non-continuous occurrence state, then the target category of the target object is determined as the historical category.

[0029] Optionally, the occurrence count carries an occurrence time; determining the occurrence status of the set category corresponding to the maximum value includes:

[0030] If the number of occurrences of the set category corresponding to the maximum value within a set time period is greater than or equal to a set threshold, and the occurrence time within the set time period is continuous, then the occurrence state is determined to be the continuous occurrence state; wherein, the end time of the set time period is the acquisition time when the radar acquires the data frame;

[0031] If the number of occurrences of the set category corresponding to the maximum value within the set time period is less than the set threshold, or if the occurrence times within the set time period are not continuous, then the occurrence state is determined to be a discontinuous occurrence state.

[0032] Optionally, determining the target category based on the frequency of occurrence of each of the defined categories in the target information table includes:

[0033] If there is only one maximum value, and the set category corresponding to the maximum value is a historical category, then the target category of the target object is determined as the historical category; the historical category refers to the target object category at the moment before the acquisition time.

[0034] Secondly, embodiments of this application provide a category determination apparatus, comprising:

[0035] An information determination unit is used to determine the obstacle information of the obstacle to be matched by using data frames collected by the radar in the vehicle and a target detection model; the target detection model is trained by a pre-built deep learning model.

[0036] The first object determination unit is used to determine a target object that matches the obstacle information from the tracking object set corresponding to the vehicle; the tracking object set includes multiple historical obstacles, and each historical obstacle carries historical obstacle information;

[0037] The first update unit is used to update the category of the target object based on the category information table of the target object and the obstacle information to obtain the target category of the target object; the category information table includes multiple set categories and the number of occurrences of each set category.

[0038] Thirdly, embodiments of this application provide an in-vehicle terminal, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the category determination method as described in any one of the first aspects above.

[0039] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the category determination method as described in any one of the first aspects above.

[0040] Fifthly, embodiments of this application provide a computer program product that, when run on an in-vehicle terminal, enables the in-vehicle terminal to execute the category determination method described in any of the first aspects above.

[0041] Sixthly, embodiments of this application provide a vehicle including an on-board terminal, the on-board terminal being used to perform the category determination method as described in any of the first aspects.

[0042] The beneficial effects of the embodiments of this application compared with the prior art are:

[0043] This application provides a category determination method that uses data frames collected by radar in a vehicle and a target detection model to determine obstacle information of the obstacle to be matched. The target detection model is trained by a pre-built deep learning model. A target object matching the obstacle information is determined from a set of tracked objects corresponding to the vehicle. The set of tracked objects includes multiple historical obstacles, each carrying historical obstacle information. The category of the target object is updated based on the category information table of the target object and the obstacle information to obtain the target category of the target object. The category information table includes multiple set categories and the frequency of occurrence of each set category. Compared with existing technologies that require information from multiple different devices, this application only needs to acquire data collected by radar to achieve category determination, eliminating the need for multiple devices. This reduces costs and avoids the complexity of category determination from information from multiple different devices, thus achieving category determination without high computing power and improving update efficiency. Furthermore, determining the category of the tracked object through a single sensor prevents category jumps caused by changes in categories detected by different sensors, avoiding impact on planning decisions and human-computer interaction. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a flowchart illustrating the implementation of a category determination method provided in an embodiment of this application;

[0046] Figure 2 This is a flowchart illustrating the implementation of a category determination method provided in another embodiment of this application;

[0047] Figure 3 This is a flowchart illustrating the implementation of a category determination method provided in another embodiment of this application;

[0048] Figure 4 This is a flowchart illustrating the implementation of a category determination method provided in another embodiment of this application;

[0049] Figure 5 This is a schematic diagram of the structure of a category determination device provided in an embodiment of this application;

[0050] Figure 6 This is a schematic diagram of the structure of an in-vehicle terminal provided in one embodiment of this application. Detailed Implementation

[0051] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0052] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0053] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0054] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0055] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0056] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0057] In practical applications, with the popularization of autonomous driving, autonomous vehicles are already able to replace drivers in some scenarios. For the planning and decision-making of autonomous vehicles and human-machine interaction, obstacle category information affects the realism and comfort of human-machine interaction, and to some extent influences the planning and decision-making of autonomous vehicles. Currently, determining obstacle categories typically requires acquiring detection information from multiple sensors, thus increasing equipment costs. Then, weighted fusion of the categories from various sensors places certain demands on algorithms and computing power.

[0058] Therefore, this application proposes a low-cost, fast, and efficient category determination method that can accurately determine the category of obstacles using a single sensor, thereby reducing costs. It can also prevent the category of the tracked object from changing due to the jump in the category detected by different sensors, thus avoiding affecting planning decisions and human-computer interaction.

[0059] The following will provide a detailed description using specific embodiments.

[0060] It should be noted that the vehicles in all embodiments of this application are vehicles with autonomous driving capabilities and equipped with radar. This radar can be a lidar system.

[0061] Please see Figure 1 , Figure 1 This is a flowchart illustrating the implementation of a category determination method according to an embodiment of this application. In this embodiment, the entity executing the category determination method is an in-vehicle terminal.

[0062] like Figure 1 As shown, a category determination method provided in one embodiment of this application may include S101 to S103, which are detailed below:

[0063] In S101, obstacle information of the obstacle to be matched is determined by the data frames collected by the radar in the vehicle and the target detection model; the target detection model is trained by a pre-built deep learning model.

[0064] In this embodiment, the radar in the vehicle can operate continuously during vehicle operation and collect data frames within its detection range in real time. These data frames are point cloud data.

[0065] After acquiring the data frame collected by the radar, the vehicle terminal can perform corresponding parsing and other processing on the data frame, and input the data frame into the target detection model for processing to obtain the obstacle information of the obstacle to be matched.

[0066] The object detection model can be obtained by training a pre-built deep learning model based on a preset sample set. Each sample data point in the preset sample set includes a sample data frame and corresponding obstacle information. During training of the pre-built deep learning model, the sample data frame from each sample data point is used as the input to the deep learning model, and the corresponding obstacle information is used as the output. Through training, the deep learning model can learn the correspondence between all possible sample data frames and obstacle information, and the trained deep learning model becomes the object detection model.

[0067] It should be noted that there can be one or more obstacles to be matched.

[0068] The categories of obstacles to be matched include, but are not limited to: other vehicles and people.

[0069] When the category of the obstacle to be matched is other vehicles, its obstacle information includes, but is not limited to: three-dimensional coordinate information, length, width and height, heading angle and driving direction, etc.

[0070] When the category of the obstacle to be matched is a person, the obstacle information includes, but is not limited to: three-dimensional coordinate information, length, width and height, and direction of movement.

[0071] In S102, a target object matching the obstacle information is determined from the tracking object set corresponding to the vehicle; the tracking object set includes multiple historical obstacles, each of which carries historical obstacle information.

[0072] It should be noted that during vehicle operation, the onboard terminal can identify each obstacle that becomes part of the vehicle as a tracking object and add it to the vehicle's corresponding tracking object set. This tracking object set includes multiple historical obstacles, each carrying historical obstacle information.

[0073] Historical obstacle information includes, but is not limited to: the three-dimensional coordinates, length, width, height, speed, and heading angle of historical obstacles.

[0074] In this embodiment of the application, after obtaining the obstacle information of the obstacle to be matched, the vehicle terminal can compare the obstacle information with the historical obstacle information of each historical obstacle in the tracking object set one by one, so as to determine the historical obstacle information similar to the obstacle information from the tracking object set, and determine the historical obstacle corresponding to the similar historical obstacle information as the target object that matches the obstacle information.

[0075] It should be noted that when there is only one obstacle to be matched, there can also be only one target object; when there are multiple obstacles to be matched, there can also be multiple target objects.

[0076] In some possible embodiments, when the target object is not present in the tracking object set, the vehicle terminal can directly add the obstacle to be matched to the tracking object set.

[0077] In one embodiment of this application, to improve matching accuracy, the vehicle terminal can specifically achieve the following: Figure 2 Steps S201 to S203, which involve determining the target object that matches the obstacle information, are detailed below:

[0078] In S201, the first position of the obstacle to be matched is determined based on the obstacle information.

[0079] In S202, based on the information of each historical obstacle, the second position of each historical obstacle at the acquisition time is predicted; the acquisition time refers to the time when the radar acquires the data frame.

[0080] In S203, the target object is determined based on the first position and each of the second positions.

[0081] In this embodiment, since the obstacle information includes three-dimensional coordinate information, the vehicle-mounted terminal can determine the position of the obstacle to be matched at the time of data acquisition, i.e., the first position, based on this three-dimensional coordinate information. The time of data acquisition refers to the moment when the radar acquires the data frame.

[0082] Since historical obstacle information includes driving speed, the vehicle terminal can predict the position of each historical obstacle at the time of data collection, i.e., the second position, based on the driving speed and three-dimensional coordinate information of each historical obstacle.

[0083] Then, the vehicle terminal can calculate the difference between the first position and each of the second positions, and identify the historical obstacle corresponding to the second position with the smallest difference as the target object.

[0084] In one embodiment of this application, in order to further improve the accuracy of identifying the target object, the vehicle terminal can specifically determine the target object according to the following steps, detailed below:

[0085] Perform coordinate system transformation on the first position to obtain the third position in the vehicle coordinate system corresponding to the second position;

[0086] The differences between the third position and each of the second positions are calculated.

[0087] Generate a cost matrix based on each of the aforementioned difference values;

[0088] The target object is obtained by matching the cost matrix according to the Hungarian algorithm.

[0089] In this embodiment, since the obstacle to be matched is determined by data frames collected by the vehicle's radar and the target detection model, the three-dimensional coordinate information of the obstacle to be matched is in the radar coordinate system, meaning the first position is also in the radar coordinate system. However, each of the second positions is in the vehicle coordinate system. Therefore, the vehicle terminal needs to perform coordinate system transformation on the first position to obtain the third position of the first position in the vehicle coordinate system.

[0090] Then, the vehicle terminal can calculate the difference between the third position and each of the second positions.

[0091] In this embodiment, the difference value can be either Euclidean distance or IOU value.

[0092] Intersection over Union (IOU) is a metric for measuring the accuracy of object detection in a specific dataset. It measures the degree of matching between predicted and ground truth bounding boxes by calculating the ratio of the intersection to the union of the predicted and ground truth bounding boxes. IOU values ​​range from 0 to 1, with values ​​closer to 1 indicating a higher degree of overlap between the predicted and ground truth bounding boxes.

[0093] It should be noted that when there is only one obstacle to be matched, there is also only one first position and one third position; when there are multiple obstacles to be matched, there are multiple first positions and multiple third positions.

[0094] Next, the vehicle-mounted terminal can create an association matrix based on the third position and each of the second positions. Specifically, each row of this association matrix corresponds to a second position, and each column corresponds to a third position. The vehicle-mounted terminal can then write each difference value into the corresponding position in this association matrix, thereby obtaining the cost matrix.

[0095] In this embodiment, after obtaining the cost matrix, the vehicle terminal can perform matching processing on the cost matrix based on the Hungarian algorithm to obtain the second position that matches the third position, and determine the historical obstacle corresponding to the matched second position as the target object.

[0096] In practical applications, the Hungarian algorithm is a combinatorial optimization algorithm that solves the task allocation problem in polynomial time.

[0097] Understandably, during the process of matching the cost matrix based on the Hungarian algorithm, the vehicle terminal can obtain the matching relationship between the obstacle information converted to the third position and a certain tracking object in the tracking object set, and identify the tracking object with the matching relationship as the target object.

[0098] Based on this, the vehicle terminal can subsequently perform a category update operation on the tracked object that has a matching relationship with the obstacle information converted to the third location (such as step S103).

[0099] In S103, the category of the target object is updated based on the category information table of the target object and the obstacle information to obtain the target category of the target object; the category information table includes multiple set categories and the number of occurrences corresponding to each set category.

[0100] It should be noted that, in this embodiment of the application, during vehicle operation, the on-board terminal pre-creates a corresponding category information table for each tracked object (i.e., historical obstacle) in the tracked object set. The category information table includes multiple defined categories and the number of occurrences for each category.

[0101] The categories can be set according to actual needs, and there are no restrictions here.

[0102] It should be noted that the multiple setting categories include all categories of obstacles that the vehicle may encounter during actual operation.

[0103] In this embodiment of the application, after determining the target object, the vehicle terminal can update the category of the target object based on the category information table of the target object and the obstacle information, thereby obtaining the final target category of the target object.

[0104] In one embodiment of this application, in order to improve the accuracy of target category determination, the vehicle terminal can specifically use, as follows: Figure 3 Steps S301 to S303, which involve determining the target category of the target object, are detailed below:

[0105] In S301, the category of the obstacle to be matched is determined based on the obstacle information.

[0106] In this embodiment, the obstacle information may also include the obstacle category. Therefore, the vehicle terminal can directly determine the category of the obstacle to be matched based on the obstacle information.

[0107] It should be noted that in some possible embodiments, the target detection model in step S101 includes detecting the obstacle category corresponding to the data frame. Therefore, in this embodiment, the vehicle terminal can input the data frame collected by radar into the above-mentioned target detection model for processing, thereby obtaining obstacle information containing the category of the obstacle to be matched.

[0108] In S302, the category information table is updated according to the category of the obstacle to be matched to obtain the target information table.

[0109] In this embodiment, after determining the category of the obstacle to be matched, the vehicle terminal can search for a set category that is the same as the category of the obstacle in the category information table of the target object. Then, since the category information table includes the occurrence count of each set category, the vehicle terminal can increase the occurrence count of the same set category by 1, thereby completing the update operation of the category information table of the target object, and determining the updated category information table as the target information table.

[0110] In S303, the target category is determined based on the number of occurrences of each of the set categories in the target information table; each of the set categories includes the category of the obstacle to be matched.

[0111] In this embodiment, after obtaining the target information table, the vehicle-mounted terminal can determine the target category based on the frequency of occurrence of each set category in the target information table in order to improve the accuracy of target category determination. Each set category includes the category of the obstacle to be matched.

[0112] Specifically, the vehicle terminal can compare the occurrence frequency of each set category one by one, determine the maximum value of each occurrence frequency, and determine the target category based on the number of maximum values.

[0113] In one embodiment of this application, when the vehicle-mounted terminal detects multiple maximum values, it indicates that at least two of the multiple set categories have the same and maximum frequency of occurrence. Since it is impossible to determine whether one set category is more accurate than the other among the at least two set categories corresponding to the maximum value, in order to reduce the number of category jumps of the target object, the vehicle-mounted terminal can directly determine the historical category of the target object as the target category of the target object. Here, the historical category refers to the category determined for the target object at the moment preceding the acquisition time, i.e., the target object category. The acquisition time refers to the moment when the radar acquires the data frame.

[0114] In another embodiment of this application, when the number of maximum values ​​detected by the vehicle terminal is one, and the set category corresponding to the maximum value is a historical category, the vehicle terminal can continue to determine the historical category as the target category of the target object.

[0115] In another embodiment of this application, when the number of detected maximum values ​​is one, and the set category corresponding to the maximum value is not a historical category, the vehicle terminal can specifically do so by, as follows: Figure 4 The steps S401 to S402 shown indicate the target category of the target object.

[0116] As can be seen from the above, the category determination method provided in this application determines the obstacle information of the obstacle to be matched by using data frames collected by radar in the vehicle and a target detection model; it determines the target object matching the obstacle information from the tracking object set corresponding to the vehicle; the tracking object set includes multiple historical obstacles, each carrying historical obstacle information; and it updates the category of the target object based on the category information table of the target object and the obstacle information to obtain the target category of the target object; the category information table includes multiple set categories and the occurrence frequency of each set category. Compared with the prior art, which requires information from multiple different devices, this application only needs to acquire data collected by radar to achieve category determination, without the participation of multiple devices, thereby reducing costs and avoiding the complexity caused by category determination based on information from multiple different devices. Therefore, category determination can be achieved without high computing power, thereby improving update efficiency. At the same time, determining the category of the tracked object through a single sensor can prevent the category of the tracked object from changing due to changes in the detected category, thus avoiding affecting planning decisions and human-computer interaction.

[0117] Please see Figure 4 , Figure 4 This is a flowchart illustrating the implementation of a category determination method provided in another embodiment of this application. Relative to... Figure 3 In the corresponding embodiment, step S303 may specifically include steps S401 to S402, as detailed below:

[0118] In S401, if there is only one maximum value and the set category corresponding to the maximum value is not a historical category, then the occurrence status of the set category corresponding to the maximum value is determined.

[0119] In this embodiment, when the vehicle terminal detects that there is only one maximum value and the set category corresponding to the maximum value is not a historical category, it indicates that the category of the target object may need to change. Therefore, in order to further determine whether the category of the target object needs to change and to further improve the accuracy of the target category determination, the vehicle terminal can determine the occurrence status of the set category corresponding to the maximum value.

[0120] The occurrence states include, but are not limited to, continuous occurrence states and non-continuous occurrence states. Continuous occurrence states are used to describe the continuous occurrence of a set category, while non-continuous occurrence states are used to describe the non-continuous occurrence of a set category.

[0121] In some possible embodiments, the vehicle terminal may determine the occurrence status of a set category corresponding to the maximum value according to the following steps, detailed below:

[0122] If the number of occurrences of the set category corresponding to the maximum value within a set time period is greater than or equal to a set threshold, and the occurrence time within the set time period is continuous, then the occurrence state is determined to be the continuous occurrence state; wherein, the end time of the set time period is the acquisition time when the radar acquires the data frame;

[0123] If the number of occurrences of the set category corresponding to the maximum value within the set time period is less than the set threshold, or if the occurrence times within the set time period are not continuous, then the occurrence state is determined to be a discontinuous occurrence state.

[0124] It should be noted that the number of occurrences includes the time of occurrence.

[0125] In this embodiment, the vehicle-mounted terminal can detect whether the occurrence time of the set category corresponding to the maximum value is continuous within a set time period, and compare the number of occurrences of the set category corresponding to the maximum value within the set time period with a set threshold. The set threshold can be set according to the number of times the vehicle-mounted terminal updates the category within the set time period in actual application.

[0126] It should be noted that the end time of the set time period is the time when the radar collects the data frame.

[0127] The start time of the set time period can be determined based on the above-mentioned collection time and the duration for which any category continuously appears at a set threshold.

[0128] When the vehicle terminal detects that the number of times the set category corresponding to the maximum value appears within a set time period is greater than or equal to the set threshold, and the occurrence time within the set time period is continuous, it indicates that the set category has been continuously updated within the set time period. Therefore, the vehicle terminal can determine that the occurrence status of the set category corresponding to the maximum value is a continuous occurrence status.

[0129] When the vehicle terminal detects that the number of times the set category corresponding to the maximum value appears within a set time period is less than a set threshold, or that the appearance time within the set time period is not continuous, it indicates that the set category has not been continuously updated within the set time period. Therefore, the vehicle terminal can determine that the appearance status of the set category corresponding to the maximum value is a non-continuous appearance status.

[0130] In one embodiment of this application, when the vehicle terminal detects that the occurrence state of the set category corresponding to the maximum value is a continuous occurrence state, it can execute step S402.

[0131] In another embodiment of this application, when the vehicle terminal detects that the occurrence state of the set category corresponding to the maximum value is not continuous, it indicates that the set category corresponding to the maximum value has not appeared continuously within the set time period. In other words, the maximum value is unreliable. Therefore, in order to avoid the category of the target object from changing and to improve the accuracy of determining the target category of the target object, the vehicle terminal can determine the historical category of the target object as the target category.

[0132] In S402, if the occurrence state is a continuous occurrence state, then the set category corresponding to the maximum value is determined as the target category.

[0133] In this embodiment, when the vehicle terminal detects that the set category corresponding to the maximum value appears continuously, it means that the set category corresponding to the maximum value appears continuously within a set time period. In other words, the maximum value is reliable. Therefore, in order to improve the accuracy of determining the target category of the target object, the vehicle terminal can determine the set category corresponding to the maximum value as the target category of the target object.

[0134] As can be seen from the above, the category determination method provided in this embodiment further improves the accuracy of updating the category of the tracked object.

[0135] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0136] Corresponding to the category determination method described in the above embodiments, Figure 5 A schematic diagram of a category determination device provided in an embodiment of this application is shown. For ease of explanation, only the parts relevant to the embodiment of this application are shown. (Refer to...) Figure 5 The category determination device 500 includes: an information determination unit 51, a first object determination unit 52, and a first update unit 53. Wherein:

[0137] The information determination unit 51 is used to determine the obstacle information of the obstacle to be matched by using the data frames collected by the radar in the vehicle and the target detection model; the target detection model is trained by a pre-built deep learning model.

[0138] The first object determination unit 52 is used to determine a target object that matches the obstacle information from the tracking object set corresponding to the vehicle; the tracking object set includes multiple historical obstacles, and each historical obstacle carries historical obstacle information.

[0139] The first update unit 53 is used to update the category of the target object based on the category information table of the target object and the obstacle information to obtain the target category of the target object; the category information table includes multiple set categories and the number of occurrences corresponding to each set category.

[0140] In one embodiment of this application, the first object determination unit 52 specifically includes: a position determination unit, a prediction unit, and a second object determination unit. Wherein:

[0141] The location determination unit is used to determine the first location of the obstacle to be matched based on the obstacle information.

[0142] The prediction unit is used to predict the second position of each of the historical obstacles at the acquisition time based on the information of each of the historical obstacles; the acquisition time refers to the time when the radar acquires the data frame.

[0143] The second object determination unit is used to determine the target object based on the first position and each of the second positions.

[0144] In one embodiment of this application, the second object determination unit specifically includes: a conversion unit, a calculation unit, a generation unit, and a processing unit. Wherein:

[0145] The transformation unit is used to perform coordinate system transformation on the first position to obtain the third position in the vehicle coordinate system corresponding to the second position.

[0146] The calculation unit is used to calculate the difference between the third position and each of the second positions.

[0147] The generation unit is used to generate a cost matrix based on each of the difference values.

[0148] The processing unit is used to match the cost matrix according to the Hungarian algorithm to obtain the target object.

[0149] In one embodiment of this application, the first update unit 53 specifically includes: a first category determination unit, a second update unit, and a second category determination unit. Wherein:

[0150] The first category determination unit is used to determine the category of the obstacle to be matched based on the obstacle information.

[0151] The second update unit is used to update the category information table according to the category of the obstacle to be matched, so as to obtain the target information table.

[0152] The second category determination unit is used to determine the target category based on the number of occurrences of each of the set categories in the target information table; each of the set categories includes the category of the obstacle to be matched.

[0153] In one embodiment of this application, the second category determination unit specifically includes a third category determination unit.

[0154] The third category determination unit is used to determine the target category of the target object as the historical category if there are multiple maximum values ​​among all occurrences; the historical category refers to the target object category at the moment before the acquisition time; the acquisition time refers to the moment when the radar acquires the data frame.

[0155] In one embodiment of this application, the second category determination unit specifically includes: a first state determination unit and a fourth category determination unit. Wherein:

[0156] The first state determination unit is used to determine the occurrence state of the set category corresponding to the maximum value if the maximum value is one and the set category corresponding to the maximum value is not a historical category.

[0157] The fourth category determination unit is used to determine the set category corresponding to the maximum value as the target category if the occurrence state is a continuous occurrence state.

[0158] In one embodiment of this application, the category determination device 500 further includes a fifth category determination unit.

[0159] The fifth category determination unit is used to determine the target category of the target object as the historical category if the occurrence state is a non-continuous occurrence state.

[0160] In one embodiment of this application, the occurrence count carries an occurrence time; the first state determination unit specifically includes: a second state determination unit and a third state determination unit. Wherein:

[0161] The second state determination unit is used to determine the occurrence state as the continuous occurrence state if the number of occurrences of the set category corresponding to the maximum value within a set time period is greater than or equal to a set threshold, and the occurrence time within the set time period is continuous; wherein, the end time of the set time period is the acquisition time when the radar acquires the data frame.

[0162] The third state determination unit is used to determine the occurrence state as a discontinuous occurrence state if the number of occurrences of the set category corresponding to the maximum value within the set time period is less than the set threshold, or if the occurrence time within the set time period is not continuous.

[0163] In one embodiment of this application, the first category determination unit specifically includes: a sixth category determination unit.

[0164] The sixth category determination unit is used to determine the target category of the target object as the historical category if the maximum value is one and the set category corresponding to the maximum value is a historical category; the historical category refers to the target object category of the target object at the moment before the acquisition time.

[0165] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0166] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0167] Figure 6 This is a schematic diagram of the structure of a vehicle-mounted terminal provided in one embodiment of this application. Figure 6 As shown, the vehicle-mounted terminal 6 in this embodiment includes: at least one processor 60 ( Figure 6 (Only one is shown) a processor, a memory 61, and a computer program 62 stored in the memory 61 and executable on the at least one processor 60, which, when executing the computer program 62, implements the steps in any of the above-described categories of determination method embodiments.

[0168] The vehicle-mounted terminal may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art will understand that... Figure 6 This is merely an example of vehicle terminal 6 and does not constitute a limitation on vehicle terminal 6. It may include more or fewer components than shown in the figure, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0169] The processor 60 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0170] In some embodiments, the memory 61 may be an internal storage unit of the vehicle terminal 6, such as the RAM of the vehicle terminal 6. In other embodiments, the memory 61 may be an external storage device of the vehicle terminal 6, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the vehicle terminal 6. Furthermore, the memory 61 may include both internal and external storage units of the vehicle terminal 6. The memory 61 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory 61 can also be used to temporarily store data that has been output or will be output.

[0171] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0172] This application provides a computer program product that, when run on an in-vehicle terminal, enables the in-vehicle terminal to execute the steps described in the above-described method embodiments.

[0173] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to an in-vehicle terminal, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0174] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0175] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for determining a category, characterized in that, include: The obstacle information of the obstacle to be matched is determined by using data frames collected by the radar in the vehicle and the target detection model. The target detection model is trained from a pre-built deep learning model; A target object matching the obstacle information is determined from the set of tracked objects corresponding to the vehicle; the set of tracked objects includes multiple historical obstacles, each of which carries historical obstacle information; The category of the target object is updated based on the category information table of the target object and the obstacle information to obtain the target category of the target object; the category information table includes multiple set categories and the number of occurrences of each set category.

2. The category determination method as described in claim 1, characterized in that, The step of determining the target object matching the obstacle information from the set of tracked objects corresponding to the vehicle includes: Based on the obstacle information, the first position of the obstacle to be matched is determined; Based on the information of each historical obstacle, the second position of each historical obstacle at the time of acquisition is predicted; the time of acquisition refers to the moment when the radar acquires the data frame. The target object is determined based on the first position and each of the second positions.

3. The category determination method as described in claim 2, characterized in that, Determining the target object based on the first position and each of the second positions includes: Perform coordinate system transformation on the first position to obtain the third position in the vehicle coordinate system corresponding to the second position; The differences between the third position and each of the second positions are calculated. Generate a cost matrix based on each of the aforementioned difference values; The target object is obtained by matching the cost matrix according to the Hungarian algorithm.

4. The category determination method according to any one of claims 1-3, characterized in that, The step of updating the category of the target object based on the category information table of the target object and the obstacle information to obtain the target category of the target object includes: Based on the obstacle information, the category of the obstacle to be matched is determined; The category information table is updated according to the category of the obstacle to be matched to obtain the target information table; The target category is determined based on the number of occurrences of each of the defined categories in the target information table; each defined category includes the category of the obstacle to be matched.

5. The category determination method as described in claim 4, characterized in that, Determining the target category based on the frequency of occurrence of each of the defined categories in the target information table includes: If there are multiple maximum values ​​among all occurrences, then the target category of the target object is determined as the historical category; the historical category refers to the target object category at the moment before the acquisition time; the acquisition time refers to the moment when the radar acquires the data frame.

6. The category determination method as described in claim 4, characterized in that, Determining the target category based on the frequency of occurrence of each of the defined categories in the target information table includes: If there is only one maximum value, and the set category corresponding to the maximum value is not a historical category, then the occurrence status of the set category corresponding to the maximum value is determined. If the occurrence state is a continuous occurrence state, then the set category corresponding to the maximum value is determined as the target category.

7. The category determination method as described in claim 6, characterized in that, After determining the occurrence state of the set category corresponding to the maximum value, the method further includes: If the occurrence state is a non-continuous occurrence state, then the target category of the target object is determined as the historical category.

8. The category determination method as described in claim 6, characterized in that, The occurrence count carries the occurrence time; determining the occurrence status of the set category corresponding to the maximum value includes: If the number of occurrences of the set category corresponding to the maximum value within a set time period is greater than or equal to a set threshold, and the occurrence time within the set time period is continuous, then the occurrence state is determined to be the continuous occurrence state; wherein, the end time of the set time period is the acquisition time when the radar acquires the data frame; If the number of occurrences of the set category corresponding to the maximum value within the set time period is less than the set threshold, or if the occurrence times within the set time period are not continuous, then the occurrence state is determined to be a discontinuous occurrence state.

9. The category determination method as described in claim 4, characterized in that, Determining the target category based on the frequency of occurrence of each of the defined categories in the target information table includes: If there is only one maximum value, and the set category corresponding to the maximum value is a historical category, then the target category of the target object is determined as the historical category; the historical category refers to the target object category at the moment before the acquisition time.

10. A category determination device, characterized in that, include: The information determination unit is used to determine the obstacle information of the obstacle to be matched by using the data frames collected by the radar in the vehicle and the target detection model; The target detection model is trained from a pre-built deep learning model; The first object determination unit is used to determine a target object that matches the obstacle information from the tracking object set corresponding to the vehicle; the tracking object set includes multiple historical obstacles, and each historical obstacle carries historical obstacle information; The first update unit is used to update the category of the target object based on the category information table of the target object and the obstacle information to obtain the target category of the target object; the category information table includes multiple set categories and the number of occurrences of each set category.

11. A vehicle-mounted terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the category determination method as described in any one of claims 1 to 9.

12. A computer program product, characterized in that, It includes a computer program that, when run, implements the category determination method as described in any one of claims 1 to 9.

Images