Target fusion evaluation method and device, readable storage medium and electronic equipment

Abstract

Embodiments of the present disclosure disclose a target fusion evaluation method and device, a computer readable storage medium and an electronic device. The method comprises: obtaining at least two target sensing position information, the at least two target sensing position information being obtained by at least two obstacle recognition methods for a target obstacle respectively; fusing the at least two target sensing position information to obtain target fusion position information of the target obstacle; generating an evaluation information set of the target obstacle based on the target fusion position information; and displaying the evaluation information set. The embodiments of the present disclosure realize accurate and quantitative analysis of the effect of target fusion, and can display evaluation information, so that users can intuitively view the effect of target fusion, which is helpful for further improving the target fusion algorithm efficiently according to the evaluation information.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a target fusion evaluation method, apparatus, computer-readable storage medium, and electronic device. Background Technology

[0002] In fields such as autonomous driving and autonomous flight, environmental perception using various sensors is a crucial method. For example, signals collected by cameras and radar can be used for target detection. To achieve more accurate target detection, target-related information detected by cameras and radar can be fused. The complementary results from both methods can yield a more accurate target location. However, in some application scenarios of target fusion algorithms, the fusion effect often deviates from actual needs. For instance, in autonomous driving scenarios, sudden changes in the position or speed of obstacles determined by target fusion algorithms can pose risks to vehicle control, while these algorithms need to avoid such sudden changes. Therefore, it is necessary to evaluate the effectiveness of target fusion and improve the algorithm based on the evaluation results. However, currently, there is a lack of tools for quantitative analysis of target fusion effects, which hinders the iterative optimization of target fusion systems. Summary of the Invention

[0003] Embodiments of this disclosure provide a target fusion evaluation method, apparatus, computer-readable storage medium, and electronic device.

[0004] The embodiments of this disclosure provide a target fusion evaluation method, which includes: acquiring at least two target sensing position information, wherein the at least two target sensing position information are obtained by using at least two obstacle recognition methods for the target obstacle; fusing the at least two target sensing position information to obtain target fusion position information of the target obstacle; generating an evaluation information set of the target obstacle based on the target fusion position information; and displaying the evaluation information set.

[0005] According to another aspect of the present disclosure, a target fusion evaluation apparatus is provided, the apparatus comprising: a first acquisition module, configured to acquire at least two target sensing position information, wherein the at least two target sensing position information are obtained by using at least two obstacle recognition methods for the target obstacle; a fusion module, configured to fuse the at least two target sensing position information to obtain target fusion position information of the target obstacle; a first generation module, configured to generate an evaluation information set of the target obstacle based on the target fusion position information; and a first display module, configured to display the evaluation information set.

[0006] According to another aspect of the present disclosure, a computer-readable storage medium is provided, which stores a computer program for performing the above-described target fusion evaluation method.

[0007] According to another aspect of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; and a processor for reading executable instructions from the memory and executing the instructions to achieve the aforementioned objective fusion evaluation method.

[0008] Based on the target fusion evaluation method, apparatus, computer-readable storage medium, and electronic device provided in the above embodiments of this disclosure, by acquiring at least two target sensing position information obtained from at least two obstacle recognition methods, and then fusing the at least two target sensing position information to obtain target fusion position information of the target obstacle, and finally generating and displaying an evaluation information set of the target obstacle based on the target fusion position information, it is possible to accurately and quantitatively analyze the effect of target fusion, and at the same time display the evaluation information, so that users can intuitively view the effect of target fusion, which helps to further improve the target fusion algorithm efficiently based on the evaluation information.

[0009] The technical solutions of this disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0010] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0011] Figure 1 This is the system diagram to which this disclosure applies.

[0012] Figure 2 This is a flowchart illustrating an exemplary embodiment of the target fusion evaluation method provided in this disclosure.

[0013] Figure 3 This is a flowchart illustrating a target fusion evaluation method provided in another exemplary embodiment of this disclosure.

[0014] Figure 4 This is a flowchart illustrating a target fusion evaluation method provided in another exemplary embodiment of this disclosure.

[0015] Figure 5 This is a flowchart illustrating a target fusion evaluation method provided in another exemplary embodiment of this disclosure.

[0016] Figure 6 This is a flowchart illustrating a target fusion evaluation method provided in another exemplary embodiment of this disclosure.

[0017] Figure 7 This is an exemplary schematic diagram of a target fusion evaluation method provided in this disclosure, showing a set of location marker information on a display interface.

[0018] Figure 8 This is a flowchart illustrating a target fusion evaluation method provided in another exemplary embodiment of this disclosure.

[0019] Figure 9 This is a flowchart illustrating a target fusion evaluation method provided in another exemplary embodiment of this disclosure.

[0020] Figure 10 This is a schematic diagram of the structure of a target fusion evaluation device provided in an exemplary embodiment of the present disclosure.

[0021] Figure 11 This is a schematic diagram of the structure of a target fusion evaluation device provided in another exemplary embodiment of this disclosure.

[0022] Figure 12 This is a structural diagram of an electronic device provided in an exemplary embodiment of this disclosure. Detailed Implementation

[0023] Hereinafter, exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present disclosure, and not all embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described herein.

[0024] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0025] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of this disclosure are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them.

[0026] It should also be understood that in the embodiments disclosed herein, "a plurality of" may refer to two or more, and "at least one" may refer to one, two or more.

[0027] It should also be understood that any component, data or structure mentioned in the embodiments of this disclosure can generally be understood as one or more unless expressly defined or given to the contrary in the context.

[0028] Furthermore, the term "and / or" in this disclosure is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this disclosure generally indicates that the preceding and following related objects have an "or" relationship.

[0029] It should also be understood that the description of the various embodiments in this disclosure emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.

[0030] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0031] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0032] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0033] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0034] The embodiments disclosed herein can be applied to electronic devices such as terminal devices, computer systems, and servers, and can operate together with a wide range of other general-purpose or special-purpose computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, and servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems, etc.

[0035] Electronic devices such as terminal devices, computer systems, and servers can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are executed by remote processing devices linked through communication networks. In distributed cloud computing environments, program modules can reside on local or remote computing system storage media, including storage devices.

[0036] Application Overview

[0037] Current target fusion analysis primarily focuses on the actual performance of equipment (e.g., autonomous vehicles). For instance, if the vehicle's lateral and longitudinal control is poor, further analysis of target fusion issues is conducted. Target fusion analysis and debugging rely heavily on the experience and visual judgment of technical personnel, making it difficult to accurately and quantitatively analyze target fusion problems, resulting in low efficiency in optimizing target fusion algorithms.

[0038] To address this issue, this embodiment analyzes at least two target sensing location information and the target fusion location to obtain an evaluation information set of the target obstacle, and displays the evaluation information set, thereby achieving automatic evaluation of the target fusion effect and improving the debugging efficiency of target fusion.

[0039] Exemplary System

[0040] Figure 1 An exemplary system architecture 100 for a target fusion evaluation method or target fusion evaluation apparatus to which embodiments of the present disclosure may be applied is shown.

[0041] like Figure 1 As shown, system architecture 100 may include terminal device 101, network 102, server 103, and obstacle sensing device 104. Network 102 serves as the medium for providing a communication link between terminal device 101 and server 103. Network 102 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0042] The obstacle sensing device 104 is used to collect sensing information about a target object. For example, the obstacle sensing device 104 may include devices such as cameras and lidar to collect images, point cloud data, etc. The obstacle sensing device 104 can be connected to the terminal device 101 or connected to the server 103 via the network 102. The obstacle sensing device 104 can be installed on mobile devices such as vehicles, airplanes, and ships, or it can be installed in a fixed location.

[0043] Users can use terminal device 101 to interact with server 103 via network 102 to receive or send messages, etc. Various communication client applications can be installed on terminal device 101, such as monitoring applications, image processing applications, instant messaging tools, etc.

[0044] Terminal device 101 can be various electronic devices, including but not limited to mobile terminals such as vehicle terminals, mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), etc., as well as fixed terminals such as digital TVs, desktop computers, etc.

[0045] Server 103 can be a server that provides various services, such as a background data processing server that processes at least two target sensing location information uploaded by terminal device 101. The background data processing server can perform target fusion based on the received at least two target sensing location information to obtain fused location information, evaluation information on target obstacles, etc.

[0046] It should be noted that the target fusion evaluation method provided in the embodiments of this disclosure can be executed by the server 103 or by the terminal device 101. Accordingly, the target fusion evaluation device can be set in the server 103 or in the terminal device 101.

[0047] It should be understood that Figure 1 The number of terminal devices, networks, servers, and obstacle sensing devices shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, servers, and obstacle sensing devices can be included. For example, if the location information of at least two of the targets does not need to be obtained remotely, the above system architecture may exclude the network and servers, including only terminal devices and obstacle sensing devices.

[0048] Exemplary methods

[0049] Figure 2 This is a schematic flowchart of a target fusion evaluation method provided in an exemplary embodiment of this disclosure. This embodiment can be applied to electronic devices (such as...). Figure 1 On the terminal device 101 or server 103 shown, such as Figure 2 As shown, the method includes the following steps:

[0050] Step 201: Obtain the sensing location information of at least two targets.

[0051] In this embodiment, the electronic device can acquire at least two target sensing location information. These at least two target sensing location information are obtained for each target obstacle using at least two obstacle recognition methods. For example... Figure 1 The obstacle sensing device 104 shown may include devices such as cameras, lidar, and three-dimensional structured light emitters. Correspondingly, at least two obstacle recognition methods may include obstacle recognition methods based on image recognition, obstacle recognition methods based on point cloud data recognition, and obstacle recognition methods based on structured light recognition.

[0052] The aforementioned target sensing location information is determined within the same coordinate system. For example, this coordinate system could be the world coordinate system, the VCS coordinate system (Vehicle Coordinate System), etc. In addition to coordinate values, each target sensing location information may also include information such as the identifier of the obstacle sensing device.

[0053] Step 202: Fuse the target sensing position information of at least two targets to obtain the target fused position information of the target obstacle.

[0054] In this embodiment, the electronic device can fuse at least two target sensing position information to obtain target fused position information of the target obstacle. The target fused position information is determined in the same coordinate system as the aforementioned at least two target sensing position information.

[0055] The fusion of sensing location information from at least two targets can be achieved using existing multi-sensor target fusion algorithms. These algorithms overcome the uncertainties of obstacle localization by coordinating, combining, and complementing the sensing location information from multiple sensors, resulting in more accurate fused target location information than that from a single sensor. Examples of existing multi-sensor target fusion algorithms include: adaptive weighted fusion estimation algorithms for multi-sensor data, weighted fusion algorithms based on the least squares principle, and trust-based multi-sensor data fusion algorithms.

[0056] Step 203: Based on the target fusion location information, generate a set of evaluation information for the target obstacle.

[0057] In this embodiment, the electronic device can generate an evaluation information set of the target obstacle based on the target fusion location information. The evaluation information set represents the effect of target fusion. As an example, the evaluation information set may include information representing the error between the target fusion location information and at least two target sensing location information, statistical information on the target fusion location information, statistical information on the at least two target sensing location information, and information representing the motion state of the target obstacle, etc.

[0058] Step 204: Display the evaluation information set.

[0059] In this embodiment, the electronic device can display a set of evaluation information on a preset display interface. Typically, the display interface can include at least one preset area, each area used to display corresponding type of evaluation information. For example, location markers can be generated based on real-time determined target fusion location information and displayed on a coordinate display interface; alternatively, a curve statistically analyzed for target fusion location information over a preset historical time period can be generated and displayed on a curve display interface.

[0060] The method provided in the above embodiments of this disclosure acquires at least two target sensing position information obtained by at least two obstacle recognition methods, then fuses the at least two target sensing position information to obtain target fused position information of the target obstacle, and finally generates and displays a set of evaluation information of the target obstacle based on the target fused position information. This achieves accurate and quantitative analysis of the target fusion effect, and can display the evaluation information, allowing users to intuitively view the target fusion effect, which helps to further improve the target fusion algorithm efficiently based on the evaluation information.

[0061] In some alternative implementations, such as Figure 3 As shown, step 203 may include:

[0062] Step 2031: Based on the target fusion location information, determine at least one motion state information of the target obstacle.

[0063] The motion state information can be determined based on the target fusion position information at the current moment and the target fusion position information recorded at the previous moment. As an example, at least one piece of motion state information may include, but is not limited to, at least one of the following: the longitudinal movement velocity, lateral movement velocity, longitudinal acceleration, and lateral acceleration of the target obstacle. Given that the target fusion position information at the current moment and the target fusion position information recorded at previous moments are already obtained, and the coordinate system of the target fusion position information (e.g., the world coordinate system) has been determined, the aforementioned velocity, acceleration, and other motion state information can be determined using known techniques.

[0064] The terms "lateral" and "longitudinal" are defined relative to the direction of movement of the target obstacle. For example, lateral movement speed refers to the speed at which the target obstacle moves left or right, while longitudinal movement speed refers to the speed at which the target obstacle moves forward or backward.

[0065] Step 2032: Obtain the set of historical motion state information generated in the first historical time period corresponding to each motion state information in at least one motion state information.

[0066] The first historical time period can be a time interval between a point in time prior to the current time and the current time itself. The duration of this time interval can be arbitrarily set, such as 20 seconds, 60 seconds, etc. Each motion state information corresponds to a set of historical motion state information. For example, for longitudinal velocity, it corresponds to a set of multiple longitudinal velocities generated within a first historical time period.

[0067] Step 2033: Perform statistics on each motion state information and the corresponding set of historical motion state information in at least one motion state information to obtain at least one statistical information.

[0068] Specifically, each of the at least one statistical information mentioned above corresponds to a motion state information and a set of historical motion state information. For a certain motion state information, statistics are performed on the motion state information and its corresponding set of historical motion state information to obtain the statistical information corresponding to the motion state information.

[0069] As an example, motion state information can be the longitudinal movement speed of a target obstacle, and the corresponding set of historical motion state information is the set of multiple longitudinal movement speeds recorded within the first historical time period. Optionally, at least one statistical information may include, but is not limited to, at least one of the following: average longitudinal movement speed, average lateral movement speed, average longitudinal acceleration, average lateral acceleration, maximum longitudinal movement speed, maximum lateral movement speed, maximum longitudinal acceleration, maximum lateral acceleration, etc., as well as the standard deviations of lateral and longitudinal velocities, and the standard deviations of lateral and longitudinal accelerations, etc.

[0070] Step 2034: Generate an evaluation information set based on at least one statistical information.

[0071] Specifically, at least one statistical information obtained can be determined as the evaluation information set, or the statistical information selected by the user can be extracted from at least one statistical information as the evaluation information included in the evaluation information set, or statistical information that meets preset conditions can be determined from at least one statistical information as the evaluation information included in the evaluation information set.

[0072] Optionally, motion state information corresponding to each target sensing position information can be determined based on the above at least two target sensing position information. Then, a set of historical motion state information corresponding to each target sensing position information (i.e., a set of motion state information determined by the original sensing position information) can be obtained. The motion state information corresponding to each target sensing position information and the historical motion state information can be statistically analyzed. The obtained statistical information can be included in at least one of the statistical information used in step 2034 above. This allows for a comparison between motion states without target fusion and motion states with target fusion, which helps to quantify the effect of target fusion.

[0073] This embodiment statistically analyzes the motion state information of each target obstacle and the corresponding historical motion state information set, and generates an evaluation information set based on the statistical information. This enables accurate monitoring of the motion state of the target obstacle, helps to accurately analyze the target fusion position information based on the statistical information, intuitively evaluate the stability and quantitative indicators of target fusion, and thus efficiently improve the target fusion algorithm.

[0074] In some alternative implementations, step 204 can be performed as follows:

[0075] In response to determining that at least one statistical information contains target statistical information that meets a preset first threshold condition, a first prompt message corresponding to the target statistical information is displayed.

[0076] Typically, each of the above-mentioned statistical information corresponds to a numerical range. If the value included in one of the statistical information exceeds the corresponding numerical range, then the statistical information is determined to meet the corresponding first threshold condition, and the corresponding first prompt information is generated and displayed. For example, if the target obstacle is a pedestrian, its maximum lateral and longitudinal velocity does not exceed 6 m / s, and its lateral and longitudinal acceleration does not exceed 3 m / s². 2 If the target obstacle is a bicycle, its maximum lateral and longitudinal velocity shall not exceed 10 m / s, and its lateral and longitudinal acceleration shall not exceed 8 m / s². 2 If the target obstacle is a car, its maximum longitudinal velocity shall not exceed 40 m / s, and its longitudinal acceleration shall not exceed 6 m / s². 2 The longitudinal deceleration shall not exceed -10 m / s², the lateral velocity shall not exceed 4 m / s, and the longitudinal and lateral accelerations shall not exceed 5 m / s². 2 .

[0077] If a statistical information item is the longitudinal velocity of a target obstacle, and the longitudinal velocity exceeds the preset velocity range corresponding to the type of the target obstacle, it indicates that the motion state of the target obstacle is abnormal. At this time, the target fusion position information may be calculated incorrectly, and a corresponding first prompt message will be generated. The first prompt message may include text such as "Longitudinal velocity abnormal" and time information such as the time when the longitudinal velocity exceeds the preset velocity range.

[0078] This embodiment displays corresponding prompts when the motion state of a target obstacle is abnormal, allowing users to promptly view abnormal motion conditions and effectively monitor the target fusion effect.

[0079] In some alternative implementations, such as Figure 4 As shown, step 203 may include:

[0080] Step 2035: Based on at least two target sensing position information and target fusion position information, generate fusion error information corresponding to each target sensing position information.

[0081] The fusion error information represents the difference between the fused target location information and the sensed target location information. For example, if the sensed target location information includes coordinates (x1, y1), and the distance from the origin is... The target fusion location information includes coordinates (x2, y2), and the distance from this coordinate to the origin is... The fusion error information corresponding to the target sensing position information can include the fusion error ratio E = |L1-L2| / L1*100%.

[0082] Step 2036: Based on the generated at least two fusion error information sets, generate an evaluation information set for the target obstacle.

[0083] Specifically, at least two generated fusion error pieces of information can be identified as evaluation information included in the evaluation information set, and the at least two fusion error pieces of information can be displayed. Alternatively, the average value of the fusion error ratios included in the at least two fusion error pieces of information can be calculated, and the fusion error ratios including the average value can be displayed.

[0084] This embodiment generates fusion error information corresponding to the sensing position information of each target, which can be used as an indicator for target fusion evaluation. This allows users to intuitively analyze the effect of target fusion by viewing the fusion error information.

[0085] In some alternative implementations, step 204 can be performed as follows:

[0086] In response to determining that at least two fusion error messages meet a preset second threshold condition, a second prompt message indicating that the target fusion error is too large is displayed.

[0087] Typically, each of the at least two fusion error pieces of information corresponds to a numerical range. If the fusion error ratio included in one of the fusion error pieces of information exceeds the corresponding numerical range, then the fusion error piece of information is determined to meet the second threshold condition, and a corresponding second prompt message is generated and displayed. For example, if the fusion error ratio included in a certain fusion error piece of information exceeds 15%, then the fusion error piece of information is determined to meet the second threshold condition, and a second prompt message is generated. The second prompt message may include text such as "fusion error too large," or it may highlight the fusion error piece of information as the second prompt message.

[0088] This embodiment displays corresponding prompts when abnormal fusion error information occurs, allowing users to promptly view the abnormal fusion error information and effectively monitor the target fusion effect.

[0089] In some alternative implementations, such as Figure 5 As shown, step 204 may include:

[0090] Step 2041: Obtain at least two sets of historical fusion error information generated in the second historical time period.

[0091] In this context, each of the at least two sets of historical fusion error information is calculated within the second historical time period based on the corresponding target sensing location information and target fusion location information. That is, each obstacle recognition method corresponds to one set of historical fusion error information. The method for calculating the historical fusion error information is the same as the method for calculating the fusion error information in the optional embodiments described above, and will not be repeated here.

[0092] The second historical time period can be a time interval between a point in time prior to the current time and the current time. The duration of this interval can be set arbitrarily, such as 20 seconds, 60 seconds, etc. The second historical time period can be the same as or different from the first historical time period mentioned above.

[0093] Step 2042: In response to determining that at least two sets of historical fusion error information meet the preset third threshold condition, a third prompt message indicating that the target fusion error of the second historical time period is too large is displayed.

[0094] Typically, a numerical range can be preset. If the fusion error ratio included in a certain historical fusion error information exceeds this numerical range, a third prompt message can be generated based on the time point corresponding to that historical fusion error information. For example, if the fusion error ratio included in a certain historical fusion error information exceeds 15%, it is determined that the fusion error information meets the third threshold condition, and a third prompt message is generated. This third prompt message may include the time information corresponding to the historical fusion error information that exceeds the numerical range, or it may include the highlighted fusion error information.

[0095] This embodiment generates and displays a third prompt when the set of historical fusion error information generated in the second historical time period meets the third threshold condition. This can help to quickly filter out the historically abnormal fusion error information and the corresponding time, which helps to analyze the effect of target fusion over a period of time, and thus facilitates the improvement of the target fusion algorithm.

[0096] In some alternative implementations, such as Figure 6 As shown, after step 202, the method further includes:

[0097] Step 601: Generate a set of location marker information to be displayed based on at least two target sensing location information and target fused location information.

[0098] The location marker information set includes at least one of the following: sensing location marker information corresponding to at least two target sensing location information respectively, fusion location marker information corresponding to target fusion location information, association marker information indicating the association relationship between target fusion location information and at least two target sensing location information, and obstacle identification information corresponding to sensing location markers and fusion location markers respectively.

[0099] Step 602: Based on at least two target sensing position information and target fusion position information, determine the first display position of each position marker information in the position marker information set on the display interface containing the pre-built coordinate system.

[0100] The coordinate system mentioned above can be preset, for example, it can be the world coordinate system or the VCS coordinate system. The first display position is determined based on the coordinates included in the two target sensing position information and the target fused position information.

[0101] Step 603: Display the corresponding position marker information at each of the determined first display positions.

[0102] like Figure 7 The diagram illustrates an exemplary schematic of a set of location marker information displayed on the aforementioned display interface. The origin of the coordinate system of this display interface is the location of the vehicle equipped with a camera and LiDAR. The x-axis and y-axis represent the vehicle's forward / backward and left / right directions, respectively. 701 and 702 are sensing location marker information, with 701 corresponding to a target sensing location detected based on point cloud data, and 702 corresponding to a target sensing location obtained based on image recognition. 703 is fused location marker information. The arrowed lines connecting 701 and 703, and between 702 and 703, represent association marker information. Their function is to distinguish between related sensing location marker information and fused location marker information among multiple sensing location marker information and multiple fused location marker information obtained from detecting multiple target obstacles. The numbers "35", "24", and "125" represent the obstacle identification information corresponding to sensing location marker information 701, 702, and fused location marker information 703, respectively.

[0103] This embodiment allows users to intuitively view the positional relationship between target sensing position information and target fusion position information by displaying a set of position marker information on the display interface. This facilitates users in viewing the effect of target fusion and helps improve the efficiency of analyzing the structure of target fusion.

[0104] In some alternative implementations, such as Figure 8 As shown, the method may further include:

[0105] Step 801: Obtain the fusion error information corresponding to each of the at least two target sensing information.

[0106] Here, the fusion error information represents the error between the corresponding target sensing information and the target fused position information. It should be noted that the fusion error information in this embodiment can be pre-calculated or calculated in real time. If the fusion error information has already been calculated before this step, it can be directly obtained; otherwise, the above can be referred to. Figure 4 The method provided in the corresponding optional embodiment calculates the fusion error information.

[0107] Step 802: Based on at least two target sensing position information and target fusion position information, determine the second display position corresponding to each fusion error information on the display interface.

[0108] The second display position is related to the position of each sensing position marker information. For example, the second display position can be the area where the aforementioned associated marker information is located.

[0109] Step 803: Display the corresponding fusion error information based on each determined second display position.

[0110] like Figure 7 As shown, the display area for the two associated marker information connected by arrows is the second display position. The fusion error information, including the fusion error ratios "16.8" and "14.8", is displayed on the two arrow lines respectively.

[0111] It should be noted that the steps included in this embodiment can be described above. Figure 6 The steps included in the corresponding embodiments can be executed before or after the fusion error information is displayed on the display interface first, and then other information is displayed, or other information is displayed first and then the fusion error information is displayed.

[0112] This embodiment displays the fusion error information corresponding to the sensing position information of each target on the display interface, allowing users to intuitively view the relationship between the fusion error information and the sensing position information of the target. This further facilitates users in viewing the effect of target fusion and improves the efficiency of analyzing the structure of target fusion.

[0113] In some alternative implementations, such as Figure 9 As shown, after step 202, the method further includes:

[0114] Step 901: Generate and display description information of the target obstacle.

[0115] The descriptive information describes the target obstacle after it has been located. Typically, the descriptive information exists in list form, including the identifier, type, and merged location information for each target obstacle. The descriptive information can be displayed on a pre-defined descriptive information display interface, which can be a fixed location on the screen or an interface triggered by clicks, touches, or other actions.

[0116] This embodiment generates and displays descriptive information about the target obstacle, allowing users to intuitively view the basic information of the target obstacle after target fusion. This facilitates users to quickly view the position, movement status, and other information of the target obstacle through the descriptive information, improving the efficiency of quantitative analysis of the target fusion effect.

[0117] In some alternative implementations, such as Figure 9 As shown, after step 901, the method further includes:

[0118] Step 902: In response to triggering a viewing operation on the description information, acquire at least one set of historical motion state information of the target obstacle.

[0119] The aforementioned viewing operations can include clicking, swiping, and inputting commands. For example, a user can click on the displayed description information of the target obstacle to obtain at least one set of historical motion state information for the target obstacle. Each set of historical motion state information represents a motion state of the target obstacle, which can be motion state information determined based on historical target fusion information (e.g., the historical longitudinal velocity and historical longitudinal acceleration of the target obstacle determined by historical target fusion position information), or motion state information determined based on at least two target sensing position information (e.g., the historical longitudinal velocity and historical longitudinal acceleration of the target obstacle determined by image recognition, point cloud recognition, etc.).

[0120] Step 903: Generate and display at least one curve corresponding to at least one set of historical motion state information.

[0121] Typically, a user can manually select one or more sets of historical motion state information to be displayed from at least one set of historical motion state information. The electronic device can then generate and display curves of the selected historical motion state information. Examples include historical longitudinal velocity curves, historical lateral velocity curves, historical longitudinal acceleration curves, and historical lateral acceleration curves of a target obstacle determined based on target fusion location information; or historical longitudinal velocity curves, historical lateral velocity curves, historical longitudinal acceleration curves, and historical lateral acceleration curves determined based on at least two target sensing location information. Optionally, if the obstacle sensing device used in the above at least two obstacle recognition methods is in motion, motion state curves such as speed and acceleration of the tool carrying the obstacle sensing device (e.g., vehicle, drone, etc.) can be generated and displayed.

[0122] Optionally, the horizontal and vertical axes in the curve display interface can be scaled, and the curve can also be quantitatively analyzed by the user. For example, the error can be compared and the trend can be analyzed between the motion state obtained based on target fusion and the motion state obtained based on the original data collected by the obstacle sensing device.

[0123] This embodiment displays at least one set of curves corresponding to historical motion state information, thereby providing an intuitive display of the changes in historical motion state information. This helps users to efficiently analyze and improve the effect of target fusion based on the displayed curves.

[0124] Exemplary device

[0125] Figure 10 This is a schematic diagram of the structure of a target fusion evaluation device provided in an exemplary embodiment of this disclosure. This embodiment can be applied to electronic devices, such as... Figure 10 As shown, the target fusion evaluation device includes: a first acquisition module 1001, used to acquire at least two target sensing position information, wherein the at least two target sensing position information are obtained by using at least two obstacle recognition methods for the target obstacle; a fusion module 1002, used to fuse the at least two target sensing position information to obtain target fusion position information of the target obstacle; a first generation module 1003, used to generate an evaluation information set of the target obstacle based on the target fusion position information; and a first display module 1004, used to display the evaluation information set.

[0126] In this embodiment, the first acquisition module 1001 can acquire at least two target sensing location information. These at least two target sensing location information are obtained for each target obstacle using at least two obstacle recognition methods. For example... Figure 1The obstacle sensing device 104 shown may include devices such as cameras, lidar, and three-dimensional structured light emitters. Correspondingly, at least two obstacle recognition methods may include obstacle recognition methods based on image recognition, obstacle recognition methods based on point cloud data recognition, and obstacle recognition methods based on structured light recognition.

[0127] The aforementioned target sensing location information is determined within the same coordinate system. For example, this coordinate system could be the world coordinate system, the VCS coordinate system (Vehicle Coordinate System), etc. In addition to coordinate values, each target sensing location information may also include information such as the identifier of the obstacle sensing device.

[0128] In this embodiment, the fusion module 1002 can fuse at least two target sensing position information to obtain target fused position information of the target obstacle. The target fused position information includes information determined in the same coordinate system as the aforementioned at least two target sensing position information.

[0129] The fusion of sensing location information from at least two targets can be achieved using existing multi-sensor target fusion algorithms. These algorithms overcome the uncertainties of obstacle localization by coordinating, combining, and complementing the sensing location information from multiple sensors, resulting in more accurate fused target location information than that from a single sensor. Examples of existing multi-sensor target fusion algorithms include: adaptive weighted fusion estimation algorithms for multi-sensor data, weighted fusion algorithms based on the least squares principle, and trust-based multi-sensor data fusion algorithms.

[0130] In this embodiment, the first generation module 1003 can generate an evaluation information set of the target obstacle based on the target fusion location information. The evaluation information set is used to represent the effect of target fusion. As an example, the evaluation information set may include information representing the error between the target fusion location information and at least two target sensing location information, statistical information on the target fusion location information, statistical information on the at least two target sensing location information, information representing the motion state of the target obstacle, etc.

[0131] In this embodiment, the first display module 1004 can display the evaluation information set on a preset display interface. Typically, the display interface can include at least one preset area, each area used to display evaluation information of a corresponding type. For example, location markers can be generated based on real-time determined target fusion location information and displayed on a coordinate display interface, or a curve statistically analyzed for target fusion location information over a preset historical time period can be generated and displayed on a curve display interface.

[0132] Reference Figure 11, Figure 11 This is a schematic diagram of the structure of a target fusion evaluation device provided in another exemplary embodiment of this disclosure.

[0133] In some optional implementations, the first generation module 1003 includes: a first determining unit 10031, used to determine at least one motion state information of a target obstacle based on the target fusion location information; a first acquiring unit 10032, used to acquire a set of historical motion state information generated in a first historical time period corresponding to each of the at least one motion state information; a statistics unit 10033, used to perform statistics on each of the at least one motion state information and the corresponding set of historical motion state information to obtain at least one statistical information; and a first generation unit 10034, used to generate an evaluation information set based on the at least one statistical information.

[0134] In some optional implementations, the first display module 1004 includes: a first display unit 10041, which is used to display a first prompt message corresponding to the target statistical information in response to determining that at least one statistical information contains target statistical information that meets a preset first threshold condition.

[0135] In some optional implementations, the first generation module 1003 includes: a second generation unit 10035, used to generate fusion error information corresponding to each target sensing position information based on at least two target sensing position information and target fusion position information; and a third generation unit 10036, used to generate an evaluation information set of target obstacles based on the generated at least two fusion error information.

[0136] In some alternative implementations, the first display module 1004 includes a second display unit 10042, which displays a second prompt message indicating that the target fusion error is too large in response to determining that at least two fusion error messages meet a preset second threshold condition.

[0137] In some optional implementations, the first display module 1004 includes: a second acquisition unit 10043, used to acquire at least two sets of historical fusion error information generated in the second historical time period, wherein each set of historical fusion error information is calculated for the corresponding target sensing position information and target fusion position information in the second historical time period; and a third display unit 10044, used to display a third prompt message indicating that the target fusion error in the second historical time period is too large in response to determining that at least two sets of historical fusion error information meet a preset third threshold condition.

[0138] In some optional implementations, the device further includes: a second generation module 1005, configured to generate a set of location marker information to be displayed based on at least two target sensing location information and target fusion location information, wherein the set of location marker information includes at least one of the following: sensing location marker information corresponding to at least two target sensing location information respectively, fusion location marker information corresponding to target fusion location information, association marker information indicating the association relationship between target fusion location information and at least two target sensing location information, and obstacle identification information corresponding to sensing location markers and fusion location markers respectively; a first determination module 1006, configured to determine a first display position of each location marker information in the set of location marker information on a display interface containing a pre-built coordinate system based on at least two target sensing location information and target fusion location information; and a second determination module 1007, configured to display the corresponding location marker information at each determined first display position.

[0139] In some optional implementations, the device further includes: a second acquisition module 1008, configured to acquire fusion error information corresponding to each of the at least two target sensing information, wherein the fusion error information represents the error between the corresponding target sensing information and the target fusion position information; a third determination module 1009, configured to determine, on the display interface, a second display position corresponding to each fusion error information based on the at least two target sensing position information and the target fusion position information; and a second display module 1010, configured to display the corresponding fusion error information based on the determined second display positions.

[0140] In some alternative implementations, the device further includes a third generation module 1011 for generating and displaying descriptive information about the target obstacle.

[0141] In some alternative implementations, the device further includes: a third acquisition module 1012, configured to acquire at least one set of historical motion state information of the target obstacle in response to triggering a viewing operation on the description information; and a third display module 1013, configured to generate and display at least one curve corresponding to each of the at least one set of historical motion state information.

[0142] The target fusion evaluation device provided in the above embodiments of this disclosure acquires at least two target sensing position information obtained by at least two obstacle recognition methods, then fuses the at least two target sensing position information to obtain target fusion position information of the target obstacle, and finally generates and displays a set of evaluation information of the target obstacle based on the target fusion position information. This achieves accurate and quantitative analysis of the target fusion effect, and can display the evaluation information so that users can intuitively view the target fusion effect, which helps to further improve the target fusion algorithm efficiently based on the evaluation information.

[0143] Exemplary electronic devices

[0144] Below, for reference Figure 12 To describe an electronic device according to embodiments of the present disclosure. The electronic device may be as follows: Figure 1 The terminal device 101 and server 103 shown, or either one or both, or a standalone device independent of them, can communicate with the terminal device 101 and server 103 to receive the collected input signals from them.

[0145] Figure 12 A block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0146] like Figure 12 As shown, the electronic device 1200 includes one or more processors 1201 and a memory 702.

[0147] The processor 1201 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 1200 to perform desired functions.

[0148] The memory 702 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1201 may execute the program instructions to implement the target fusion evaluation methods of the various embodiments of this disclosure described above and / or other desired functions. Various contents such as target sensing location information and target fusion location information may also be stored in the computer-readable storage medium.

[0149] In one example, the electronic device 1200 may also include an input device 1203 and an output device 1204, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0150] For example, when the electronic device is a terminal device 101 or a server 103, the input device 1203 can be a camera, lidar, or other device used to input target sensing location information. When the electronic device is a standalone device, the input device 1203 can be a communication network connector used to receive the input target sensing location information from the terminal device 101 and the server 103.

[0151] The output device 1204 can output various information to the outside, including a set of generated evaluation information. The output device 1204 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0152] Of course, for the sake of simplicity, Figure 12 Only some of the components of the electronic device 1200 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 1200 may include any other suitable components depending on the specific application.

[0153] Exemplary computer program products and computer-readable storage media

[0154] In addition to the methods and apparatus described above, embodiments of this disclosure may also be computer program products comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the target fusion evaluation methods according to various embodiments of this disclosure as described in the "Exemplary Methods" section of this specification.

[0155] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0156] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions that, when executed by a processor, cause the processor to perform the steps in the target fusion evaluation method according to various embodiments of this disclosure as described in the "Exemplary Methods" section above.

[0157] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0158] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0159] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0160] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0161] The methods and apparatus of this disclosure may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of this disclosure are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, this disclosure may also be implemented as a program recorded on a recording medium, the program including machine-readable instructions for implementing the methods according to this disclosure. Thus, this disclosure also covers recording media storing programs for performing the methods according to this disclosure.

[0162] It should also be noted that in the apparatus, devices, and methods of this disclosure, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions to this disclosure.

[0163] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0164] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A target fusion evaluation method, comprising: obtaining at least two target sensing position information, the at least two target sensing position information being obtained by at least two obstacle recognition manners for a target obstacle respectively; fusing the at least two target sensing position information to obtain target fusion position information of the target obstacle; generating an evaluation information set of the target obstacle based on the target fusion position information; displaying the evaluation information set; wherein the generating the evaluation information set of the target obstacle based on the target fusion position information comprises: determining at least one motion state information of the target obstacle based on the target fusion position information; obtaining a historical motion state information set generated in a first historical time period corresponding to each of the at least one motion state information; statistically processing each of the at least one motion state information and the corresponding historical motion state information set respectively to obtain at least one statistical information; generating the evaluation information set based on the at least one statistical information.

2. The method of claim 1, wherein, The displaying the evaluation information set comprises: in response to determining that there is a target statistical information meeting a preset first threshold condition in the at least one statistical information, displaying first prompt information corresponding to the target statistical information.

3. The method according to any one of claims 1-2, wherein, The generating the evaluation information set of the target obstacle based on the target fusion position information further comprises: generating fusion error information corresponding to each target sensing position information based on the at least two target sensing position information and the target fusion position information; generating the evaluation information set of the target obstacle based on the generated at least two fusion error information; The generating the evaluation information set of the target obstacle based on the generated at least two fusion error information comprises: determining the generated at least two fusion error information as evaluation information included in the evaluation information set.

4. The method of claim 3, wherein, The displaying the evaluation information set comprises: in response to determining that the at least two fusion error information meet a preset second threshold condition, displaying second prompt information indicating that the target fusion error is too large.

5. The method of claim 3, wherein, The displaying the evaluation information set comprises: obtaining at least two groups of historical fusion error information sets generated in a second historical time period, wherein each of the at least two groups of historical fusion error information sets is calculated based on corresponding target sensing position information and target fusion position information in the second historical time period; in response to determining that the at least two groups of historical fusion error information sets meet a preset third threshold condition, displaying third prompt information indicating that the target fusion error in the second historical time period is too large.

6. The method of claim 1, wherein, After the obtaining the target fusion position information of the target obstacle, the method further comprises: generate a set of position marker information to be displayed based on the at least two target sensing position information and the target fusion position information, wherein the set of position marker information comprises at least one of the following: sensing position marker information corresponding to each of the at least two target sensing position information, fusion position marker information corresponding to the target fusion position information, association marker information representing an association relationship between the target fusion position information and the at least two target sensing position information, and obstacle identification information corresponding to the sensing position marker and the fusion position marker, respectively; determine first display positions of each of the set of position marker information on a display interface comprising a pre-constructed coordinate system based on the at least two target sensing position information and the target fusion position information; display the corresponding position marker information at the determined first display positions.

7. The method of claim 6, after the target fusion position information of the target obstacle is obtained, the method further comprises: obtain fusion error information corresponding to each of the at least two target sensing position information, wherein the fusion error information represents an error between the corresponding target sensing position information and the target fusion position information; determine second display positions corresponding to each of the fusion error information on the display interface based on the at least two target sensing position information and the target fusion position information; display the corresponding fusion error information based on the determined second display positions.

8. The method of claim 1, wherein, after the target fusion position information of the target obstacle is obtained, the method further comprises: generate and display description information of the target obstacle.

9. The method of claim 8, wherein, after the description information is displayed, the method further comprises: in response to triggering a viewing operation on the description information, obtain at least one set of historical motion state information of the target obstacle; generate and display at least one curve corresponding to at least one set of the historical motion state information, respectively.

10. A target fusion evaluation device, comprising: a first obtaining module configured to obtain at least two target sensing position information, wherein the at least two target sensing position information is obtained by using at least two obstacle recognition methods on a target obstacle, respectively; a fusion module configured to fuse the at least two target sensing position information to obtain target fusion position information of the target obstacle; a first generating module configured to generate a set of evaluation information of the target obstacle based on the target fusion position information; a first display module configured to display the set of evaluation information. The first generation module comprises: a first determination unit configured to determine at least one motion state information of the target obstacle based on the target fusion position information; a first acquisition unit configured to acquire a set of historical motion state information corresponding to each of the at least one motion state information and generated in a first historical time period; a statistical unit configured to respectively perform statistics on each of the at least one motion state information and the corresponding set of historical motion state information to obtain at least one statistical information; and a first generation unit configured to generate the set of evaluation information based on the at least one statistical information. 11.A computer readable storage medium, the storage medium storing a computer program, the computer program being configured to execute the method of any one of claims 1-9. 12.An electronic device, the electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the method of any one of claims 1-9.

Images