Method and device for determining target state information, medium, equipment and program product

Abstract

The invention discloses a target state information determination method and device, a medium, equipment and a program product, and the method comprises the steps: obtaining first state information of a target at a first moment and second state information of a second moment observed through different observation modes, and enabling the second moment to be earlier than the first moment; for any observation mode, determining a first sub-weight based on the first state information and the second state information; determining a second sub-weight based on the first state information; for any observation mode, based on the first sub-weight and the second sub-weight, determining a fusion weight; and determining the state information of the target at the first moment based on the first state information observed in different observation modes and the corresponding fusion weight. According to the embodiment of the invention, the fusion weight is adaptively determined through the state information observed in any observation mode, and the reliability of the state information obtained through fusion is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent driving technology, and in particular to a method, apparatus, medium, device, and program product for determining target state information. Background Art

[0002] In the field of intelligent driving, vehicles are usually equipped with multiple sensors, such as visual sensors and lidar, to observe the vehicle's driving environment and then estimate the state of surrounding dynamic targets, providing an accurate basis for downstream tasks such as target behavior prediction, target trajectory prediction, and vehicle driving path planning.

[0003] Due to the different installation locations and sensor parameters of different sensors, the observation values ​​of the same target in the real physical world observed by different sensors are different, which leads to errors in the state estimation of the target, thereby affecting the execution of downstream tasks. Summary of the Invention

[0004] In order to solve the above technical problems, the present disclosure provides a method, device, medium, equipment and program product for determining target state information, which is used to dynamically design fusion weights according to the stability and reliability of the state information of the target observed under different observation methods, and realize adaptive fusion of the state information of the target observed under different observation methods.

[0005] A first aspect embodiment of the present disclosure provides a method for determining target state information, including: obtaining first state information of a target at a first moment and second state information at a second moment observed by different observation methods, the second moment being earlier than the first moment; for any observation method, determining a first sub-weight based on the first state information and the second state information; and determining a second sub-weight based on the first state information; for any observation method, determining a fusion weight based on the first sub-weight and the second sub-weight; determining the state information of the target at the first moment based on the first state information observed by different observation methods and the corresponding fusion weight.

[0006] The second aspect embodiment of the present disclosure provides a device for determining target state information, including: a state acquisition module, used to obtain first state information of a target at a first moment and second state information of a target at a second moment observed by different observation methods, the second moment being earlier than the first moment; a weight determination module, used to determine a first sub-weight based on the first state information and the second state information for any observation method; and, to determine the second sub-weight based on the first state information; a fusion weight determination module, used to determine a fusion weight based on the first sub-weight and the second sub-weight for any observation method; the state determination module is used to determine the state information of the target at the first moment based on the first state information observed by different observation methods and the corresponding fusion weight.

[0007] A third aspect embodiment of the present disclosure provides a computer-readable storage medium, wherein the storage medium stores computer program instructions, and when the computer program instructions are executed, the method for determining the target state information described above is implemented.

[0008] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising:

[0009] a memory for storing a computer program product;

[0010] The processor is configured to execute the computer program product stored in the memory, and when the computer program product is executed, the method for determining the target state information disclosed above is implemented.

[0011] A fifth aspect embodiment of the present disclosure provides a computer program product, including computer program instructions, which, when executed by a processor, implement the above-mentioned method for determining target state information of the present disclosure.

[0012] Based on the embodiments of the present disclosure, the first state information of the target at the first moment and the second state information at the second moment observed by different observation methods can be obtained, where the second moment is earlier than the first moment; for any observation method, the first sub-weight is determined based on the first state information and the second state information; and the second sub-weight is determined based on the first state information; for any observation method, the fusion weight is determined based on the first sub-weight and the second sub-weight; based on the first state information observed by different observation methods and the corresponding fusion weight, the state information of the target at the first moment is determined. Therefore, the technical solution of the present disclosure adaptively determines the fusion weight through the state information observed under any observation method, which helps to improve the reliability of the state information obtained by fusion, and further helps to provide an accurate basis for downstream tasks such as target behavior prediction, target trajectory prediction, vehicle driving path planning, vehicle driving control, etc., thereby improving the safety of vehicle driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and other purposes, features, and advantages of the present disclosure will become more apparent through a more detailed description of the embodiments of the present disclosure in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and are not intended to limit the present disclosure. In the drawings, the same reference numerals generally represent the same components or steps.

[0014] Figure 1 is an exemplary application scenario diagram provided by an exemplary embodiment of the present disclosure;

[0015] Figure 2 is a flowchart of determining target state information provided by an exemplary embodiment of the present disclosure;

[0016] Figure 3 is a flowchart of determining the first sub-weight in step 202 provided by an exemplary embodiment of the present disclosure;

[0017] Figure 4 is a flowchart of determining the second sub-weight in step 202 provided by an exemplary embodiment of the present disclosure;

[0018] Figure 5 is a flowchart of step 203 provided by an exemplary embodiment of the present disclosure;

[0019] Figure 6 is a structural diagram of a device for determining target state information provided by an exemplary embodiment of the present disclosure;

[0020] Figure 7 is a structural diagram of a device for determining target state information provided by another exemplary embodiment of the present disclosure;

[0021] Figure 8 is a structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] To explain the present disclosure, example embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. It should be understood that the present disclosure is not limited to the example embodiments.

[0023] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

[0024] Overview of the Disclosure

[0025] In the process of realizing the present disclosure, the inventors discovered through research that in the field of intelligent driving (including assisted driving and autonomous driving), state estimation of dynamic targets, such as pedestrians and vehicles, is indispensable. For example, target state estimation can provide an accurate basis for downstream tasks such as target behavior prediction, target trajectory prediction, and vehicle driving path planning.

[0026] In related technologies, for the same dynamic target, due to differences in observation range and observation errors of sensors corresponding to different observation methods, the observed state information of the target is also different, which cannot provide accurate basis for downstream tasks such as target behavior prediction, target trajectory prediction, and vehicle driving path planning, thereby affecting the execution of downstream tasks.

[0027] In the embodiment of the present disclosure, the observation method is determined based on the location and / or type of sensors deployed on the vehicle.

[0028] Based on the different sensor positions, the observation mode may include, but is not limited to, any one of the observation modes such as forward observation, rearward observation, side observation, surround observation, and bird's eye view (BEV) observation. The embodiments of the present disclosure do not limit the specific observation mode. For example, forward observation information can be obtained based on image data collected by a forward-looking camera and point cloud data collected by a forward-facing laser radar. For another example, rearward observation information can be obtained based on image data collected by a rear-facing camera and point cloud data collected by a rear-facing millimeter-wave radar.

[0029] Based on different sensor types, the observation method can include, but is not limited to, visual observation, laser observation, radar observation, multimodal observation, and other observation methods. The embodiments of this disclosure do not limit the specific observation method. For example, multimodal observation information can be obtained based on data collected by cameras, lidar, and millimeter-wave radar. By fusing multimodal sensor data, the robustness and spatiotemporal consistency of the observation information can be improved, thereby enhancing the stability and reliability of the perception system.

[0030] Example Applications

[0031] Figure 1 This is an exemplary application scenario diagram provided by an exemplary embodiment of the present disclosure, such as Figure 1As shown, a computing platform 120 and a plurality of sensors 130 can be deployed on the vehicle 110 as required. The sensors 130 are used to sense (i.e., observe) the external environment of the vehicle 110. The plurality of sensors 130 can be deployed at different positions and / or orientations on the vehicle 110 according to the sensing requirements to perform environmental perception on the corresponding areas in the external environment of the vehicle 110 (e.g., the left front area, the front area, the right front area, the left rear area, the rear area, the right rear area, the left side area of ​​the vehicle body, the rear side area of ​​the vehicle body, etc.). The plurality of sensors 130 can include sensors of the same type or different types, for example, visual sensors (i.e., cameras), LiDAR (Light Detection and Ranging, LiDAR), millimeter wave radar (Millimeter Wave, MMW), ultrasonic radar (Ultrasonic Sensor System, USS), etc. Among them, the distance, direction, height, movement speed, posture, etc. of the target in the external environment of the vehicle 110 relative to the vehicle 110 can be obtained by observing the LiDAR, thereby realizing the detection, tracking, and identification of the target.

[0032] While the vehicle 110 is driving, the multiple sensors 130 on the vehicle 110 observe the surrounding environment of the vehicle 110, and obtain first state information of the target at the first moment and second state information at the second moment observed by different observation methods. The targets may include dynamic targets, such as vehicle 1, pedestrian 2, etc. The computing platform 120 can determine a first sub-weight for the first state information of the target at the first moment and the second state information at the second moment observed by any observation method, and determine a second sub-weight based on the first state information of the observed target at the first moment, and then determine a fusion weight of the first state information observed under different observation methods based on the first sub-weight and the second sub-weight, and fuse the state information of the target at the first moment based on the fusion weight.

[0033] For any observation method, the fitting error can be determined by fitting based on the first state information of the target observed at the first moment and the second state information of the target observed at the second moment (a historical moment earlier than the first moment), and the observation stability of the corresponding observation method can be determined based on the fitting error; and the relative position of the target and the vehicle can be determined based on the first state information of the target observed at the first moment, and then the observation reliability of the corresponding observation method can be determined. The state information of the target observed by different observation methods is weighted and fused according to the observation stability and observation reliability of any observation method to obtain the state information of the target, which effectively improves the accuracy and reliability of the target state information, and provides an accurate basis for downstream tasks such as target behavior prediction, target trajectory prediction, vehicle driving path planning, and vehicle driving control, thereby improving the safety of vehicle driving.

[0034] Figure 1 This is only an implementation method of an exemplary application scenario of the embodiment of the present disclosure. Those skilled in the art can know based on the records of the embodiment of the present disclosure that the embodiment of the present disclosure can also adopt any other feasible implementation method. For example, the computing platform can also be deployed on a cloud server or terminal device (such as a mobile terminal, a tablet computer, a PC, etc.), and receive observation information from multiple sensors 130 by communicating with the vehicle driving control system. The information is processed using the target state information determination method provided by the embodiment of the present disclosure to obtain the state information of the target at the first moment and feed it back to the vehicle driving control system.

[0035] Exemplary Methods

[0036] Figure 2 This is a flow chart of a method for determining target state information provided by an exemplary embodiment of the present disclosure. This embodiment can be applied to electronic devices, which can be set in the above Figure 1 On the vehicle 110 shown in FIG. Figure 2 As shown, the following steps are included:

[0037] Step 201 : obtaining first state information of a target at a first moment and second state information of a target at a second moment observed by different observation methods, wherein the second moment is earlier than the first moment.

[0038] In the embodiments of the present disclosure, the observation method can be any observation method that can be realized based on the sensors deployed on the vehicle, for example, it can include but is not limited to any one of the observation methods such as forward observation, rearward observation, side observation, surround observation, and bird's eye view (BEV) observation. The embodiments of the present disclosure do not limit the specific observation method. Among them, the sensors deployed on the vehicle can include, for example, but are not limited to, any number of sensors of the same type or different types deployed at different locations and / or in different orientations on the vehicle, such as visual sensors, lidar, millimeter wave radar, ultrasonic radar, etc.

[0039] In some implementations, observation results from visual sensors deployed on the vehicle with different positions and orientations may be spliced ​​together to obtain state information observed using a surround view observation method.

[0040] In some implementations, multi-source data fusion processing can be performed on any number of observation results from visual sensors, lidar, millimeter-wave radar, ultrasonic radar, etc. deployed on the vehicle to obtain the BEV plane observation result, which serves as the state information observed by the BEV observation method. The specific implementation of multi-source data fusion processing can be referenced in related technologies and will not be described here.

[0041] In the disclosed embodiments, the first moment may be any moment at which the state information of the target is observed by an observation method. In application scenarios requiring immediate target tracking (e.g., intelligent driving scenarios), the first moment may be the moment (which may be referred to as the current moment) at which the sensor corresponding to each observation method most recently collected data during vehicle travel. In application scenarios requiring target tracking based on historically collected data, the first moment may be the acquisition moment of the current video frame or data frame to be subjected to target detection and tracking processing.

[0042] The second moment is before the first moment. Assuming the first moment is moment t, the second moment can be a plurality of moments earlier than the first moment, such as moment t-1, moment t-2, moment t-3, and moment tn (n is an integer greater than 3). With respect to the first state information observed at the first moment, the state information of the target observed at the second moment is the second state information.

[0043] In the embodiment of the present disclosure, the target is any target that can be observed based on various observation methods, and may include dynamic targets and static targets. The first state information is the state information of the target obtained at the first moment through different observation methods. The state information may include, but is not limited to, any one or more of the following information of the target: type, position, size, direction (orientation or heading), speed, etc. Among them, in other possible implementation methods, the state information may include only any part of the information therein according to actual needs, or may also include more information. The embodiment of the present disclosure does not limit the specific information content included in the first state information. The second state information is the state information of the target obtained at the second moment through different observation methods.

[0044] In the disclosed embodiments, the first state information and the second state information may be state information in a specified coordinate system. The specified coordinate system may include, but is not limited to, any one or more of the following: a vehicle coordinate system, a world coordinate system, and an image coordinate system. These coordinate systems may be transformed into each other, for example, by transforming the state information between corresponding coordinate systems based on predetermined transformation parameters between the coordinate systems. In some implementations, the same coordinate system may be specified for different observation modes enabled by sensors deployed on the vehicle, or a separate coordinate system may be specified for each observation mode.

[0045] In a specific implementation, when there are multiple sensors deployed on a vehicle, each sensor can collect data according to its preset frame rate. For a frame of status data collected by any sensor at any data collection moment (as the first moment), target detection or perception processing can be performed through corresponding data processing methods, such as deep learning models or other algorithm models, to obtain the first status information at the first moment (also called perception result).

[0046] Step 202 : For any observation mode, determine a first sub-weight based on the first state information and the second state information; and determine a second sub-weight based on the first state information.

[0047] In the embodiment of the present disclosure, the first sub-weight is a weight value that can characterize the stability of the state information of the target observed by the observation method. The first sub-weight corresponding to any observation method can be measured according to the state information sequence of the target observed under any observation method. The state information sequence of the target observed by any observation method includes: the state information of the target observed by the observation method at the first moment and multiple second moments earlier than the first moment, arranged based on the time series relationship. For example, through the forward-looking observation method, the first state information S1 of the target is obtained at the first moment, and the second state information S2, S3, S4, ..., Sn of the target is obtained at multiple second moments, then S1, S2, S3, S4, ..., Sn is the state information sequence of the target observed by the forward-looking observation method.

[0048] In the embodiment of the present disclosure, the stability of the target observed by any observation method can be determined according to the amount of outlier information in each state information in the state information sequence of the target observed by any observation method, and then the first sub-weight can be determined.

[0049] In the disclosed embodiments, the second sub-weight is a weight value that characterizes the reliability of the state information of the target observed by the observation method. The second sub-weight corresponding to any observation method can be determined based on the state information of the target observed at the first moment under any observation method and the sensor acquisition parameters of the sensor corresponding to the observation method.

[0050] Because different observation methods correspond to different sensor deployment positions and orientations, as well as differences in sensor acquisition parameters, the reliability of target status information collected by different observation methods varies. For example, a forward-looking sensor can cover targets within 150 meters ahead, while a BEV sensor can cover targets within a 100-meter radius. If a target is 125 meters ahead, the forward-looking sensor will have a higher reliability of the target's status, while the BEV sensor will have a lower reliability.

[0051] Step 203: For any observation mode, determine a fusion weight based on the first sub-weight and the second sub-weight.

[0052] In the embodiment of the present disclosure, the fusion weight is the weight coefficient of fusing the first state information of the target observed by different observation methods to obtain the state information of the target at the first moment. The fusion weight corresponding to any observation method can be obtained by calculating the first sub-weight and second sub-weight corresponding to each observation method.

[0053] The sum of the fusion weights corresponding to all observation modes is 1. For example, the observation modes include forward-looking observation mode, BEV observation mode, and rearward-looking observation mode. The fusion weights of the three observation modes are k1, k2, and k3, respectively. The values ​​of k1, k2, and k3 are respectively greater than 0 and less than 1, and the sum of k1, k2, and k3 is 1.

[0054] Step 204 : Determine the state information of the target at the first moment based on the first state information observed by different observation methods and the corresponding fusion weights.

[0055] In the disclosed embodiments, the target's state information at the first moment is fused state information obtained by fusing first state information observed using different observation methods. This fused state information can be fed back to the vehicle's driving control system to provide an accurate basis for downstream tasks such as target behavior prediction, target trajectory prediction, vehicle path planning, and vehicle driving control.

[0056] The method for determining target state information provided by the embodiment of the present disclosure is to obtain the first state information of the target at the first moment and the second state information at the second moment observed by different observation methods, wherein the second moment is earlier than the first moment; for any observation method, determine the first sub-weight based on the first state information and the second state information; and determine the second sub-weight based on the first state information; for any observation method, determine the fusion weight based on the first sub-weight and the second sub-weight; determine the state information of the target at the first moment based on the first state information observed by different observation methods and the corresponding fusion weight. Therefore, the technical solution of the present disclosure adaptively determines the fusion weight through the state information observed under any observation method, which helps to improve the reliability of the state information obtained by fusion, and further helps to provide an accurate basis for downstream tasks such as target behavior prediction, target trajectory prediction, vehicle driving path planning, vehicle driving control, etc., thereby improving the safety of vehicle driving.

[0057] In some implementations, after fusion is performed to obtain the target's state information at a first moment, the state information at the first moment can be further added to an observation sequence, where the observation sequence includes the target's state information at a second moment. Adding the fused target's state information at the first moment to the observation sequence facilitates the vehicle's driving control system to use the state information in the observation sequence as a basis when executing downstream tasks based on the target's state information, thereby improving the accuracy and reliability of executing downstream tasks.

[0058] The observation sequence can be a sliding window sequence. A sliding window sequence is a fixed-size observation sequence in which the data in the sliding window sequence includes the target's state information at a first moment and the target's state information at a predetermined second moment. In this implementation, setting the observation sequence as a sliding window sequence facilitates capturing the target's local temporal characteristics.

[0059] The first state information and the second state information obtained by different observation methods include the speed of the target, such as Figure 3 As shown in the above Figure 2 Based on the illustrated embodiment, the operation of determining the first sub-weight in step 202 includes the following steps:

[0060] Step 221 , fitting the speed information of the target at the first moment and the speed information of the target at the second moment to obtain a fitting error.

[0061] Among them, fitting is a data analysis method. By fitting the speed information at the first moment and the speed information at multiple second moments, the law and change of the speed of the observed target can be determined, and then the stability of the speed of the observed target can be determined.

[0062] In the embodiment of the present disclosure, the fitting error is used to evaluate the data fitting effect and can measure the degree of deviation between the fitting curve and the original data. The larger the fitting error, the worse the stability of the speed sequence composed of the speed information of the target observed at the first moment and multiple second moments before the first moment. The smaller the fitting error, the better the stability of the speed sequence composed of the speed information of the target observed at the first moment and multiple second moments before the first moment.

[0063] In the embodiment of the present disclosure, the speed information at the first moment and the speed information at the second moment can be fitted by a variety of fitting methods, such as ordinary least squares method, stepwise regression algorithm, polynomial fitting method, logarithmic fitting method, etc. The embodiment of the present disclosure does not limit the fitting method.

[0064] Step 222: Determine a first sub-weight based on the fitting error.

[0065] In the embodiment of the present disclosure, the first sub-weight may be determined based on the fitting error and a preset mapping relationship between the fitting error and the first sub-weight.

[0066] In the embodiment of the present disclosure, in order to automatically determine the corresponding first sub-weight according to the fitting error, the device for executing the method for determining the target state information can be pre-installed, such as Figure 1 A mapping relationship between the fitting error and the first sub-weight is set for vehicle 110 in the example. This mapping relationship can be learned based on a large amount of offline data. The larger the fitting error, the less stable the target's speed information at the first moment and the speed information at multiple second moments, and thus the smaller the corresponding first sub-weight. For example, when the fitting error is 0.1, the first sub-weight is 0.6, and when the fitting error is 0.3, the first sub-weight is 0.4.

[0067] The method for determining the first sub-weight provided in the embodiment of the present disclosure, for any observation mode, determines the first sub-weight associated with the observation stability of the observation mode by fitting the first state information of the target at the first moment and the second state information of the target at the second moment, which helps to suppress the influence of observation jitter or frequent jumping observation modes on the final fusion state information, thereby improving the accuracy and reliability of the final fusion state information.

[0068] Among them, the first state information obtained through different observation methods includes the target's posture information, such as Figure 4 As shown in the above Figure 2 Based on the illustrated embodiment, the operation of determining the second sub-weight in step 202 includes the following steps:

[0069] Step 223: Based on the position information of the target in the first state information, determine the relative position information of the target and the vehicle at the first moment.

[0070] The position information of the vehicle may be any one or more coordinate systems including the vehicle coordinate system, the world coordinate system, and the image coordinate system.

[0071] In the embodiment of the present disclosure, if the acquired target posture information is posture information in the ego-vehicle coordinate system, the relative posture of the target and the ego-vehicle can be directly determined based on the target posture information.

[0072] In the embodiment of the present disclosure, if the acquired target posture information is not a coordinate in the self-vehicle coordinate system, but a coordinate in the world coordinate system or the image coordinate system, the target posture information can be switched to the self-vehicle coordinate system based on the predetermined transformation parameters between the coordinate systems, and then the relative posture of the target and the self-vehicle can be determined based on the coordinate information in the self-vehicle coordinate system.

[0073] Step 224 : Determine a second sub-weight based on the relative posture information and acquisition parameter information of the data acquisition sensor corresponding to any observation mode.

[0074] In the embodiment of the present disclosure, the acquisition parameter information of the data acquisition sensor is used to indicate the sensor parameters. For example, the acquisition parameter information of the lidar includes but is not limited to: ranging range, ranging accuracy, field of view angle, measurement range, measurement accuracy, etc. The acquisition parameter information of the visual sensor includes but is not limited to: field of view angle, focal length, etc.

[0075] In the embodiment of the present disclosure, the second sub-weight can be determined by the following steps: based on the acquisition parameter information of the data acquisition sensor corresponding to any observation method, the scene prior information corresponding to any observation method is determined, and the scene prior information is used to characterize the mapping relationship between the relative posture information and the second sub-weight; based on the relative posture information of the target and the vehicle at the first moment, and the scene prior information, the second sub-weight is determined.

[0076] Specifically, based on the data acquisition sensor's acquisition parameters and installation location, we can determine the range of targets within which the sensor can and cannot observe targets, and thus determine the scene prior information corresponding to different observation methods. This range can be described by the relative position of the vehicle.

[0077] The scene prior information is used to characterize the mapping relationship between relative pose information and the second sub-weight. The second sub-weight is a weight value that represents the reliability of the state information of the target observed by the observation method. Therefore, the scene prior information is used to characterize the reliability of the state information of the target observed by the corresponding observation method under different relative pose information. The scene prior information corresponding to different observation methods can be derived based on the acquisition parameters of the data acquisition sensor corresponding to the observation method, or it can be summarized based on a large amount of empirical data. For example, for a target at the edge of the forward-looking camera, the state information observed by the forward-looking observation method is less reliable because the entire target may not be observed. However, the state information derived from the BEV observation method is not affected by this factor. In this scenario, the fusion weight of the BEV observation method can be increased. For another example, based on empirical data, for targets located in the center of the road, the forward-looking observation method is generally more accurate and reliable than the state information obtained from the BEV observation method. Therefore, for targets located in the center of the road, the second sub-weight corresponding to the forward-looking observation method can be increased.

[0078] Among them, the mapping relationship between the relative posture information and the second sub-weight can be obtained based on the acquisition parameter information and / or experience data of the data acquisition sensor and can be stored in an electronic device used to execute the method for determining the target state information.

[0079] The method for determining the second sub-weight provided in the embodiment of the present disclosure, for any observation mode, determines the relative posture information of the target and the vehicle at the first moment by analyzing the posture information of the target in the first state information of the target at the first moment, and then determines the second sub-weight associated with the observation reliability of the observation mode, which helps to improve the influence of the state information of the observation mode of the dominant scene on the final fusion state information, and thus improves the accuracy and reliability of the final fusion state information.

[0080] like Figure 5 As shown in the above Figure 2 Based on the embodiment shown, step 203 includes the following steps:

[0081] Step 231: Determine an initial fusion weight based on the first sub-weight and the second sub-weight.

[0082] In the embodiment of the present disclosure, for any observation method, the initial fusion weight corresponding to any observation method can be obtained by performing weight addition, multiplication, and other operations on the first sub-weight and the second sub-weight.

[0083] In step 232, the initial fusion weights corresponding to different observation modes are normalized to obtain the fusion weights corresponding to different observation modes.

[0084] In this embodiment, by normalizing the initial fusion weights corresponding to different observation methods, the fusion weights of different observation methods can be made to be numbers greater than 0 and less than 1, and the sum of the fusion weights of different observation methods is 1.

[0085] Exemplarily, assuming that the embodiment of the present disclosure observes the first state information of the target at the first moment and the second state information at the second moment through three observation methods, and the first sub-weights determined according to the first state information observed by each observation method and the second state information at the second moment are 1 / 5, 2 / 5, and 2 / 5 respectively, and the second sub-weights determined according to the first state information observed by each observation method are 1 / 3, 1 / 6, and 1 / 2 respectively, then the initial fusion weights of the three observation methods obtained by multiplying the weights are 1 / 5*1 / 3=1 / 15, 2 / 5*1 / 6=1 / 15, and 2 / 5*1 / 2=1 / 5 respectively; by setting the sum of the initial fusion weights of the three observation methods to 1, the normalized fusion weights of the three observation methods can be obtained as 1 / 5, 1 / 5, and 3 / 5 respectively.

[0086] In addition to obtaining the initial fusion weight of each observation method based on the first sub-weight and the second sub-weight by multiplying the weights in the above example, the initial fusion weight of each observation method can also be obtained by adding the weights, and then normalizing them to obtain the fusion weight.

[0087] The method for determining fusion weights provided in the embodiments of the present disclosure obtains an initial fusion weight for any observation method based on the first and second sub-weights corresponding to any observation method, and then performs weight normalization to obtain fusion weights corresponding to different observation methods. By effectively fusing the first and second sub-weights, the embodiments of the present disclosure achieve adaptive determination of fusion weights based on the stability and reliability of state information observed under any observation method.

[0088] Exemplary devices

[0089] Figure 6 This is a schematic diagram of the structure of a device for determining target state information provided by an exemplary embodiment of the present disclosure. The device for determining target state information of the embodiment of the present disclosure can be used to implement the method for determining target state information of any embodiment of the present disclosure. Figure 6 As shown, in one embodiment, the target state information determining device includes:

[0090] A state acquisition module 61 is configured to acquire first state information of a target at a first moment and second state information of a target at a second moment observed by different observation methods, where the second moment is earlier than the first moment;

[0091] A weight determination module 62 is configured to determine, for any observation mode, a first sub-weight based on the first state information and the second state information; and determine a second sub-weight based on the first state information;

[0092] a fusion weight determination module 63, configured to determine a fusion weight based on the first sub-weight and the second sub-weight for any observation mode;

[0093] The state determination module 64 is configured to determine the state information of the target at the first moment based on the first state information observed by different observation modes and the corresponding fusion weights.

[0094] Figure 7 FIG. 1 is a schematic diagram of a structure of a device for determining target state information provided by another exemplary embodiment of the present disclosure. Figure 7 As shown, in Figure 6 Based on the illustrated embodiment, in another embodiment, the state information includes speed information;

[0095] The weight determination module 62 may include:

[0096] The fitting submodule 621 is used to fit the speed information of the target at the first moment and the speed information at the second moment to obtain a fitting error;

[0097] The first determining submodule 622 is configured to determine a first sub-weight based on the fitting error.

[0098] The first determining submodule 622 is specifically configured to determine the first sub-weight based on the fitting error and a preset mapping relationship between the fitting error and the first sub-weight.

[0099] In another embodiment, the weight determination module 62 may include:

[0100] A posture determination submodule 623 is configured to determine the relative posture information of the target and the ego-vehicle at the first moment based on the posture information of the target in the first state information;

[0101] The second determining submodule 624 is configured to determine a second sub-weight based on the relative posture information and acquisition parameter information of the data acquisition sensor corresponding to any observation mode.

[0102] In another embodiment, the second determination submodule 624 is specifically used to: determine the scene prior information corresponding to any observation method based on the acquisition parameter information of the data acquisition sensor corresponding to any observation method, and the scene prior information is used to characterize the mapping relationship between the relative posture information and the second sub-weight; determine the second sub-weight based on the relative posture information of the target and the vehicle at the first moment, and the scene prior information.

[0103] In another embodiment, the fusion weight determination module 63 may include:

[0104] A third determining submodule 631 is configured to determine an initial fusion weight based on the first sub-weight and the second sub-weight;

[0105] The normalization submodule 632 is used to normalize the initial fusion weights corresponding to different observation methods to obtain the fusion weights corresponding to different observation methods.

[0106] In another embodiment, the target state information determining device may further include:

[0107] The storage module 65 is configured to add the state information at the first moment to the observation sequence, where the observation sequence includes the state information of the target at the second moment.

[0108] Among them, the observation sequence is a sliding window sequence.

[0109] The beneficial technical effects corresponding to the exemplary embodiment of this device can be found in the corresponding beneficial technical effects of the above exemplary method part, which will not be repeated here.

[0110] Exemplary electronic devices

[0111] Figure 8 FIG. 1 is a structural block diagram of an electronic device provided by an exemplary embodiment of the present disclosure. Figure 8 As shown, the electronic device includes at least one processor 10 and a memory 20 .

[0112] The processor 10 may be a central processing unit (CPU) or other form of processing unit having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions. The memory 20 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 10 may execute one or more computer program instructions to implement the target state information determination method and / or other desired functions of the various embodiments of the present disclosure described above.

[0113] In one example, the electronic device may further include an input device 30 and an output device 40, which are interconnected via a bus system and / or other connection mechanisms (not shown). The input device 30 may also include, for example, a keyboard, a mouse, etc. The output device 40 may output various information to the outside, and may include, for example, a display, a speaker, a printer, a communication network, and remote output devices connected thereto.

[0114] Of course, to simplify, Figure 8 Only some of the components related to the present disclosure in the electronic device are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, the electronic device may further include any other appropriate components according to specific application scenarios.

[0115] Exemplary computer program products and computer-readable storage media

[0116] In addition to the above-mentioned methods and devices, embodiments of the present disclosure may also provide a computer program product, including computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method for determining target state information of various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section.

[0117] The computer program product may be written in any combination of one or more programming languages ​​to implement the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0118] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enables the processor to execute the steps in the method for determining target state information of various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section.

[0119] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium is, for example, but not limited to, a system, device or component comprising electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0120] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, the advantages, strengths, and effects mentioned in this disclosure are merely illustrative and not restrictive, and should not be considered as essential to each embodiment of the present disclosure. Furthermore, the specific details disclosed above are provided for illustrative purposes and to facilitate understanding, rather than as limitations. These details do not limit the present disclosure to necessarily being implemented using these specific details.

[0121] Those skilled in the art may make various changes and modifications to the present disclosure without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is intended to include these modifications and variations.

Claims

1. A method for determining target state information, comprising: Acquire first state information of a target at a first moment and second state information of a target at a second moment observed by different observation modes, where the second moment is earlier than the first moment; For any of the observation modes, determining a first sub-weight based on the first state information and the second state information; and, determining a second sub-weight based on the first state information; For any of the observation modes, determining a fusion weight based on the first sub-weight and the second sub-weight; The state information of the target at the first moment is determined based on the first state information observed in different observation modes and the corresponding fusion weights.

2. The method according to claim 1, wherein The state information includes speed information; The determining the first sub-weight based on the first state information and the second state information includes: Fitting the speed information of the target at the first moment and the speed information of the target at the second moment to obtain a fitting error; The first sub-weight is determined based on the fitting error.

3. The method according to claim 2, wherein determining the first sub-weight based on the fitting error comprises: The first sub-weight is determined based on the fitting error and a preset mapping relationship between the fitting error and the first sub-weight.

4. The method according to any one of claims 1 to 3, wherein: The determining the second sub-weight based on the first state information includes: Determining relative position information of the target and the vehicle at the first moment based on the position information of the target in the first state information; The second sub-weight is determined based on the relative posture information and acquisition parameter information of the data acquisition sensor corresponding to any one of the observation modes.

5. The method according to claim 4, wherein The determining the second sub-weight based on the relative posture information and acquisition parameter information of the data acquisition sensor corresponding to any one of the observation modes includes: Determining scene prior information corresponding to any of the observation modes based on acquisition parameter information of a data acquisition sensor corresponding to any of the observation modes, wherein the scene prior information is used to characterize a mapping relationship between relative pose information and the second sub-weight; The second sub-weight is determined based on the relative pose information between the target and the ego vehicle at the first moment and the scene prior information.

6. The method according to any one of claims 1 to 5, wherein: The determining of a fusion weight based on the first sub-weight and the second sub-weight includes: Determining an initial fusion weight based on the first sub-weight and the second sub-weight; The initial fusion weights corresponding to different observation modes are normalized to obtain fusion weights corresponding to different observation modes.

7. The method according to any one of claims 1 to 6, wherein: After determining the state information of the target at the first moment, the method further includes: The state information at the first moment is added to an observation sequence, where the observation sequence includes the state information of the target at the second moment.

8. The method according to claim 7, wherein: The observation sequence is a sliding window sequence.

9. A device for determining target state information, comprising: A state acquisition module, configured to acquire first state information of a target at a first moment and second state information at a second moment observed by different observation methods, where the second moment is earlier than the first moment; a weight determination module, configured to determine, for any of the observation modes, a first sub-weight based on the first state information and the second state information; and, determining a second sub-weight based on the first state information; a fusion weight determination module, configured to determine a fusion weight based on the first sub-weight and the second sub-weight for any of the observation modes; A state determination module is used to determine the state information of the target at the first moment based on the first state information observed by different observation methods and the corresponding fusion weights.

10. A computer-readable storage medium, wherein the storage medium stores computer program instructions, and when the computer program instructions are executed, the method according to any one of claims 1 to 8 is implemented.

11. An electronic device, comprising: a memory for storing a computer program product; A processor is configured to execute a computer program product stored in the memory, and when the computer program product is executed, implements the method described in any one of claims 1 to 8.

12. A computer program product comprising computer program instructions, characterized in that When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images