Target detection method, vehicle control method, device, vehicle, medium and chip

Abstract

The embodiment of the invention provides a target detection method, a vehicle control method, equipment, a vehicle, a medium and a chip, and relates to the technical field of automatic driving. The target detection method comprises the following steps: acquiring initial position information of a target object at each of a plurality of moments according to environmental data at the plurality of moments; based on a preset first detection target, according to the initial position information of the target object at the multiple moments, determining initial speed information of the target object at each moment, so that the initial speed information accords with the preset first detection target; and determining target state information of the target object at each moment according to the initial position information and the initial speed information at the plurality of moments. Thus, based on the initial position information of multiple moments, through optimization solution of two stages, the determined target state information can be more accurate and smoother in the time sequence dimension, and the problem of state information hopping of the target object is avoided.

Description

Technical Field

[0001] The present disclosure relates to the field of autonomous driving technology, and more specifically, to a target detection method, a vehicle control method, a device, a vehicle, a medium, and a chip. Background Art

[0002] In the field of autonomous driving, the position, speed and other status information of target objects in the vehicle's surrounding environment can be detected based on the environmental data around the vehicle. The status information of these target objects can be used for vehicle decision-making planning and control.

[0003] In related technologies, the environmental data around the vehicle can be analyzed and processed based on the target object detection model to obtain the status information of the target object. However, the results predicted by the model are inaccurate and jumpy, which affects the decision-making planning and control of the autonomous driving vehicle. Summary of the invention

[0004] In view of this, the embodiments of the present disclosure propose a new technical solution for target detection.

[0005] According to a first aspect of an embodiment of the present disclosure, a target detection method is provided, the method comprising:

[0006] According to the environmental data at multiple moments, the initial position information of the target object at each of the multiple moments is obtained; wherein the environmental data includes real-scene scanning data of the external environment of the vehicle, and the target object is a dynamic object in the external environment of the vehicle;

[0007] Based on a preset first detection target, initial speed information of the target object at each moment is determined according to initial position information of the target object at multiple moments, so that the initial speed information meets the first detection target; wherein the initial speed information includes a first speed component of the target object along a first coordinate axis and a second speed component of the target object along a second coordinate axis, and the first detection target includes a speed difference of the target object along the same coordinate axis at adjacent moments being less than or equal to a preset speed threshold;

[0008] According to the initial position information and initial speed information at the multiple moments, the target state information of the target object at each moment is determined; wherein the target state information includes the target position information, target speed information and target heading angle information of the target object.

[0009] Optionally, the initial position information includes the center point coordinates of the target object, and the center point coordinates include a first coordinate of the center point of the target object on the first coordinate axis and a second coordinate of the second coordinate axis;

[0010] The first detection target includes constraint items determined based on a first coordinate, a second coordinate, a first velocity component, and a second velocity component of the target object at different times.

[0011] Optionally, the constraint item of the first detection target includes at least one of the following:

[0012] The difference between the first coordinate difference and the first movement distance of the center point of the target object between the first moment and the second moment is less than or equal to a first preset threshold; wherein the first moment is a moment earlier than the second moment and adjacent to the second moment among the multiple moments, the first coordinate difference is the difference between the first coordinate of the center point at the first moment and the first coordinate of the second moment, and the first movement distance is a distance calculated based on the first velocity component and the time difference between the first moment and the second moment;

[0013] The difference between the second coordinate difference of the center point of the target object between the first moment and the second moment and the second movement distance is less than or equal to a second preset threshold; wherein the second coordinate difference is the difference between the second coordinate of the center point at the first moment and the second coordinate at the second moment, and the second movement distance is the distance calculated based on the second velocity component and the time difference between the first moment and the second moment;

[0014] The difference between the first velocity component of the target object at the first moment and the first velocity component of the target object at the second moment is less than or equal to a third preset threshold;

[0015] The difference between the second velocity component of the target object at the first moment and the second velocity component of the target object at the second moment is less than or equal to a fourth preset threshold.

[0016] Optionally, the initial position information is the three-dimensional position information of the target object, the center point coordinates further include the third coordinates of the center point of the target object on a third coordinate axis, and the initial velocity information further includes a third velocity component of the target object along the direction of the third coordinate axis;

[0017] The constraint item of the first detection target also includes at least one of the following:

[0018] The difference between the third coordinate difference of the center point at the first moment and the second moment and the third movement distance is less than or equal to a fifth preset threshold; wherein the third coordinate difference is the difference between the third coordinate of the center point at the first moment and the third coordinate of the center point at the second moment, and the second movement distance is the distance calculated based on the third velocity component and the time difference between the first moment and the second moment;

[0019] A difference between the third velocity component of the target object at the first moment and the third velocity component of the target object at the second moment is less than or equal to a sixth preset threshold.

[0020] Optionally, determining the target state information of the target object at each moment according to the initial position information and the initial speed information at the multiple moments includes:

[0021] Based on a preset second detection target, determining the target state information of the target object at each moment according to the initial position information and the initial speed information at the multiple moments, so that the target state information meets the second detection target;

[0022] The second detection target includes constraints determined based on a state quantity of the target object, and the state quantity includes at least one of the center point coordinates, size, heading angle, speed, acceleration and angular velocity of the target object.

[0023] Optionally,

[0024] The second detection target includes: a difference between a target speed predicted at each moment and an initial speed of the target object is less than or equal to a preset difference, and the initial speed is a speed value calculated according to the initial speed information.

[0025] Optionally, the second detection target includes constraints corresponding to multiple parts of the target object, wherein:

[0026] The constraints of different parts of the target object correspond to different confidence levels, and the confidence level of the constraints of each part is determined based on the number of position points corresponding to the part in the environmental data; the confidence level is used to characterize the weight of the constraints of the part among multiple part constraints.

[0027] Optionally,

[0028] The multiple parts of the target object include a head and a tail of the target object;

[0029] The constraint conditions of each part of the target object are constraint conditions determined based on the center point coordinates of the part, the center point coordinates of the target object, the size of the target object, and the heading angle of the target object.

[0030] Optionally, the second detection target includes at least one of the following:

[0031] The acceleration difference of the target object at adjacent moments is less than or equal to a preset acceleration threshold;

[0032] The angular velocity difference of the target object at adjacent moments is less than or equal to a preset angular velocity threshold.

[0033] Optionally, acquiring the initial position information of the target object at each of the multiple moments according to the environmental data at the multiple moments includes:

[0034] Based on the target object detection model, determine at least one candidate object at each moment and attribute information of the candidate object at the moment according to the environmental data; wherein the attribute information includes position information of the candidate object, and size information and / or heading angle information of the candidate object;

[0035] The candidate objects are matched according to the similarity of the attribute information between the candidate objects at different times, the attribute information of the same candidate object at each time is determined, and the candidate object is used as the target object, and the position information in the attribute information of the candidate object is used as the initial position information of the target object at each time.

[0036] According to a second aspect of an embodiment of the present disclosure, a vehicle control method is provided, the method comprising:

[0037] Determine the target object in the external environment where the vehicle is located;

[0038] Controlling the vehicle to travel according to the target state information of the target object;

[0039] Among them, the target state information is the state information of the target object at each moment determined according to the initial position information and initial speed information of the target object at multiple moments, the initial position information is the position information of the target object obtained according to the environmental data at multiple moments, and the initial speed information is the initial speed information of the target object at each moment determined based on a preset first detection target and according to the initial position information; the environmental data includes real-scene scanning data of the external environment of the vehicle, the target object is a dynamic object in the external environment of the vehicle, the initial speed information includes a first speed component of the target object along the first coordinate axis and a second speed component of the target object along the second coordinate axis, the first detection target includes the speed difference of the target object along the same coordinate axis at adjacent moments being less than or equal to a preset speed threshold, and the target state information includes the target position information, target speed information and target heading angle information of the target object.

[0040] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising a memory and a processor, wherein the memory is used to store computer instructions, and the processor is used to call the computer instructions from the memory to execute a method as described in any one of the first aspect and / or the second aspect.

[0041] According to a fourth aspect of an embodiment of the present disclosure, a vehicle is provided, comprising a memory and a processor, wherein the memory is used to store computer instructions, and the processor is used to call the computer instructions from the memory to execute a method as described in any one of the first aspect and / or the second aspect.

[0042] According to a fifth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method described in any one of the first aspect and / or the second aspect is implemented.

[0043] According to a sixth aspect of an embodiment of the present disclosure, a chip is provided, comprising a processing unit, wherein the processing unit is configured to execute a method as described in any one of the first aspect and / or the second aspect.

[0044] Based on the target detection method provided by the embodiment of the present disclosure, the initial position information at multiple times can be used to perform a two-stage optimization solution to make the determined target state information more accurate and smooth in the time series dimension, thereby avoiding the problem of state information jumps of the target object, thereby improving the accuracy of decision-making planning and control of the autonomous driving vehicle.

[0045] Other features and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0047] Figure 1 The present invention is a schematic diagram of an intelligent network connection system to which the method provided by the embodiment of the present disclosure can be applied.

[0048] Figure 2 is based on Figure 1 The illustrated embodiment provides a schematic structural diagram of a vehicle.

[0049] Figure 3 It is a flowchart of a target detection method provided by an embodiment of the present disclosure.

[0050] Figure 4 It is a flowchart of a target detection method provided by an embodiment of the present disclosure.

[0051] Figure 5 It is a flow chart of a vehicle control method provided by an embodiment of the present disclosure.

[0052] Figure 6 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0053] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0054] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0055] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the above-mentioned technologies, methods, and equipment should be considered as part of the specification.

[0056] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0057] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0058] First, the application scenarios of the embodiments of the present disclosure are described.

[0059] Figure 1 1 is a schematic diagram of an intelligent network system 100 to which the method provided by the embodiment of the present disclosure can be applied. Figure 1 As shown, the intelligent network connection system 100 may include: a vehicle 101, a server 102, and a user terminal 103.

[0060] In some examples, vehicle 101 may be a vehicle with an autonomous driving function. Autonomous driving is also known as unmanned driving or intelligent driving. Vehicles with autonomous driving functions can achieve driving tasks such as environmental perception, decision planning, and control execution. The level of autonomous driving can refer to the automotive intelligence classification standards formulated by the Society of Automotive Engineers (SAE). For example, L0 is manual driving, L1 is assisted driving, L2 is partial autonomous driving, L3 is conditional autonomous driving, L4 is highly autonomous driving, and L5 is fully autonomous driving. The above classification method for autonomous driving levels is only used as an example, and the disclosed embodiments do not limit the classification standards and levels of autonomous driving.

[0061] In some examples, the server 102 may be a single server or a distributed server cluster consisting of multiple servers, and its deployment method may include a local server or a cloud server. The server 102 may communicate with the vehicle 101 and / or the user terminal 103 based on a communication network, and provide various services for the vehicle 101 and / or the user terminal 103. For example, the server may receive the perception data sent by the vehicle, and provide the vehicle with services such as high-precision maps, data analysis, and decision-making planning. For another example, the server may receive query instructions or control instructions sent by the user terminal, and provide corresponding services to the user.

[0062] In some examples, the user terminal 103 may be any form of electronic device that provides services to users, such as a personal computer, a laptop, a smart tablet, a smart phone, a smart wearable device, etc. The user may interact with the vehicle or server through the human-computer interaction terminal configured in the vehicle 101, or may interact with the vehicle or server through the user terminal 103, for example, querying the status and / or parameters of the vehicle through the user terminal, or controlling the vehicle to perform set tasks and / or modify configuration parameters, etc.; wherein the user terminal runs an application based on the intelligent network connection system to achieve interaction with the vehicle or server, and the application may be a local application, a web application, a small program, etc., which is not limited here.

[0063] In some examples, the above-mentioned application running on the user terminal can provide authentication or authorization services for the user. The user who is successfully authenticated and granted the corresponding authority can query and / or control the vehicle within the scope of the granted authority.

[0064] The vehicle 101, the server 102 and the user terminal 103 can communicate with each other through the communication link provided by the communication network 104. The communication network 104 may include one or more networks of any type, for example, the communication network 104 may include the Internet, a local area network (LAN), a wide area network (WAN), a virtual private network (VPN), a public switched telephone network (PSTN), a satellite communication network, Wi-Fi, 2G, 3G, 4G, 5G, 6G, NB-IoT, eMTC, infrared, Bluetooth, NFC and other networks providing communication, or a combination of the above multiple networks. The communication networks between the vehicle 101 and the server 102, between the user terminal 103 and the server 102, and between the user terminal 103 and the vehicle 101 may be the same or different.

[0065] It should be noted that Figure 1The structure of the intelligent network connection system 100 shown in the figure is only illustrative. The intelligent network connection system in the embodiment of the present disclosure is not limited to the above structure, and may include more or fewer devices as needed, and may also combine or split the devices. For example, the intelligent network connection system may not include a user terminal and / or a server; for another example, the user terminal and the server may be deployed together.

[0066] Figure 2 is based on Figure 1 The illustrated embodiment provides a schematic diagram of a vehicle 101. Figure 2 As shown, the vehicle 101 may include a perception component 1011, a computing platform 1012, an execution component 1013, etc. The perception component 1011, the computing platform 1012, and the execution component 1013 may be connected via a bus or other means.

[0067] In some examples, the perception component 1011 can be used to collect information about the vehicle itself or outside. The perception component 1011 may include at least one of a visual sensor unit, a radar, a positioning and navigation unit, an inertial measurement unit (IMU) or other sensing units, wherein the visual sensor unit may include one or more cameras, the radar may include at least one of a laser radar, a millimeter wave radar, an ultrasonic radar or other radars, and the positioning and navigation unit may include at least one of a GPS system, a Beidou system or other global positioning systems.

[0068] In some examples, the computing platform 1012 may include a device with computing capabilities, which is used to process the perception information collected by the perception component 1011 to obtain control information, and send corresponding control instructions to the execution component 1013, so that the execution component 1013 performs corresponding actions, thereby realizing control of the vehicle 101. For example, the computing platform 1012 can perform positioning and mapping (Simultaneous Localization and Mapping, SLAM), path planning, behavior decision-making and other behaviors on the vehicle, thereby realizing autonomous control of the vehicle. The computing platform 1012 may include at least one processor and at least one memory, and each processor may execute instructions stored in the memory individually or collectively to implement the method provided in the embodiment of the present disclosure. The processor in the embodiment of the present disclosure may include a central processing unit (CPU), a graphic processing unit (GPU), a neural-network processing unit (NPU), a tensor processing unit (TPU), a data processing unit (DPU), a digital signal processor (DSP), a field programmable gate array (FPGA), a system on chip (SOC), an application specific integrated circuit (ASIC), a microcontroller unit (MCU) or at least one of other processors. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a disk or an optical disk. In addition to storing instructions, the memory can also store data, such as high-precision maps, path information, vehicle location, direction, speed and other data. The data stored in the memory can be retrieved and used by the processor.

[0069] In some examples, the vehicle's computing platform can independently perform computing tasks or communicate with a server to complete computing tasks. For example, the vehicle's computing platform can cooperate with the server to complete corresponding computing tasks.

[0070] The computing platform 1012 may be arranged in the vehicle 101, and part or all of the computing platform 1012 may also be arranged in a server corresponding to the vehicle. For example, some functions of the computing platform 1012 with higher real-time requirements may be arranged in the vehicle, and other functions with lower real-time requirements may be arranged in the server corresponding to the vehicle.

[0071] In some examples, the execution component 1013 is used to execute corresponding actions based on the control of the computing platform 1012, so that the vehicle 101 completes the moving task. The execution component 1013 may include, for example, a power component, a brake component, a transmission component, a steering component, and the like.

[0072] It should be noted that Figure 2 The structure of the vehicle 101 shown in the figure is only illustrative. The vehicle in the embodiment of the present disclosure is not limited to the above structure, and may include more or fewer components as needed, and the devices may be combined or split. For example, the vehicle may not include the above computing platform. For another example, the vehicle may also include a communication component, an interface component, a multimedia component, an input component, an output component, etc.

[0073] The disclosed embodiments can be applied to autonomous driving scenarios, where it is necessary to detect the position, speed and other state information of target objects in the vehicle's surrounding environment to assist in decision-making planning and control. The target objects can be dynamic objects such as other vehicles and pedestrians.

[0074] In related technologies, the environmental data around the vehicle can be analyzed and processed based on the target object detection model to obtain the state information of the target object. However, the results predicted by the model are inaccurate and jumpy, which affects the decision-making planning and control of the autonomous driving vehicle. For example, due to the limited sensor perception range of the vehicle, if the sensor's observation of the target object (such as another vehicle) is incomplete or the quality of the obtained sensor data is mixed, it will cause a large jump in the state information of the detected target object (such as position, physical size, speed, etc.), which will bring erroneous signals to the path decision planning of the vehicle and thus affect the normal and stable driving of the vehicle.

[0075] In response to the problems of related technologies, an embodiment of the present disclosure provides a target detection method, which can be based on the initial position information at multiple times, through a two-stage optimization solution, so that the determined target state information is more accurate and smooth in the time series dimension, avoiding the problem of state information jumps of the target object, thereby improving the accuracy of decision-making planning and control of the autonomous driving vehicle.

[0076] Figure 3 is a flow chart of a target detection method provided by an embodiment of the present disclosure. The target detection method can be Figure 1 The vehicle and / or server are executed as shown. Figure 3 As shown, the target detection method of this embodiment may include the following steps S310 to S330.

[0077] Step S310, obtaining initial position information of the target object at each of the multiple moments according to the environmental data at the multiple moments.

[0078] In some examples, the environmental data may include real-scene scanning data of the vehicle's external environment. Here, the "real scene" in the real-scene scanning data refers to the real world or the scene in the real world, that is, the external environment as the data collection object is the external environment that exists in the real world; and the "scan" in the real-scene scanning data is intended to reflect that the sensor collects data for the external environment to realize the scanning of the external environment, and is not intended to limit the specific way in which the sensor collects data. Among them, the real-scene scanning data may include at least one of the image data collected by the camera for the vehicle's external environment and the point cloud data collected by the radar for the vehicle's external environment. For example, the environmental data may include point cloud data, image data, or a combination of the two, and the environmental data may also be fused data obtained after feature fusion based on point cloud data and image data.

[0079] In some examples, the above-mentioned multiple moments may be a continuous time series, for example, the multiple moments may be a periodic sequence with a set time interval, and the set time interval may be any pre-set time interval such as 5 milliseconds, 20 milliseconds or 100 millimeters. For example, the real scene scanning data of the external environment of the vehicle may be periodically acquired through the sensor based on the set time interval as the environmental data. In this way, continuous environmental data in the time dimension may be obtained.

[0080] In some examples, the target object may be a dynamic object in the external environment of the vehicle. For example, the target object may include other vehicles (other vehicles other than the vehicle itself), pedestrians, animals and other dynamic objects.

[0081] For example, the environmental data can be input into a pre-generated target object detection model to detect the target object and the initial position information of the target object at each moment. If the environmental data is point cloud data, the target object detection model can be a point cloud detection model; if the environmental data is image data, the target object detection model can be an image detection model.

[0082] Step S320, based on the preset first detection target, according to the initial position information of the target object at multiple moments, determine the initial speed information of the target object at each moment, so that the initial speed information meets the first detection target.

[0083] In some examples, the initial velocity information may include a first velocity component of the target object along a first coordinate axis and a second velocity component of the target object along a second coordinate axis. The first coordinate axis and the second coordinate axis may be perpendicular to each other. For example, the first coordinate axis is an x-axis, and the first velocity component is used to indicate the velocity of the target object along the x-axis; the second coordinate axis is a y-axis, and the second velocity component is used to indicate the velocity of the target object along the y-axis.

[0084] In some examples, the first detection target may include a speed difference of the target object along the same coordinate axis at adjacent moments that is less than or equal to a preset speed threshold. The preset speed threshold may be a preset threshold, such as 0 or a value close to 0. It should be noted that the difference in the embodiment of the present disclosure may be an absolute value of the difference, for example, the speed difference may be an absolute value of the speed difference. Optionally, the first detection target may be referred to as a uniform motion constraint or a constant speed constraint.

[0085] In some examples, step S320 can solve the optimal solution of speed information that meets the first detection target through linear optimization or nonlinear optimization, and use it as initial speed information, which can also be called initial speed value.

[0086] Step S330, determining the target state information of the target object at each moment according to the initial position information and initial velocity information at multiple moments.

[0087] In some examples, the target state information may include target position information, target speed information, and target heading angle information of the target object.

[0088] In some examples, the target state information may include at least one of target position information, target speed information, target heading angle information, target acceleration information, target angular velocity information, and target size information of the target object.

[0089] The target state information of the target object may also be referred to as geometric attributes or properties.

[0090] It should be noted that the target position information and the initial position information may be the same or different. For example, the initial position information may be optimized and solved through the above steps to obtain new target position information. Optionally, the target speed information may include a speed scalar of the target object.

[0091] In some examples, step S330 can solve the optimal solution of the state information that meets the detection target by linear optimization or nonlinear optimization, as the target state information. In this way, a two-stage optimization solution for target object detection is implemented through steps S320 and S330. The first stage (step S320) obtains initial velocity information (the quasi-true value of the velocity) based on the first detection target, and the second stage (step S330) can use the initial velocity information as the initial value and observation item to determine the target state information of the target object (such as target position information, target velocity information, and target heading angle information), so that more stable and accurate target state information can be obtained.

[0092] In some examples, the above-mentioned multiple moments may be a continuous time series, and the time series changes of the target object may be obtained based on the environmental data of the target object in the continuous time series, so that the target state information of the detected target object is smoother in the time series dimension.

[0093] By adopting the method of steps S310 to S330, a new target detection method is provided, which obtains the initial position information of the target object at each of the multiple moments according to the environmental data at the multiple moments; based on the preset first detection target, the initial speed information of the target object at each moment is determined according to the initial position information of the target object at the multiple moments, so that the initial speed information meets the preset first detection target; and then the target state information of the target object at each moment is determined according to the initial position information and initial speed information at the multiple moments; wherein the target state information includes the target position information, target speed information and target heading angle information of the target object. In this way, based on the initial position information at the multiple moments, through the two-stage optimization solution, the determined target state information can be made more accurate and smooth in the time series dimension, avoiding the problem of state information jump of the target object, thereby improving the accuracy of decision-making planning and control of the autonomous driving vehicle.

[0094] In some embodiments of the present disclosure, the initial position information of the target object determined in the above step S310 may be information based on a detection box (BOX), and the detection box may be an area surrounding the above target object, and the position information of the target object (i.e., initial position information) may be represented by the detection box. Optionally, the detection box may also represent the size (i.e., initial size information) and heading angle (i.e., initial heading angle information) of the target object, and the size may include the length, width, and height of the detection box.

[0095] For example, the detection frame can be a two-dimensional detection frame or a three-dimensional detection frame (3DBOX), for example, the detection frame can be a rectangular frame or a cuboid. Taking the three-dimensional detection frame as an example, the attribute information of the three-dimensional detection frame can include one or more of the center point coordinates (x, y, z) of the target object, the size of the target object (length l, width w, height h), the heading angle (i.e., direction angle) of the target object, the category of the target object (such as vehicle, pedestrian, bicycle, etc.) and the object identification (ID) of the target object. Among them, the center point coordinates of the target object determined based on the detection frame can be used as the above-mentioned initial position information. Similarly, the size information of the target object determined based on the detection frame can also be called the initial size information of the target object, and the heading angle of the target object determined based on the detection frame can also be called the initial heading angle information of the target object.

[0096] In some embodiments, the initial position information may include the center point coordinates of the target object, the center point coordinates include the first coordinates of the center point of the target object on the first coordinate axis and the second coordinates of the second coordinate axis. For example, the first coordinate axis is the x-axis and the second coordinate axis is the y-axis, that is, the center point coordinates may include two-dimensional coordinates (x, y).

[0097] In some examples, the first detection target in the above step S320 may include constraint items determined based on the first coordinates, the second coordinates, the first velocity component, and the second velocity component of the target object at different times.

[0098] In this way, based on the constraint items determined by the first coordinate, the second coordinate, the first velocity component, and the second velocity component, the constraint items can be used to define the constraints and scope of the solution, so that the initial velocity information obtained by the solution meets the first detection goal, that is, the initial velocity information at multiple moments is more accurate and smooth in the time series dimension, thereby avoiding speed jumps at adjacent moments.

[0099] For example, the constraint item of the first detection target may include at least one of the following first to fourth constraint items:

[0100] The first constraint item: the difference between the first coordinate difference and the first movement distance of the center point of the target object between the first moment and the second moment is less than or equal to a first preset threshold; wherein the first moment is a moment earlier than the second moment and adjacent to the second moment among multiple moments, for example, the first moment is the i-1th moment, and the second moment is the i-th moment, the first coordinate difference is the difference between the first coordinate of the center point at the first moment and the first coordinate of the second moment, the first movement distance is the distance calculated based on the first velocity component and the time difference between the first moment and the second moment, and the first preset threshold can be a preset value of 0 or close to 0.

[0101] Taking the first preset threshold as 0 as an example, the first constraint item can be expressed by the following formula (1):

[0102]

[0103] in, represents the first coordinate of the center point of the target object at the i-th moment (for example, the second moment mentioned above), represents the first coordinate of the center point of the target object at the i-1th moment (for example, the first moment mentioned above), represents the first velocity component of the target object on the first coordinate axis (x-axis) at the i-th moment, and Δt represents the time difference between the i-th moment and the i-1-th moment.

[0104] The second constraint item: the difference between the second coordinate difference of the center point of the target object at the first moment and the second moment and the second movement distance is less than or equal to a second preset threshold; wherein the second coordinate difference is the difference between the second coordinate of the center point at the first moment and the second coordinate of the center point at the second moment, the second movement distance is a distance calculated based on the second velocity component and the time difference between the first moment and the second moment, and the second preset threshold can be a preset value of 0 or close to 0.

[0105] Taking the second preset threshold as 0 as an example, the second constraint item can be expressed by the following formula (2):

[0106]

[0107] in, represents the second coordinate of the center point of the target object at the i-th moment (for example, the second moment mentioned above), represents the second coordinate of the center point of the target object at the i-1th moment (for example, the first moment mentioned above), represents the second velocity component of the target object on the second coordinate axis (x-axis) at the i-th moment, and Δt represents the time difference between the i-th moment and the i-1-th moment.

[0108] Third constraint: a difference between the first velocity component of the target object at the first moment and the first velocity component of the target object at the second moment is less than or equal to a third preset threshold, which may be 0 or a preset value close to 0.

[0109] Taking the third preset threshold as 0 as an example, the third constraint item can be expressed by the following formula (3):

[0110]

[0111] in, represents the first velocity component of the target object at the i-th moment (for example, the second moment mentioned above), Represents the first velocity component of the target object at the i-1th moment (for example, the first moment mentioned above).

[0112] Fourth constraint: a difference between the second velocity component of the target object at the first moment and the second velocity component of the target object at the second moment is less than or equal to a fourth preset threshold, which may be a preset value of 0 or close to 0.

[0113] Taking the fourth preset threshold as 0 as an example, the fourth constraint item can be expressed by the following formula (4):

[0114]

[0115] in, represents the second velocity component of the target object at the i-th moment (for example, the second moment mentioned above), Represents the second velocity component of the target object at the i-1th moment (for example, the first moment mentioned above).

[0116] It should be noted that the first to fourth constraints mentioned above are constructed individually or in any combination to obtain the first detection target. For example, the first detection target may include the first constraint item and the third constraint item, or the second constraint item and the fourth constraint item, or the first constraint item and the second constraint item, or the third constraint item and the fourth constraint item, or the first to fourth constraints as a whole.

[0117] In this way, based on the above-mentioned first to fourth constraints, the target object can be constrained to move at a uniform speed on the x-axis and / or y-axis, thereby avoiding speed jumps at adjacent moments, making the initial speed information at multiple moments more accurate and smooth in the time series dimension.

[0118] In some examples, the initial position information may be three-dimensional position information of the target object, the center point coordinates may further include the third coordinates of the center point of the target object on the third coordinate axis, and the initial velocity information may further include the third velocity component of the target object on the third coordinate axis. The third coordinate axis may be perpendicular to the first coordinate axis and the second coordinate axis to form a three-dimensional coordinate axis. For example, the first coordinate axis may be the x-axis, the second coordinate axis may be the y-axis, and the third coordinate axis may be the z-axis, that is, the center point coordinates may be three-dimensional coordinates (x, y, z), and the initial velocity information may include the velocity components of the target object along the x-axis, y-axis, and z-axis directions.

[0119] In some examples, the constraint item of the first detection target may further include at least one of the following fifth constraint item to sixth constraint item:

[0120] Fifth constraint item: the difference between the third coordinate difference of the center point at the first moment and the second moment and the third movement distance is less than or equal to the fifth preset threshold; wherein the third coordinate difference is the difference between the third coordinate of the center point at the first moment and the third coordinate of the second moment, the second movement distance is the distance calculated based on the third velocity component and the time difference between the first moment and the second moment, and the first preset threshold can be a preset value of 0 or close to 0.

[0121] Taking the fifth preset threshold as 0 as an example, the fifth constraint item can be expressed by the following formula (5):

[0122]

[0123] in, represents the third coordinate of the center point of the target object at the i-th moment (for example, the second moment mentioned above), represents the third coordinate of the center point of the target object at the i-1th moment (for example, the first moment mentioned above), represents the third velocity component of the target object on the third coordinate axis (z axis) at the i-th moment, and Δt represents the time difference between the i-th moment and the i-1-th moment.

[0124] Sixth constraint: the difference between the third velocity component of the target object at the first moment and the third velocity component of the target object at the second moment is less than or equal to a sixth preset threshold, which may be a preset value of 0 or close to 0.

[0125] Taking the sixth preset threshold as 0 as an example, the sixth constraint item can be expressed by the following formula (6):

[0126]

[0127] in, represents the third velocity component of the target object at the i-th moment (for example, the second moment mentioned above), Represents the third velocity component of the target object at the i-1th moment (eg, the first moment mentioned above).

[0128] In this way, based on the fifth to sixth constraints, the target object can be constrained to move at a uniform speed on the z-axis, thereby avoiding speed jumps at adjacent moments, making the initial speed information at multiple moments more accurate and smooth in the time series dimension.

[0129] In some embodiments of the present disclosure, the above step S330 can be implemented based on the following method:

[0130] Based on the preset second detection target, the target state information of the target object at each moment is determined according to the initial position information and the initial speed information at multiple moments, so that the target state information meets the second detection target.

[0131] In some examples, the second detection target may include a constraint condition determined based on a state quantity of the target object, the state quantity may include at least one of the center point coordinates, size, heading angle, speed, acceleration, and angular velocity of the target object, and the size may include at least one of the length, width, and height of the target object. The state quantity may also be referred to as an optimized state quantity, that is, an optimized state quantity set in an optimization model corresponding to the second detection target, and the optimization model may be a multi-constrained optimization model.

[0132] For example, the state quantity X can be expressed as the following formula (7):

[0133]

[0134] Wherein, X represents the state quantity, x represents the first coordinate of the center point of the target object on the first coordinate axis (x-axis), y represents the second coordinate of the center point of the target object on the second coordinate axis (y-axis), θ represents the heading angle of the target object, v represents the speed of the target object, Ω represents the angular velocity of the target object, and a represents the acceleration of the target object.

[0135] It should be noted that the above formula (7) is only an example, and the state quantity may include more or fewer parameters, for example, it may also include the size of the target object: at least one of the length l, width w, and height h.

[0136] In some examples, the second detection target may include at least one of the following:

[0137] The acceleration difference of the target object at adjacent moments is less than or equal to a preset acceleration threshold (e.g., 0);

[0138] The angular velocity difference of the target object at adjacent moments is less than or equal to a preset angular velocity threshold (eg, 0).

[0139] For example, constant acceleration and constant angular velocity (constant turning rate) may be used as basic constraints of the second detection target to determine the constraints corresponding to the second detection target.

[0140] Taking the example that the preset acceleration threshold and the preset angular velocity threshold are both 0, the second detection target may include the constraint item represented by the following formula (8):

[0141]

[0142] Among them, the i-1th moment is the previous moment adjacent to the i-th moment among multiple moments, x i represents the first coordinate of the center point of the target object at the i-th moment (for example, the second moment), y i represents the second coordinate of the center point of the target object at the i-th moment, θi represents the heading angle of the target object at the i-th moment, v i represents the speed of the target object at the i-th moment, Ω i represents the angular velocity of the target object at the i-th moment, a i represents the acceleration of the target object at the i-th moment, x i-1 represents the first coordinate (e.g., the coordinate on the x-axis of the first coordinate axis) of the center point of the target object at the i-1th moment (e.g., the first moment), y i-1 represents the second coordinate of the center point of the target object at the i-1th moment (for example, the coordinate of the second coordinate axis y axis), θ i-1 represents the heading angle of the target object at the i-1th moment, v i-1 represents the speed of the target object at the i-1th moment, Ω i-1 represents the angular velocity of the target object at the i-1th moment, a i-1 represents the acceleration of the target object at the i-1th moment, δt represents the time difference between the i-th moment and the i-1th moment, represents the displacement of the target object along the x-axis within time δt, represents the displacement of the target object along the y-axis within time δt, Ω i-1 *δt represents the angle change of the target object within the time δt, a i *δt represents the speed change of the target object within time δt.

[0143] In some examples, the second detection target may include: a difference between a target speed predicted at each moment and an initial speed of the target object is less than or equal to a preset difference, and the initial speed is a speed value calculated based on the initial speed information.

[0144] Taking the preset difference value as 0 as an example, the second detection target may include the constraint item represented by the following formula (9):

[0145]

[0146] The subscript i can represent any moment in multiple moments. It represents the first velocity component of the target object on the first coordinate axis x-axis at the i-th moment, that is represents the first velocity component in the initial velocity information determined in step S320, It represents the second velocity component of the second coordinate axis y-axis of the target object at the i-th moment, that is represents the second velocity component in the initial velocity information determined in step S320, v i Represents the target speed scalar of the target object at the i-th moment, that is, the scalar representation of the target speed information of the optimized target in step S330.

[0147] In this way, the target speed of the vehicle is constrained by using the initial speed information determined in step S302, which not only enables the optimization speed to converge quickly, but also makes other optimization results in the target state information more accurate in conjunction with other constraint items.

[0148] In some examples, the second detection target may further include constraints corresponding to multiple parts of the target object, wherein: the constraints of different parts of the target object correspond to different confidences, and the confidence of the constraints of each part is determined based on the number of position points corresponding to the part in the environmental data; the confidence is used to characterize the weight of the constraint of the part among the multiple constraint conditions of the part, and optionally, the confidence can be any value between 0 and 1, and the greater the confidence, the greater the weight.

[0149] In some examples, if the number of location points of a certain part is greater, the confidence of the part is higher, that is, the weight is higher.

[0150] In some examples, the multiple parts of the target object include the head and tail of the target object, and the constraints of the various parts of the target object can be constraints determined based on the center point coordinates of the part, the center point coordinates of the target object, the size of the target object, and the heading angle of the target object.

[0151] For example, the constraint conditions corresponding to the head and tail of the target object included in the second detection target can be expressed by the following formulas (10) and (11):

[0152]

[0153] The subscript i can represent any moment in multiple moments, and x i Indicates the first coordinate (x-axis coordinate) of the center point of the target object at the i-th moment, y i represents the second coordinate (y-axis coordinate) of the center point of the target object at the i-th moment, l i represents the length of the target object at the i-th moment, θ i represents the heading angle of the target object at the i-th moment, and represents the observed value of the head center point of the target object (i.e., the x-axis coordinate and y-axis coordinate of the head center point of the detection frame obtained based on the target object detection model), and Represents the observation value of the tail center point of the target object (the x-axis coordinate and y-axis coordinate of the tail center point of the detection box obtained based on the target object detection model).

[0154] Above and It can be expressed as the following formulas (12) and (13):

[0155]

[0156] in, Represents the x-axis coordinate of the center point of the target object's head at the i-th moment, represents the x-axis coordinate of the center point of the target object detected based on the target object detection model at the i-th moment (i.e., the observed value of the x-axis coordinate of the center point), Represents the y-axis coordinate of the center point of the target object's head at the i-th moment, represents the y-axis coordinate of the center point of the target object detected based on the target object detection model at the i-th moment (i.e., the observed value of the y-axis coordinate of the center point), represents the length of the target object in its initial size at the i-th moment (i.e., the observed value of the physical size length) detected based on the target object detection model. It represents the initial heading angle (i.e., the observed heading angle) of the target object at the i-th moment detected based on the target object detection model.

[0157] Above and It can be expressed as the following formulas (14) and (15):

[0158]

[0159] in, Represents the x-axis coordinate of the center point of the tail of the target object at the i-th moment, represents the x-axis coordinate of the center point of the target object detected based on the target object detection model at the i-th moment (i.e., the observed value of the x-axis coordinate of the center point), Indicates the y-axis coordinate of the center point of the tail of the target object at the i-th moment, represents the y-axis coordinate of the center point of the target object detected based on the target object detection model at the i-th moment (i.e., the observed value of the y-axis coordinate of the center point), represents the length of the target object in its initial size at the i-th moment (i.e., the observed value of the physical size length) detected based on the target object detection model. It represents the initial heading angle (i.e., the observed heading angle) of the target object at the i-th moment detected based on the target object detection model.

[0160] Based on the center point of the head, the number of position points corresponding to the head can be determined, and then the confidence of the constraints corresponding to the head can be determined; similarly, based on the center point of the tail, the number of position points corresponding to the tail can be determined, and then the confidence of the constraints corresponding to the tail can be determined.

[0161] For example, based on the detection frame of the target object (including initial position information, initial size information and initial heading angle information), multiple position points located in the detection frame can be screened out from the point set of the original environmental data (such as the point set of point cloud data, or the pixel set of image data) as the first point set P i .

[0162] According to the coordinates of the center point of the head, the position points whose distance from the head is less than or equal to a specific distance are selected from the first point set as the position point set corresponding to the head, thereby obtaining the number of position points corresponding to the head. The specific distance may be the width w of the target object or any other set value.

[0163] Furthermore, the confidence of the constraint condition corresponding to the head can be determined based on the number of position points corresponding to the head. For example, the confidence of the head can be calculated by the following formula (16):

[0164]

[0165] in, represents the confidence of the constraint corresponding to the head, F i represents the set of position points corresponding to the head, |F i | represents the number of position points corresponding to the head, M is a preset parameter, which is used to indicate that when the number of position points corresponding to the head is greater than or equal to M, the confidence of the head is a maximum value of 1, and M can be any pre-set positive integer, such as 20 or 100.

[0166] According to the coordinates of the center point of the tail, the position points whose distance from the center point of the tail is less than or equal to a specific distance are selected from the first point set as the position point set corresponding to the tail, thereby obtaining the number of position points corresponding to the tail. The specific distance may be the width w of the target object.

[0167] Furthermore, the confidence of the constraint condition corresponding to the tail can be determined based on the number of position points corresponding to the tail. For example, the confidence of the tail can be calculated by the following formula (17):

[0168]

[0169] in, represents the confidence of the constraint corresponding to the tail, B i represents the set of position points corresponding to the tail, |B i | represents the number of position points corresponding to the tail, M is a preset parameter, which is used to indicate that if the number of position points corresponding to the tail is greater than or equal to M, the confidence of the tail is a maximum value of 1, and M can be any pre-set positive integer, such as 20 or 100.

[0170] In this way, when constructing the constraint conditions corresponding to the second detection target, different confidence levels are set for different parts of the target object, which can further improve the accuracy of the target state information obtained based on the constraint conditions. Taking the target object as another car and the environmental data as point cloud data as an example, the following is explained: if the other car is directly in front of the self-car, only the tail of the other car can be observed and the point cloud is also distributed at the tail of the other car. At this time, the detection frame (such as 3DBOX) obtained based on the target object detection model (such as the deep learning model) is very accurate in fitting the tail of the other car, but its physical size length and the center point fitting of the head of the other car are very low in accuracy; on the contrary, if the other car is directly behind the self-car, only the head of the other car can be observed and the point cloud is also distributed at the head of the other car. At this time, the detection frame obtained based on the target object detection model is very accurate in fitting the head of the other car, but its physical size length and the center point fitting of the tail of the other car are very low in accuracy. Therefore, the need to consider the observation covariance when constructing the constraint can be achieved by setting the confidence level of different parts, so that the entire target detection result is smoother and the detection accuracy is higher.

[0171] In some examples, the second detection target may further include an observation constraint based on a physical dimension length. For example, the constraint based on a physical dimension length may be expressed by the following formula (18):

[0172]

[0173] Among them, l i The target size length (i.e., the state quantity of the physical size length) in the target state information of the target object is represented. It represents the length of the initial size of the target object detected based on the target object detection model (i.e., the observed value of the physical size length).

[0174] In some examples, the second detection target may also include an observation constraint based on a heading angle. For example, the constraint based on the physical dimension length may be expressed by the following formula (19):

[0175]

[0176] Among them, θ i represents the target heading angle (i.e. the state quantity of the heading angle) in the target state information of the target object, It represents the initial heading angle of the target object (i.e., the observed heading angle) detected based on the target object detection model.

[0177] In this way, the accuracy of the detection results can be further increased and the precision of the target detection results can be improved.

[0178] In some examples, in the above step S330, the above one or more constraints included in the second detection target can be added to the same nonlinear optimization problem, and then a preset nonlinear optimization algorithm (for example, based on the Gauss-Newton iteration method) is used to iteratively update and solve the target state information of the target object (for example, one or more of the center point position, size, speed, acceleration and angular velocity of the target object), so that smoother and more accurate state information of the target object can be solved, thereby improving the accuracy and smoothness of target detection.

[0179] In some embodiments of the present disclosure, the above step S310 may include the following steps S3101 and S3102.

[0180] Step S3101, based on the target object detection model, determine at least one candidate object at each moment and the attribute information of the candidate object at the moment according to the environmental data at multiple moments.

[0181] The attribute information includes location information of the candidate object, and size information and / or heading angle information of the candidate object.

[0182] In some examples, the position information may be position information based on a world coordinate system.

[0183] For example, based on the target object detection model, the position information of at least one candidate object at each moment in the vehicle coordinate system can be determined; according to the vehicle posture information, the position information in the vehicle coordinate system is converted into the position information in the world coordinate system, so that the position information in the world coordinate system is used as the position information in the attribute information of the target object.

[0184] The vehicle coordinate system can also be called the vehicle's own coordinate system or the body coordinate system. It is a coordinate system established with the vehicle as the reference point. The origin of the vehicle coordinate system can be the center point of the vehicle, and the direction of the coordinate axis can be determined according to the direction of movement of the vehicle. For example, the vehicle coordinate system contains the x-axis, y-axis, and z-axis. The x-axis can point to the direction of the vehicle's forward movement, the y-axis can point to the left side of the vehicle, and the z-axis can be perpendicular to the ground where the vehicle is located and point to the top of the vehicle. The vehicle coordinate system can be used to describe the state of the vehicle's motion relative to itself and the vehicle's surrounding environment, such as the vehicle's speed, acceleration, steering angle, etc.

[0185] The world coordinate system, also known as the global coordinate system, can establish a global reference frame to describe the global position and orientation of the vehicle in a map or the entire environment. The origin of the world coordinate system can be a selected fixed point, such as a geographical location or a specific reference point on a map, and the direction of the coordinate axis can also be a pre-set direction, such as the absolute directions of the earth's east, south, west and north. For example, the x-axis of the world coordinate system can point to the north of the earth's surface, the y-axis can point to the east of the earth's surface, and the z-axis can be perpendicular to the earth's surface and point above the ground.

[0186] During vehicle driving, since the origin of the vehicle coordinate system may be different at different times, after converting the position information of the target object from the vehicle coordinate system to the world coordinate system, it is beneficial to align the position information at different times to the same coordinate origin to facilitate the processing of multiple position information.

[0187] In some examples, the vehicle's attitude information may include the vehicle's position and heading angle. Optionally, the vehicle's attitude information may be obtained based on SLAM (Simultaneous Localization and Mapping). For example, based on SLAM, the vehicle's positioning may be achieved, thereby obtaining the vehicle's position and heading angle, and obtaining the vehicle's attitude information.

[0188] Step S3102, match the candidate objects according to the similarity of the attribute information between the candidate objects at different times, determine the attribute information of the same candidate object at each time, and use the candidate object as the target object, and use the position information in the attribute information of the candidate object as the initial position information of the target object at each time.

[0189] For example, the environmental data at each moment can be called a frame of environmental data, the attribute information of the candidate objects can be represented in the form of detection frames, and the similarity of the attribute information between the candidate objects at different moments can be the intersection over union ratio. In this way, the detection frames of the candidate objects in each frame of environmental data can be matched by calculating the intersection over union (IOU) of the detection frames pairwise to obtain the detection frame sequence of each candidate object at each moment, thereby obtaining the initial position information of the target object (i.e., the candidate object) at each moment.

[0190] Taking the environmental data at two moments before and after as an example, three candidate objects A1, B1 and C1 are detected in the environmental data at the first moment, and two candidate objects A2 and B2 are detected in the environmental data at the second moment. The intersection-and-union ratio between each two is calculated according to the attribute information (such as detection boxes) of different candidate objects to determine which candidate objects in the environmental data at these two moments represent the same object. For example, by comparing their intersection-and-union ratios, it can be determined that A1 and A2 represent the same object, and B1 and B2 represent another object, so that the attribute information of the target objects A and B at the two moments can be obtained, and the initial position information of the target objects A and B at the two moments can also be obtained.

[0191] In this way, the initial position information of the target object at each of the multiple moments can be accurately obtained based on the environmental data at the multiple moments.

[0192] In some embodiments of the present disclosure, the above environmental data can be obtained by any of the following methods:

[0193] In some examples, the above environmental data can be collected by the vehicle through its own sensors. For example, the environmental data may include point cloud data collected by a radar (such as a lidar) of the vehicle, image data collected by a visual acquisition unit (such as a camera) of the vehicle, or fused data obtained by feature fusion of point cloud data and image data.

[0194] In other examples, the environmental data may also be collected by other sensors other than the vehicle. For example, the environmental data may include image data of the vehicle's surrounding environment collected based on sensor equipment (such as a camera) installed on the road.

[0195] In some other examples, the environmental data may also be collected by the vehicle's own sensors and other sensors outside the vehicle. For example, the environmental data may include point cloud data collected by the vehicle's radar, image data of the vehicle's surrounding environment collected by sensor equipment installed on the road, or fused data obtained by feature fusion of point cloud data and image data.

[0196] In this way, environmental data can be obtained in multiple ways to improve the accuracy of target detection.

[0197] It should be noted that the above-mentioned environmental data may also be other data collected by other means, and the embodiments of the present disclosure do not limit the specific category and acquisition method of the environmental data.

[0198] Figure 4 is a flow chart of another target detection method provided by an embodiment of the present disclosure. The target detection method can be Figure 1The vehicle and / or server are executed as shown. Figure 4 As shown, the target detection method of this embodiment may include the following steps S410 to S460.

[0199] Step S410, determining a detection frame at each moment according to environmental data at multiple moments.

[0200] For example, based on a target object detection model (such as a deep learning point cloud detection model), at least one detection box at each moment can be detected in the environmental data collected by the vehicle at multiple moments. The detection box can surround the candidate object. For example, the detection box can include the candidate object and the attribute information of the candidate object at that moment, that is, the attribute information of the candidate object can be implemented based on the detection box.

[0201] Step S420, determining the posture information of the vehicle at each of the multiple moments.

[0202] For example, the SLAM technology can be used to calculate the posture of the vehicle's motion at the timestamp of each frame of environmental data (i.e., at each moment) as the vehicle's posture information. The posture information may include the vehicle's position and heading angle, and the position may be the vehicle's position coordinates in the world coordinate system.

[0203] Step S430: convert the coordinates of the detection frame into coordinates in the world coordinate system according to the vehicle body posture.

[0204] Step S440: Match and track the detection frames at multiple moments to determine the detection frame of the target object at each moment, where the detection frame includes initial position information of the target object.

[0205] For example, the detection frames at multiple moments may be matched in the world coordinate system by calculating the intersection over union (IOU) of the detection frames in pairs, to obtain the detection frames of each target object at each moment.

[0206] In some examples, candidate objects can be matched based on the similarity of attribute information between candidate objects at different times, attribute information of the same candidate object at each time can be determined, and the candidate object can be used as the target object, and the position information in the attribute information of the candidate object can be used as the initial position information of the target object at each time. The detection box can include the initial position information of the target object.

[0207] Step S450, based on the preset first detection target, according to the initial position information of the target object at multiple moments, the initial velocity information of the target object at each moment is determined, so that the initial velocity information meets the first detection target.

[0208] The specific implementation of step S450 may refer to the description of step S320 in the aforementioned embodiment of the present disclosure, and will not be repeated here.

[0209] Step S460, determining the target state information of the target object at each moment according to the initial position information and initial velocity information at multiple moments.

[0210] In some examples, the target state information includes at least one of target position information, target speed information, target heading angle information, target acceleration information, target angular velocity information, and target size information of the target object.

[0211] The specific implementation of step S460 may refer to the description of step S330 in the aforementioned embodiment of the present disclosure, which will not be repeated here.

[0212] The above method uses the detection results (such as initial position information) based on the target object detection model and the vehicle posture, combined with the kinematic model.

[0213] A two-stage optimization solution is adopted, that is, the first stage optimizes to obtain the initial velocity information of the target object (the quasi-true value of the velocity), and the initial velocity information can be used as the initial value and observation item of the second stage optimization. Then, in the second stage, based on the constant acceleration constraint and / or constant angular velocity constraint in the kinematic model, the center points of different parts of the target object are used as observation items and different confidence levels are calculated respectively. The heading angle constraint and the physical size length constraint are integrated to optimize the overall state information of the target object, and output more stable and accurate target state information, so that the vehicle can also be more stable in the process of planning and controlling based on the target state information of the target object, so that the user can have a more comfortable experience.

[0214] The above two-stage optimization target detection method in the embodiment of the present disclosure has smaller error and higher accuracy than the single-stage optimization target detection method (considering only the constant speed model) in the related art. Specifically, under the same test conditions (the vehicle and road conditions are the same), the error comparison of the two methods is shown in the following table:

[0215]

[0216] In the above table, the longitudinal velocity absolute value error, the lateral velocity absolute value error, the longitudinal position absolute value error, the lateral position absolute value error and the heading angle absolute value error are all the differences between the detection results and the actual results obtained based on the corresponding target detection method. Among them: the longitudinal velocity can be the first velocity component of the target velocity of the target object along the first coordinate axis, the lateral velocity can be the second velocity component of the target velocity of the target object along the second coordinate axis, the longitudinal position can be the first coordinate of the target position of the target object on the first coordinate axis, the lateral position can be the second coordinate of the target position of the target object on the second coordinate axis, and the heading angle can be the target heading angle of the target object.

[0217] In some embodiments of the present disclosure, the target detection method can be used in data labeling. For example, multiple environmental data are acquired through a data collection vehicle. Based on the target detection method, the environmental data can be labeled to determine the target object and the target state information of the target object. The labeled data can be used to train the model of the autonomous driving vehicle to improve the accuracy of the vehicle's detection of target objects.

[0218] In some other embodiments of the present disclosure, the target detection method may be used in vehicle control. For example, target state information of a target object may be determined based on the target detection method, and vehicle driving may be controlled based on the target state information.

[0219] Figure 5 is a flow chart of a vehicle control method provided by an embodiment of the present disclosure. The vehicle control method can be Figure 1 The vehicle and / or server are executed as shown. Figure 5 As shown, the vehicle control method of this embodiment may include:

[0220] Step S510, determining a target object in the external environment where the vehicle is located.

[0221] Step S520: Control the vehicle to travel according to the target state information of the target object.

[0222] Among them, the target state information is the state information of the target object at each moment determined according to the initial position information and initial speed information of the target object at multiple moments, the initial position information is the position information of the target object obtained according to the environmental data at multiple moments, and the initial speed information is the initial speed information of the target object at each moment determined based on the preset first detection target and according to the initial position information; the environmental data includes real-scene scanning data of the external environment of the vehicle, the target object is a dynamic object in the external environment of the vehicle, the initial speed information includes a first speed component of the target object along the first coordinate axis and a second speed component of the target object along the second coordinate axis, the first detection target includes a speed difference of the target object along the same coordinate axis at adjacent moments that is less than or equal to a preset speed threshold, and the target state information includes target position information, target speed information and target heading angle information of the target object.

[0223] For the method of acquiring the target state information of the target object in this embodiment, reference may be made to the description in the foregoing embodiments of the present disclosure, which will not be repeated here.

[0224] In this way, since the detection results of the target object are smoother and more accurate, the vehicle planning and control process is smoother, giving users a more comfortable experience.

[0225] Figure 6 Schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Figure 6 As shown, the electronic device 1000 may include a memory 1010 and a processor 1020. The memory 1010 may be used to store computer instructions, and the processor 1020 may be used to call computer instructions from the memory 1010 to execute all or part of the steps of any method in the aforementioned embodiments of the present disclosure. The processor may be one or more, and the one or more processors may execute instructions individually or together, and the memory may also be one or more, and the one or more memories may store the above-mentioned computer instructions individually or together.

[0226] The embodiment of the present disclosure also provides a vehicle, which may include a memory and a processor, wherein the memory may be used to store computer instructions, and the processor may be used to call computer instructions from the memory to execute all or part of the steps of any method in the aforementioned embodiment of the present disclosure. The processor may be one or more, and the one or more processors may execute instructions individually or collectively, and the memory may also be one or more, and the one or more memories may store the above-mentioned computer instructions individually or collectively.

[0227] The vehicle in the foregoing embodiments of the present disclosure may be an electric vehicle, a hybrid vehicle, a fuel cell vehicle, or other types of vehicles. The vehicle may be an autonomous vehicle or a non-autonomous vehicle. For example, the vehicle provided in this embodiment may be Figure 1 or Figure 2 Vehicle shown.

[0228] The embodiments of the present disclosure also provide a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, all or part of the steps of any method in the aforementioned embodiments of the present disclosure are implemented. Optionally, the computer-readable storage medium may be a non-transitory storage medium, but is not limited thereto, and may also be a transient storage medium.

[0229] The present disclosure also provides a chip, which may include a processing unit, which may be used to execute all or part of the steps of any method in the foregoing embodiments of the present disclosure. The chip may be a chip in the form of an application-specific integrated chip ASIC, a system on chip SOC, a field programmable gate array FPGA, etc., which is not limited in the present embodiment.

[0230] The embodiments of the present disclosure further provide a computer program product, which may include a computer program. When the computer program is executed by a processor, any method in the aforementioned embodiments of the present disclosure may be implemented.

[0231] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement any method in the foregoing embodiments of the present disclosure.

[0232] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0233] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0234] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, which may include object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0235] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0236] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so as to produce a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device for implementing the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions make the computer, programmable data processing device, and / or other equipment work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0237] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0238] The flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of an instruction, and the module, a program segment or a part of an instruction contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions. It should be noted that implementation by hardware, implementation by software, and implementation by a combination of software and hardware are all equivalent.

[0239] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the present disclosure is defined by the appended claims.

Claims

1. A target detection method, characterized in that: The method comprises: According to the environmental data at multiple moments, the initial position information of the target object at each of the multiple moments is obtained; wherein the environmental data includes real-scene scanning data of the external environment of the vehicle, and the target object is a dynamic object in the external environment of the vehicle; Based on a preset first detection target, initial speed information of the target object at each moment is determined according to initial position information of the target object at multiple moments, so that the initial speed information meets the first detection target; wherein the initial speed information includes a first speed component of the target object along a first coordinate axis and a second speed component of the target object along a second coordinate axis, and the first detection target includes a speed difference of the target object along the same coordinate axis at adjacent moments being less than or equal to a preset speed threshold; According to the initial position information and initial speed information at the multiple moments, the target state information of the target object at each moment is determined; wherein the target state information includes the target position information, target speed information and target heading angle information of the target object.

2. The method according to claim 1, characterized in that The initial position information includes the center point coordinates of the target object, and the center point coordinates include a first coordinate of the center point of the target object on the first coordinate axis and a second coordinate of the second coordinate axis; The first detection target includes constraint items determined based on a first coordinate, a second coordinate, a first velocity component, and a second velocity component of the target object at different times.

3. The method according to claim 2, characterized in that The constraint item of the first detection target includes at least one of the following: The difference between the first coordinate difference and the first movement distance of the center point of the target object between the first moment and the second moment is less than or equal to a first preset threshold; wherein the first moment is a moment earlier than the second moment and adjacent to the second moment among the multiple moments, the first coordinate difference is the difference between the first coordinate of the center point at the first moment and the first coordinate of the second moment, and the first movement distance is a distance calculated based on the first velocity component and the time difference between the first moment and the second moment; The difference between the second coordinate difference of the center point of the target object between the first moment and the second moment and the second movement distance is less than or equal to a second preset threshold; wherein the second coordinate difference is the difference between the second coordinate of the center point at the first moment and the second coordinate at the second moment, and the second movement distance is the distance calculated based on the second velocity component and the time difference between the first moment and the second moment; The difference between the first velocity component of the target object at the first moment and the first velocity component of the target object at the second moment is less than or equal to a third preset threshold; The difference between the second velocity component of the target object at the first moment and the second velocity component of the target object at the second moment is less than or equal to a fourth preset threshold.

4. The method according to claim 3, characterized in that The initial position information is the three-dimensional position information of the target object, the center point coordinates also include the third coordinates of the center point of the target object on the third coordinate axis, and the initial velocity information also includes the third velocity component of the target object along the direction of the third coordinate axis; The constraint item of the first detection target also includes at least one of the following: The difference between the third coordinate difference of the center point at the first moment and the second moment and the third movement distance is less than or equal to a fifth preset threshold; wherein the third coordinate difference is the difference between the third coordinate of the center point at the first moment and the third coordinate of the center point at the second moment, and the second movement distance is the distance calculated based on the third velocity component and the time difference between the first moment and the second moment; A difference between the third velocity component of the target object at the first moment and the third velocity component of the target object at the second moment is less than or equal to a sixth preset threshold.

5. The method according to claim 1, characterized in that The step of determining target state information of the target object at each moment according to the initial position information and initial velocity information at the plurality of moments comprises: Based on a preset second detection target, determining the target state information of the target object at each moment according to the initial position information and the initial speed information at the multiple moments, so that the target state information meets the second detection target; The second detection target includes constraints determined based on a state quantity of the target object, and the state quantity includes at least one of the center point coordinates, size, heading angle, speed, acceleration and angular velocity of the target object.

6. The method according to claim 5, characterized in that The second detection target includes: a difference between a target speed predicted at each moment and an initial speed of the target object is less than or equal to a preset difference, and the initial speed is a speed value calculated according to the initial speed information.

7. The method according to claim 5, characterized in that The second detection target includes constraints corresponding to multiple parts of the target object, wherein: The constraints of different parts of the target object correspond to different confidence levels, and the confidence level of the constraints of each part is determined based on the number of position points corresponding to the part in the environmental data; the confidence level is used to characterize the weight of the constraints of the part among multiple part constraints.

8. The method according to claim 7, characterized in that The multiple parts of the target object include a head and a tail of the target object; The constraint conditions of each part of the target object are constraint conditions determined based on the center point coordinates of the part, the center point coordinates of the target object, the size of the target object, and the heading angle of the target object.

9. The method according to claim 5, characterized in that The second detection target includes at least one of the following: The acceleration difference of the target object at adjacent moments is less than or equal to a preset acceleration threshold; The angular velocity difference of the target object at adjacent moments is less than or equal to a preset angular velocity threshold.

10. The method according to any one of claims 1 to 9, characterized in that The step of acquiring the initial position information of the target object at each of the multiple moments according to the environmental data at the multiple moments includes: Based on the target object detection model, determine at least one candidate object at each moment and attribute information of the candidate object at the moment according to the environmental data; wherein the attribute information includes position information of the candidate object, and size information and / or heading angle information of the candidate object; The candidate objects are matched according to the similarity of the attribute information between the candidate objects at different times, the attribute information of the same candidate object at each time is determined, and the candidate object is used as the target object, and the position information in the attribute information of the candidate object is used as the initial position information of the target object at each time.

11. A vehicle control method, characterized in that: The method comprises: Determine the target object in the external environment where the vehicle is located; Controlling the vehicle to travel according to the target state information of the target object; Among them, the target state information is the state information of the target object at each moment determined according to the initial position information and initial speed information of the target object at multiple moments, the initial position information is the position information of the target object obtained according to the environmental data at multiple moments, and the initial speed information is the initial speed information of the target object at each moment determined based on a preset first detection target and according to the initial position information; the environmental data includes real-scene scanning data of the external environment of the vehicle, the target object is a dynamic object in the external environment of the vehicle, the initial speed information includes a first speed component of the target object along the first coordinate axis and a second speed component of the target object along the second coordinate axis, the first detection target includes the speed difference of the target object along the same coordinate axis at adjacent moments being less than or equal to a preset speed threshold, and the target state information includes the target position information, target speed information and target heading angle information of the target object.

12. An electronic device, characterized in that: The invention comprises a memory and a processor, wherein the memory is used to store computer instructions, and the processor is used to call the computer instructions from the memory to execute: the method according to any one of claims 1 to 11.

13. A vehicle, characterized in that: The invention comprises a memory and a processor, wherein the memory is used to store computer instructions, and the processor is used to call the computer instructions from the memory to execute: the method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the computer program implements: the method according to any one of claims 1 to 11.

15. A chip, characterized in that: The method comprises a processing unit configured to execute the method according to any one of claims 1 to 11.

Images