Multi-sensor based fusion target generation method, system, device and terminal

Abstract

The application relates to the technical field of vehicle sensors and discloses a multi-sensor-based fusion target generation method, system, device and terminal, wherein if the number of original sensors is multiple, data matching is performed, target sensors are divided into reference sensors and verification sensors, target coordinate data corresponding to the reference sensors are taken as reference coordinate data, the target coordinate data corresponding to the verification sensors are subjected to coordinate matching, a coordinate matching result is obtained, and if the number of the verification sensors is multiple, data matching is also performed on the verification sensors until the number of the verification sensors obtained through the data matching is one; if the reference coordinate data matching is successful, it is indicated that more than two sensors detect the reference coordinate data, error coordinate data detected by a single sensor is avoided from being added to the fusion target, and therefore the accuracy of the fusion target is improved.

Description

Technical Field

[0001] This invention relates to the field of vehicle sensor technology, and in particular to a method, system, device and terminal for generating targets based on multi-sensor fusion. Background Technology

[0002] In recent years, autonomous driving research has not only been a hot topic in the scientific community, but also a key focus for the technological implementation of Level 3 autonomous driving projects. Among them, multi-sensor target fusion can utilize the perceptual advantages of each sensor to complement each other and achieve comprehensive and accurate perception of environmental targets. As a result, perception of environmental targets based on multi-sensor fusion has almost become an industry standard solution.

[0003] However, when the target moves or multiple targets overlap, due to differences in detection accuracy between sensors, multiple sensors detecting the same target may lead to errors or even missed detections in the position information output by some sensors. In such cases, synchronously fusing the position information output by multiple sensors may result in a fused target that does not meet the requirements of autonomous driving. Summary of the Invention

[0004] To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general commentary, nor is it intended to identify key / important components or describe the scope of protection of these embodiments, but rather as a prelude to the detailed description that follows.

[0005] In view of the shortcomings of the prior art described above, the present invention discloses a multi-sensor-based fusion target generation method, system, device and terminal to improve the accuracy of fusion targets.

[0006] This invention provides a multi-sensor-based fusion target generation method, comprising: acquiring a set of coordinate data corresponding to original sensors, wherein the set of coordinate data is used to store target coordinate data detected by the original sensors; if there are multiple original sensors, each original sensor is used as a target sensor for data matching, wherein the data matching includes dividing the target sensors into reference sensors and verification sensors, and using the target coordinate data corresponding to the reference sensors as reference coordinate data to perform coordinate matching on the target coordinate data corresponding to the verification sensors, thereby obtaining a coordinate matching result, wherein the coordinate matching result includes successful matching or failed matching; if there are multiple verification sensors obtained through data matching, the successfully matched target coordinate data is deleted from the target coordinate data corresponding to each verification sensor, and each verification sensor is used as a target sensor for data matching; if there is only one verification sensor obtained through data matching, a fusion target is generated based on the successfully matched reference coordinate data.

[0007] This invention provides a multi-sensor-based fusion target generation system, comprising: an acquisition module for acquiring a set of coordinate data corresponding to original sensors, wherein the set of coordinate data is used to store target coordinate data detected by the original sensors; a first matching module for performing data matching on each of the original sensors as a target sensor if there are multiple original sensors, wherein the data matching includes dividing the target sensors into reference sensors and verification sensors, and performing coordinate matching on the target coordinate data corresponding to the verification sensors using the target coordinate data corresponding to the reference sensors as reference coordinate data to obtain a coordinate matching result, wherein the coordinate matching result includes successful matching or failed matching; a second matching module for deleting successfully matched target coordinate data from the target coordinate data corresponding to each verification sensor if there are multiple verification sensors obtained through data matching, and performing data matching on each verification sensor as a target sensor; and a generation module for generating a fusion target based on the successfully matched reference coordinate data if there is only one verification sensor obtained through data matching.

[0008] The present invention provides an electronic device, comprising: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to cause the electronic device to perform the above-described method.

[0009] The present invention provides a vehicle terminal, comprising: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to enable the vehicle terminal to perform the above-described method.

[0010] The beneficial effects of this invention are:

[0011] If there are multiple original sensors, data matching is performed, dividing the target sensor into a reference sensor and a calibration sensor. The target coordinate data corresponding to the reference sensor is used as the reference coordinate data to match the target coordinate data corresponding to the calibration sensor, obtaining the coordinate matching result. Furthermore, if there are multiple calibration sensors, data matching is also performed on the calibration sensors until only one calibration sensor is identified through data matching. The fused target is then generated based on the successfully matched reference coordinate data. In this way, when multiple sensors are detecting the target, data matching of the original sensors is performed cyclically. If the reference coordinate data matches successfully, it means that two or more sensors have detected that reference coordinate data. The successfully matched reference coordinate data is used to generate the fused target, avoiding the inclusion of erroneous coordinate data detected by a single sensor in the fused target, thereby improving the accuracy of the fused target. Attached Figure Description

[0012] Figure 1 This is a flowchart illustrating a multi-sensor-based target generation method in an embodiment of the present invention.

[0013] Figure 2 This is a flowchart illustrating another multi-sensor-based fusion target generation method in an embodiment of the present invention;

[0014] Figure 3 This is a schematic diagram of the structure of a multi-sensor fusion target generation system in an embodiment of the present invention;

[0015] Figure 4 This is a schematic diagram of the structure of a computer system of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0016] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and sub-samples in the embodiments can be combined with each other.

[0017] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0018] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0019] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0020] Unless otherwise stated, the term "multiple" means two or more.

[0021] In this embodiment of the disclosure, the character " / " indicates that the objects before and after it are in an "or" relationship. For example, A / B means: A or B.

[0022] The term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0023] Combination Figure 1 As shown, this disclosure provides a multi-sensor fusion target generation method, including:

[0024] Step S101: Obtain the set of coordinate data corresponding to the original sensor;

[0025] The coordinate data set is used to store the target coordinate data detected by the original sensor;

[0026] Step S102: If there are multiple original sensors, then each original sensor is used as a target sensor for data matching.

[0027] The data matching process includes dividing the target sensors into roles to obtain a reference sensor and a calibration sensor, and using the target coordinate data corresponding to the reference sensor as the reference coordinate data to perform coordinate matching with the target coordinate data corresponding to the calibration sensor to obtain the coordinate matching result.

[0028] The coordinate matching result includes whether the match was successful or failed.

[0029] Step S103: If the number of verification sensors obtained through data matching is multiple, then delete the successfully matched target coordinate data from the target coordinate data corresponding to each verification sensor, and use each verification sensor as a target sensor for data matching.

[0030] Step S104: If the number of verification sensors obtained through data matching is one, then generate a fusion target based on the successfully matched reference coordinate data.

[0031] The multi-sensor fusion target generation method provided in this disclosure involves several steps. If there are multiple original sensors, data matching is performed. The target sensors are divided into reference sensors and verification sensors. The target coordinate data corresponding to the reference sensors is used as the reference coordinate data to match the target coordinate data corresponding to the verification sensors, resulting in a coordinate matching result. If there are multiple verification sensors, data matching is also performed on the verification sensors until only one verification sensor is detected. The fused target is then generated based on the successfully matched reference coordinate data. In this way, when multiple sensors detect the target, data matching is performed on the original sensors cyclically. If the reference coordinate data matches successfully, it indicates that two or more sensors have detected the reference coordinate data. The successfully matched reference coordinate data is used to generate the fused target, avoiding the inclusion of erroneous coordinate data detected by a single sensor in the fused target, thereby improving the accuracy of the fused target.

[0032] Optionally, obtaining the target coordinate data detected by the original sensors includes: establishing a vehicle coordinate system corresponding to the target vehicle, wherein the target vehicle is equipped with multiple vehicle sensors; determining the target detection area of ​​the target vehicle in the vehicle coordinate system, and determining the original sensor corresponding to the target detection area from each vehicle sensor; and performing target detection on the target detection area through the original sensor to obtain the target coordinate data.

[0033] In some embodiments, a Cartesian coordinate system is established with the center of the rear axle of the target vehicle as the origin, to form the vehicle coordinate system corresponding to the target vehicle, wherein the left side of the vehicle is the positive X-axis and the forward direction of the vehicle is the positive Y-axis.

[0034] Optionally, the vehicle sensors include at least one of the following: a front camera (FC) for detecting target coordinate data located in front of the target vehicle; an around camera (AC) for detecting target coordinate data within a surrounding viewing area, wherein the surrounding viewing area is a region located at a preset surrounding viewing distance from the outer contour of the target vehicle; a front radar (FR) for detecting target coordinate data located in front of the target vehicle; a front corner radar (FCR) for detecting target coordinate data located to the left front and right front of the target vehicle; and a rear corner radar (RCR) for detecting target coordinate data located to the left rear and right rear of the target vehicle.

[0035] In some embodiments, obstacle targets are detected by vehicle sensors; the detected target coordinate data is received and synchronized with the controller in time; the time-synchronized target coordinate data is placed into the corresponding coordinate data set, wherein the coordinate data set is established in the form of an array, and the coordinate data set includes FC_Input[], AC_Input[], FR_Input[], FCR_Input[], and RCR_Input[].

[0036] Optionally, the target detection area includes at least one of the following: a surround-view detection area, comprising a surround-view region, wherein the original sensor corresponding to the surround-view detection area includes at least one of a forward image sensor, a surround-view image sensor, a forward radar sensor, a front corner radar sensor, and a rear corner radar sensor; a forward-facing detection area, wherein the original sensor corresponding to the forward-facing detection area includes at least one of a forward image sensor and a forward radar sensor, wherein there is no overlap between the forward-facing detection area and the surround-view detection area; a front corner detection area, wherein the original sensor corresponding to the front corner detection area includes a forward radar sensor, wherein there is no overlap between the front corner detection area and the surround-view detection area; and a rear corner detection area, wherein the original sensor corresponding to the rear corner detection area includes a rear corner radar sensor, wherein there is no overlap between the rear corner detection area and the surround-view detection area.

[0037] In some embodiments, the area within 20 meters of the target vehicle is designated as the surround-view detection area.

[0038] In some embodiments, the area that can be detected by the forward image sensor is obtained, and the overlapping area between the forward detection area and the surrounding detection area is subtracted to obtain the forward detection area.

[0039] In some embodiments, the area that can be detected by the forward radar sensor is obtained, and the overlapping area between the forward radar sensor and the surrounding detection area and the overlapping area between the forward radar sensor and the surrounding detection area are subtracted to obtain the forward radar detection area.

[0040] In some embodiments, the area that can be detected by the front corner radar sensor is obtained, and the overlapping area between the front corner radar sensor and the surrounding detection area and the overlapping area between the front corner radar sensor and the forward detection area are subtracted to obtain the front corner detection area.

[0041] In some embodiments, the area that can be detected by the rear corner radar sensor is obtained, and the overlapping area between it and the surrounding detection area is subtracted to obtain the rear corner detection area.

[0042] Optionally, if there are multiple original sensors, each original sensor is used as a target sensor for data matching, including: pre-setting the sensor priority corresponding to the original sensor; if there are multiple original sensors, counting the target coordinate data corresponding to each original sensor to obtain the number of detected targets corresponding to each original sensor; if the number of any detected target is greater than or equal to two, each original sensor is used as a target sensor for data matching; if the number of each detected target is less than two, a priority sensor is determined from the original sensors that have detected target coordinate data according to the sensor priority, and a fused target is generated based on the target coordinate data detected by the priority sensor.

[0043] In some embodiments, sensor priorities are set for each original sensor according to the detection characteristics of the original sensors. The original sensors are ordered from high to low according to their priority, and are in the following order: surround view image sensor, forward image sensor, forward radar sensor, rear corner radar sensor, and front corner radar sensor.

[0044] In some embodiments, if the number of each detected target is less than two, it indicates that at least one original sensor has detected an obstacle target. The original sensors are sequentially determined to detect obstacle targets in descending order of sensor priority, so as to determine the priority sensor with the highest sensor priority from the original sensors that have detected target coordinate data, and generate a fused target based on the target coordinate data detected by the priority sensor.

[0045] Optionally, the target sensors are assigned responsibilities to obtain a reference sensor and a calibration sensor, including: identifying the target sensor with the highest sensor priority as the reference sensor, and identifying the target sensors other than the reference sensor as calibration sensors.

[0046] Optionally, the target coordinate data corresponding to the reference sensor is used as the reference coordinate data to perform coordinate matching with the target coordinate data corresponding to the verification sensor to obtain a coordinate matching result. This includes: calculating the reference coordinate data according to a preset threshold interval to obtain a reference coordinate threshold, and comparing the target coordinate data according to the reference coordinate threshold; if the target coordinate data corresponding to the verification sensor is within the reference coordinate threshold, the coordinate matching result is determined to be a successful match; if the target coordinate data corresponding to the verification sensor is outside the reference coordinate threshold, the coordinate matching result is determined to be a failed match.

[0047] In some embodiments, the preset threshold interval is [0.8x, 1.2x][0.8y, 1.2y], where x is the abscissa of the reference coordinate data and y is the ordinate of the reference coordinate data.

[0048] Optionally, the method further includes: if the number of original sensors is one, generating a fused target based on the target coordinate data detected by the original sensors.

[0049] In some embodiments, the original number of sensors in the front radar detection area is forward radar sensors, and the fused target in the front radar detection area is generated based on the target coordinate data detected by the forward radar sensors.

[0050] In some embodiments, the original number of sensors in the front corner detection region is a front corner radar sensor, and a fused target in the front corner detection region is generated based on the target coordinate data detected by the forward radar sensor.

[0051] In some embodiments, the original number of sensors in the rear corner detection region is a rear corner radar sensor, and a fused target in the rear corner detection region is generated based on the target coordinate data detected by the rear radar sensor.

[0052] Optionally, if the number of verification sensors obtained through data matching is one, a fusion target is generated based on the successfully matched reference coordinate data, including: if the number of verification sensors obtained through data matching is one, and the number of original sensors is equal to two, then the unverified coordinates that failed to match are extracted from the target coordinate data corresponding to the verification sensors, and a fusion target is generated based on the successfully matched reference coordinate data and the unverified coordinates; if the number of verification sensors obtained through data matching is one, and the number of original sensors is greater than two, then a fusion target is generated based on the successfully matched reference coordinate data.

[0053] In some embodiments, the original sensors corresponding to the surround-view detection area include a forward image sensor, a surround-view image sensor, a forward radar sensor, a front-angle radar sensor, and a rear-angle radar sensor. The reference coordinate data detected by the forward image sensor includes data A and data B; the reference coordinate data detected by the surround-view image sensor includes data C and data D; and the reference coordinate data detected by the forward radar sensor includes data E, data F, and data G. Using the forward image sensor as the reference sensor and data A and data B as reference coordinate data, the reference coordinate data are compared with data C, data D, data E, data F, and data G according to the threshold intervals [0.8x, 1.2x] and [0.8y, 1.2y], respectively. This yields data C and data E located within the reference coordinate threshold of data A, and data F located within the threshold of data B. Within the reference coordinate threshold, the coordinate matching results of data A, B, C, E, and F are recorded as True, while the coordinate matching results of data D and G are recorded as False. Successfully matched target coordinate data are deleted from the target coordinate data corresponding to each verification sensor. The reference coordinate data for the surround-view imaging sensor includes data D, and the reference coordinate data for the forward-facing radar sensor includes data G. Using the surround-view imaging sensor as the reference sensor and data D as the reference coordinate data, data G is compared according to the threshold interval [0.8x, 1.2x][0.8y, 1.2y], indicating that data G is within the reference coordinate threshold of data D. Successfully matched reference coordinate data includes data A, B, and D. Synchronous fusion is performed based on data A, B, and D to generate a fused target.

[0054] In some embodiments, the original sensor corresponding to the forward detection area includes a forward image sensor and a forward radar sensor; the forward image sensor is used as a reference sensor, and the target coordinate data of the forward image sensor is used as the reference coordinate data; it is determined whether the target coordinate data of the forward radar sensor is within the reference coordinate threshold corresponding to the reference coordinate data, the coordinate matching result of the target coordinate data within the reference coordinate threshold is recorded as True, and the coordinate matching result of the target coordinate data outside the reference coordinate threshold is recorded as False; synchronous fusion is performed based on the successfully matched reference coordinate data and the unmatched target coordinate data in the forward radar sensor to generate a fused target.

[0055] Combination Figure 2 As shown, this disclosure provides a multi-sensor fusion target generation method, including:

[0056] Step S201: Obtain the set of coordinate data corresponding to the original sensor;

[0057] The coordinate data set is used to store the target coordinate data detected by the original sensor;

[0058] The original sensor includes at least one of a forward image sensor, a surround image sensor, a forward radar sensor, a front corner radar sensor, and a rear corner radar sensor.

[0059] Among them, the sensor priority corresponding to the original sensor is set;

[0060] Step S202: Determine whether there are multiple original sensors. If yes, proceed to step S203; otherwise, proceed to step S213.

[0061] Step S203: Determine whether the number of each detected target is less than two. If yes, proceed to step S214; otherwise, proceed to step S204.

[0062] Step S204: Use each original sensor as the target sensor;

[0063] Step S205: Assign responsibilities to the target sensor to obtain the reference sensor and the calibration sensor;

[0064] Among them, the target sensor with the highest sensor priority is determined as the reference sensor, and the target sensors other than the reference sensor are determined as calibration sensors;

[0065] Step S206: Use the target coordinate data corresponding to the reference sensor as the reference coordinate data to perform coordinate matching with the target coordinate data corresponding to the verification sensor to obtain the coordinate matching result;

[0066] Specifically, the reference coordinate data is calculated based on a preset threshold interval to obtain the reference coordinate threshold, and the target coordinate data is compared based on the reference coordinate threshold.

[0067] If the target coordinate data corresponding to the verification sensor is within the reference coordinate threshold, the coordinate matching result is determined to be a successful match.

[0068] If the target coordinate data corresponding to the verification sensor is outside the reference coordinate threshold, the coordinate matching result will be determined as a matching failure.

[0069] Step S207: Determine whether there are multiple verification sensors. If yes, proceed to step S208; otherwise, proceed to step S210.

[0070] Step S208: Delete the successfully matched target coordinate data from the target coordinate data corresponding to each verification sensor;

[0071] Step S209: Select each calibration sensor as the target sensor and proceed to step S205;

[0072] Step S210: Determine if the number of original sensors is equal to two. If yes, proceed to step S211; otherwise, proceed to step S212.

[0073] Step S211: Extract the unverified coordinates that failed to match from the target coordinate data corresponding to the verification sensor, and generate the fused target based on the successfully matched reference coordinate data and the unverified coordinates.

[0074] Step S212: Generate the fusion target based on the successfully matched reference coordinate data.

[0075] Step S213: Generate a fused target based on the target coordinate data detected by the original sensor.

[0076] Step S214: Determine a priority sensor from the original sensors that detected the target coordinate data according to the sensor priority;

[0077] Step S215: Generate a fused target based on the target coordinate data detected by the priority sensor.

[0078] The multi-sensor fusion target generation method provided in this disclosure involves several steps. If there are multiple original sensors, data matching is performed. The target sensors are divided into reference sensors and verification sensors. The target coordinate data corresponding to the reference sensors is used as the reference coordinate data to match the target coordinate data corresponding to the verification sensors, resulting in a coordinate matching result. If there are multiple verification sensors, data matching is also performed on the verification sensors until only one verification sensor is detected. The fused target is then generated based on the successfully matched reference coordinate data. In this way, when multiple sensors detect the target, data matching is performed on the original sensors cyclically. If the reference coordinate data matches successfully, it indicates that two or more sensors have detected the reference coordinate data. The successfully matched reference coordinate data is used to generate the fused target, avoiding the inclusion of erroneous coordinate data detected by a single sensor in the fused target, thereby improving the accuracy of the fused target.

[0079] Combination Figure 3 As shown, this disclosure provides a multi-sensor-based fusion target generation system, including an acquisition module 301, a first matching module 302, a second matching module 303, and a generation module 304.

[0080] The acquisition module 301 is used to acquire the coordinate data set corresponding to the original sensor, wherein the coordinate data set is used to store the target coordinate data detected by the original sensor.

[0081] The first matching module 302 is used to perform data matching on each original sensor as a target sensor if there are multiple original sensors. The data matching includes dividing the target sensors into roles to obtain a reference sensor and a calibration sensor, and using the target coordinate data corresponding to the reference sensor as the reference coordinate data to perform coordinate matching on the target coordinate data corresponding to the calibration sensor to obtain a coordinate matching result, which includes whether the matching is successful or unsuccessful.

[0082] The second matching module 303 is used to delete the successfully matched target coordinate data from the target coordinate data corresponding to each verification sensor if the number of verification sensors obtained through data matching is multiple, and to use each verification sensor as a target sensor for data matching.

[0083] The generation module 304 is used to generate a fusion target based on the successfully matched reference coordinate data if the number of verification sensors obtained through data matching is one.

[0084] The multi-sensor fusion target generation system provided in this disclosure involves several steps. If there are multiple original sensors, data matching is performed. The target sensors are divided into reference sensors and verification sensors. The target coordinate data corresponding to the reference sensors is used as the reference coordinate data to match the target coordinate data corresponding to the verification sensors, resulting in a coordinate matching result. If there are multiple verification sensors, data matching is also performed on the verification sensors until only one verification sensor is identified. The fused target is then generated based on the successfully matched reference coordinate data. In this way, when multiple sensors detect the target, data matching is performed on the original sensors cyclically. If the reference coordinate data matches successfully, it indicates that two or more sensors have detected the reference coordinate data. The successfully matched reference coordinate data is used to generate the fused target, avoiding the inclusion of erroneous coordinate data detected by a single sensor in the fused target, thereby improving the accuracy of the fused target.

[0085] This disclosure also provides an electronic device, including: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to cause the device to perform the above-described method.

[0086] like Figure 4 As shown, the device also includes a schematic diagram of a computer system suitable for implementing the embodiments of this application. It should be noted that... Figure 4 The computer system 400 of the device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0087] like Figure 4As shown, the computer system 400 includes a Central Processing Unit (CPU) 401, which can perform various appropriate actions and processes, such as executing the methods described in the above embodiments, based on programs stored in Read-Only Memory (ROM) 402 or programs loaded from Storage Unit 408 into Random Access Memory (RAM) 403. The RAM 403 also stores various programs and data required for system operation. The CPU 401, ROM 402, and RAM 403 are interconnected via a bus 404. An Input / Output (I / O) interface 405 is also connected to the bus 404.

[0088] The following components are connected to I / O interface 405: an input section 406 including a keyboard, mouse, etc.; an output section 407 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 409 performs communication processing via a network such as the Internet. Drive 140 is also connected to I / O interface 405 as needed. Removable media 411, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 140 as needed so that computer programs read from it can be installed into storage section 408 as needed.

[0089] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 409, and / or installed from removable medium 411. When the computer program is executed by central processing unit (CPU) 401, it performs various functions defined in the system of this application.

[0090] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0091] The electronic device disclosed in this embodiment includes a processor, a memory, a transceiver, and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication between them. The memory is used to store computer programs, the communication interface is used to perform communication, and the processor and the transceiver are used to run the computer programs, so that the electronic device performs the various steps of the above method.

[0092] In this embodiment, the memory may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.

[0093] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), graphics processing units (GPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0094] This disclosure also provides a vehicle terminal, including: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the vehicle terminal performs the above-described method.

[0095] The foregoing description and accompanying drawings fully illustrate embodiments of this disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, procedural, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and subsamples of some embodiments may be included in or replace parts and subsamples of other embodiments. Moreover, the terminology used in this application is for describing embodiments only and is not intended to limit the claims. As used in the description of embodiments and claims, the singular forms “a,” “an,” and “the” are intended to equally include the plural forms unless the context clearly indicates otherwise. Similarly, the term “and / or” as used herein means including one or more of the associated listed items and all possible combinations thereof. Additionally, when used in this application, the term "comprise" and its variations "comprises" and / or "comprising" refer to the presence of stated subsamples, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other subsamples, wholes, steps, operations, elements, components, and / or groups thereof. Without further limitations, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, or apparatus that includes the element. In this document, each embodiment may focus on the differences from other embodiments, and similar or identical parts between embodiments can be referred to mutually. For methods, products, etc., disclosed in the embodiments, if they correspond to the method section disclosed in the embodiments, the relevant parts can be referred to the description of the method section.

[0096] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this disclosure. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0097] The methods and products (including but not limited to devices and equipment) disclosed in the embodiments herein can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units may be merely a logical division of responsibilities, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some sub-samples may be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed units may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms. Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to implement this embodiment according to actual needs. Furthermore, the functional units in the embodiments of this disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0098] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

Claims

1. A multi-sensor fusion target generation method, characterized in that, include: Obtain the coordinate data set corresponding to the original sensor, wherein the coordinate data set is used to store the target coordinate data detected by the original sensor; If there are multiple original sensors, each original sensor is used as a target sensor for data matching. The data matching includes dividing the target sensors into roles to obtain a reference sensor and a calibration sensor. The target coordinate data corresponding to the reference sensor is used as the reference coordinate data to perform coordinate matching on the target coordinate data corresponding to the calibration sensor to obtain a coordinate matching result. The coordinate matching result includes successful matching or failed matching. If the number of verification sensors obtained through data matching is multiple, then the successfully matched target coordinate data is deleted from the target coordinate data corresponding to each verification sensor, and each verification sensor is used as a target sensor for data matching. If the number of verification sensors obtained through data matching is one, then a fusion target is generated based on the successfully matched reference coordinate data.

2. The method according to claim 1, characterized in that, If there are multiple original sensors, then each original sensor is used as a target sensor for data matching, including: Pre-set the sensor priority corresponding to the original sensor; If there are multiple original sensors, the number of target coordinate data corresponding to each original sensor is counted to obtain the number of detected targets corresponding to each original sensor. If the number of any detection target is greater than or equal to two, then each of the original sensors will be used as the target sensor for data matching. If the number of each of the aforementioned detection targets is less than two, then a priority sensor is determined from the original sensors that detected the target coordinate data according to the sensor priority, and a fused target is generated based on the target coordinate data detected by the priority sensor.

3. The method according to claim 2, characterized in that, The responsibilities of the target sensors are divided to obtain the reference sensor and the calibration sensor, including: The target sensor with the highest sensor priority is identified as the reference sensor, and the target sensors other than the reference sensor are identified as calibration sensors.

4. The method according to claim 1, characterized in that, The target coordinate data corresponding to the reference sensor is used as the reference coordinate data to perform coordinate matching with the target coordinate data corresponding to the verification sensor, and the coordinate matching result is obtained, including: The baseline coordinate data is calculated based on the preset threshold interval to obtain the baseline coordinate threshold, and the target coordinate data is compared based on the baseline coordinate threshold. If the target coordinate data corresponding to the verification sensor is within the reference coordinate threshold, the coordinate matching result is determined to be a successful match. If the target coordinate data corresponding to the verification sensor is outside the reference coordinate threshold, the coordinate matching result is determined to be a matching failure.

5. The method according to claim 1, characterized in that, The method further includes: If the number of original sensors is one, a fused target is generated based on the target coordinate data detected by the original sensors.

6. The method according to claim 1, characterized in that, If the number of verification sensors obtained through data matching is one, then a fusion target is generated based on the successfully matched reference coordinate data, including: If the number of verification sensors obtained through data matching is one, and the number of original sensors is equal to two, then the unverified coordinates that failed to match are extracted from the target coordinate data corresponding to the verification sensor, and a fusion target is generated based on the successfully matched reference coordinate data and the unverified coordinates. If the number of verification sensors obtained through data matching is one, and the number of original sensors is greater than two, then a fusion target is generated based on the successfully matched reference coordinate data.

7. The method according to any one of claims 1 to 6, characterized in that, Obtain the raw target coordinate data detected by the sensor, including: Establish a vehicle coordinate system corresponding to the target vehicle, wherein the target vehicle is equipped with multiple vehicle sensors; The target detection area of ​​the target vehicle is determined in the vehicle coordinate system, and the original sensor corresponding to the target detection area is determined from each of the vehicle sensors; The target detection area is detected by the original sensor to obtain target coordinate data.

8. The method according to claim 7, characterized in that, The vehicle sensor includes at least one of the following: A forward-facing image sensor is used to detect the coordinate data of a target located in front of the target vehicle; A surround-view image sensor is used to detect target coordinate data within a surround-view area, wherein the surround-view area is a region that is a preset surround-view distance away from the outer contour of the target vehicle; A forward-facing radar sensor is used to detect the coordinate data of a target located in front of the target vehicle; A front-angle radar sensor is used to detect the target coordinates located at the left and right front of the target vehicle; The rear corner radar sensor is used to detect the target coordinates located to the left and right rear of the target vehicle.

9. The method according to claim 8, characterized in that, The target detection region includes at least one of the following: The surround view detection area includes the surround view area, and the original sensor corresponding to the surround view detection area includes at least one of the forward image sensor, the surround view image sensor, the forward radar sensor, the front corner radar sensor, and the rear corner radar sensor; The forward detection region includes at least one of the forward image sensor and the forward radar sensor, wherein there is no overlap between the forward detection region and the surround view detection region. The front radar detection area, the original sensor corresponding to the front radar detection area includes the forward radar sensor, wherein there is no overlap between the front radar detection area, the forward detection area and the surround view detection area; A front corner detection area, wherein the original sensor corresponding to the front corner detection area includes the front corner radar sensor, and there is no overlapping area between the front corner detection area and the surround view detection area; The rear corner detection area, the original sensor corresponding to the rear corner detection area includes the rear corner radar sensor, wherein there is no overlapping area between the rear corner detection area and the surround view detection area.

10. A multi-sensor fusion target generation system, characterized in that, include: The acquisition module is used to acquire a set of coordinate data corresponding to the original sensor, wherein the set of coordinate data is used to store the target coordinate data detected by the original sensor; The first matching module is used to perform data matching on each of the original sensors as a target sensor if there are multiple original sensors. The data matching includes dividing the target sensors into roles to obtain a reference sensor and a calibration sensor, and using the target coordinate data corresponding to the reference sensor as the reference coordinate data to perform coordinate matching on the target coordinate data corresponding to the calibration sensor to obtain a coordinate matching result. The coordinate matching result includes matching success or matching failure. The second matching module is used to delete the successfully matched target coordinate data from the target coordinate data corresponding to each of the verification sensors if the number of verification sensors obtained through data matching is multiple, and to use each of the verification sensors as target sensors for data matching. The generation module is used to generate a fusion target based on the successfully matched reference coordinate data if the number of verification sensors obtained through data matching is only one.

11. An electronic device, characterized in that, include: Processor and memory; The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to cause the electronic device to perform the method as described in any one of claims 1 to 9.

12. A vehicle terminal, characterized in that, include: Processor and memory; The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to cause the vehicle terminal to perform the method as described in any one of claims 1 to 9.

Images