A data fusion method, device and computer equipment

Abstract

Embodiments of the present application provide a data fusion method, device and computer equipment. The method comprises: acquiring sensing data collected by a target sensing device and other sensing devices respectively; wherein the sensing data at least comprises a first position of a detection object; converting the first position of the detection object detected by the other sensing devices based on a position conversion relationship between the target sensing device and the other sensing devices, to obtain a second position of the detection object detected by the other sensing devices in the target sensing device; determining the detection object based on a judgment result of whether the second position and the first position of the detection object detected by the target sensing device are in a preset region; wherein the detection object is an object whose sensing data of the other sensing devices and sensing data of the target sensing device are the same.

Description

Technical Field

[0001] This application relates to, but is not limited to, the field of computer technology, and in particular to a data fusion method, apparatus, and computer equipment. Background Technology

[0002] When multiple sensors are used to collect data simultaneously, such as in vehicle-to-infrastructure (V2I) scenarios, each sensor possesses its own relatively accurate attribute data. For instance, radar provides a more accurate description of speed and location information, while cameras offer more accurate descriptions of visual attributes such as license plates, vehicle colors, and vehicle types. To achieve more comprehensive information on traffic participants, the fusion of traffic participant information collected by different sensors presents a crucial technical challenge. Summary of the Invention

[0003] In view of this, embodiments of this application provide a data fusion method, apparatus, and computer device.

[0004] The technical solution of this application is implemented as follows:

[0005] In a first aspect, embodiments of this application provide a data fusion method, the method comprising: acquiring sensing data collected by a target sensing device and other sensing devices respectively; wherein the sensing data includes at least a first position of a detected object; based on the position transformation relationship between the target sensing device and the other sensing devices, transforming the first position of the detected object detected by the other sensing devices to obtain a second position of the detected object detected by the other sensing devices in the target sensing device; determining a detected object based on a judgment result of whether the second position and the first position of the detected object detected by the target sensing device are within a preset area; wherein the detected object is an object whose sensing data from the other sensing devices and the sensing data from the target device are the same.

[0006] Secondly, embodiments of this application provide a data fusion apparatus, the apparatus comprising:

[0007] The first acquisition module is used to acquire sensing data collected by the target sensing device and other sensing devices respectively; wherein, the sensing data includes at least the first position of the detected object;

[0008] The conversion module is used to convert the first position of the detected object detected by the other sensing devices based on the position conversion relationship between the target sensing device and the other sensing devices, so as to obtain the second position of the detected object detected by the other sensing devices in the target sensing device.

[0009] The first determining module is used to determine the detection object based on the judgment result of whether the second position and the first position of the detection object detected by the target sensing device are in a preset area; wherein, the detection object is an object whose sensing data of the other sensing devices and the sensing data of the target sensing device are the same.

[0010] Thirdly, embodiments of this application provide a computer device, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the program to implement some or all of the steps in the above-described method.

[0011] In this embodiment, the positional transformation relationship between the target sensing device and other sensing devices is utilized to transform the first position of the detected object in the sensing data collected by other sensing devices to the target sensing device, thereby obtaining the second position of the detected object detected by other sensing devices in the target sensing device. Then, based on the judgment result of whether the second position and the first position of the detected object detected by the target sensing device are within a preset area, it is determined that the sensing data of other sensing devices and the sensing data of the target sensing device belong to the same detected object, thereby realizing the data fusion of the sensing data of the target sensing device and the sensing data of other sensing devices.

[0012] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this application. Attached Figure Description

[0013] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0014] Figure 1 A schematic diagram illustrating the implementation process of a data fusion method provided in an embodiment of this application;

[0015] Figure 2 A schematic diagram of the implementation process of step S103 provided in this application embodiment. Figure 1 ;

[0016] Figure 3 A schematic diagram of the implementation process of step S201 provided in an embodiment of this application. Figure 2 ;

[0017] Figure 4 A schematic diagram of another implementation process of step S201 provided in the embodiments of this application Figure 3 ;

[0018] Figure 5 A schematic diagram of another implementation process of step S103 provided in the embodiments of this application Figure 4;

[0019] Figure 6 A schematic diagram illustrating the implementation process of another data fusion method provided in this application embodiment;

[0020] Figure 7(a) illustrates the verification effect of a data fusion method provided in an embodiment of this application. Figure 1 ;

[0021] Figure 7(b) illustrates the verification effect of a data fusion method provided in an embodiment of this application. Figure 2 ;

[0022] Figure 8 This is a schematic diagram of the composition structure of a data fusion device provided in an embodiment of this application;

[0023] Figure 9 This is a schematic diagram of the hardware entity of a computer device provided in an embodiment of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application are further described in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0026] The terms “first / second / third” are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that “first / second / third” may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0027] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terminology used herein is for descriptive purposes only and is not intended to limit the scope of this application.

[0028] This application provides a data fusion method, referencing... Figure 1 The method may include the following steps S101 to S103, wherein:

[0029] Step S101: Acquire sensing data collected by the target sensing device and other sensing devices respectively; wherein, the sensing data includes at least the first position of the detected object;

[0030] Here, the target sensing device can be an image sensor, such as a camera; other sensing devices can be signal receivers, such as radar; the detection object can be a fixed object or a moving object. For example, a fixed object can be a fixed tree, building, parked vehicle, etc., and a moving object can be a moving vehicle, a walking person, etc. That is to say, the detection object in this application embodiment can be a vehicle in traffic, a non-motorized vehicle, a pedestrian, a driver or passenger in a vehicle, etc. This application embodiment does not limit the type of detection object.

[0031] In practice, if the object of detection is a vehicle moving in traffic, the sensing data collected by the target sensing device may include the vehicle's position, vehicle type, and vehicle color. Other sensing devices may also collect data such as the vehicle's position, vehicle type, and vehicle color.

[0032] Step S102: Based on the position conversion relationship between the target sensing device and the other sensing devices, the first position of the detected object detected by the other sensing devices is converted to obtain the second position of the detected object in the target sensing device.

[0033] Here, the position transformation relationship refers to the transformation of the first position of the detected object in the sensing data of other sensing devices to that of the target sensing device. Therefore, the position transformation relationship between the target sensing device and other sensing devices can be realized through a transformation matrix.

[0034] In some embodiments, the first position of the detected object detected by the other sensing device is characterized as the centroid of the detected object detected by the other sensing device. The implementation of step S102, "based on the position transformation relationship between the target sensing device and the other sensing device, transforming the first position of the detected object detected by the other sensing device to obtain the second position of the detected object detected by the other sensing device in the target sensing device", may include: determining the second position of the detected object detected by the other sensing device in the target sensing device based on the spatial coordinates of the centroid and the transformation matrix.

[0035] In practice, given the known first position (i.e., centroid coordinates) of the detected object in the sensing data of other sensing devices, a transformation matrix can be used to transform the position (i.e., the first position) of the detected object in the sensing data of other sensing devices to the target sensing device, thereby determining the second position of the detected object detected by other sensing devices in the target sensing device.

[0036] Step S103: Based on the judgment result of whether the second position and the first position of the detected object detected by the target sensing device are in a preset area, the detection object is determined; wherein, the detection object is an object whose sensing data of the other sensing devices and the sensing data of the target sensing device are the same.

[0037] Here, the preset area refers to the area where the detected object is located. In practice, the preset area can include the area of ​​the first position. That is, the preset area is determined based on the first position. In some special cases, if the first position is represented by a detection box, the preset area can also be the detection box representing the first position. The size of the preset area is not limited in the embodiments of this application.

[0038] During implementation, if the detected object in the sensing data of other sensing devices is at the second position in the target sensing device and / or the first position in the sensing data of the target sensing device is not within the preset area, it cannot be determined that the detected object in the sensing data of other sensing devices and the detected object in the sensing data of the target sensing device belong to the same detected object, and the above steps S101 and S102 need to be re-executed.

[0039] In some embodiments, if the detected object in the sensing data of other sensing devices is located within a preset area in both the second position in the sensing data of the target sensing device and the first position of the detected object detected by the target sensing device, then it is determined that the detected object in the sensing data of other sensing devices and the detected object in the sensing data of the target sensing device belong to the same detected object, so as to realize the fusion of the sensing data collected by other sensing devices and the sensing data collected by the target sensing device.

[0040] In some embodiments, reference Figure 2 The target sensing device is an image sensor, and the other sensing devices are signal receivers. The implementation of step S103, "determining the detection object based on the judgment result of whether the second position and the first position of the detection object detected by the target sensing device are within a preset area," may include step S201:

[0041] Step S201: When the detected object in the sensing data of the signal receiver is represented as a pixel region in the second position of the image sensor, the detected object is determined based on the judgment result of whether the pixel region and the first position of the detected object detected by the image sensor are in a preset area.

[0042] In practice, since the second position of the detected object in the image sensor is obtained through the transformation matrix in the sensor data of the signal receiver, the second position is the pixel area of ​​the detected object in the image sensor in the sensor data of the signal receiver. Thus, the detected object is determined by judging whether the pixel area and the first position of the detected object in the image sensor are in the preset area.

[0043] In some embodiments, the first position of the detected object by the image sensor can be a detection box. For ease of implementation, the preset area in this embodiment can be the detection box. Figure 3 The implementation of "determining the detection object based on the judgment result of whether the pixel region and the first position corresponding to the detection object detected by the image sensor are in the preset region" in step S201 may include the following steps S301 and S302, wherein:

[0044] Step S301: In the case of using a detection box to characterize the first position of the detected object detected by the image sensor, determine the positional relationship between the pixel region and the detection box;

[0045] In practice, when the image sensor acquires the detection object in the sensing data, the detection object is enclosed by a rectangular detection box. This detection box represents the position of the detection object (i.e., the first position), thereby determining the positional relationship between the pixel area of ​​the detection object in the sensing data of the signal receiver in the image sensor and the detection box corresponding to the first position of the detection object in the sensing data of the image sensor.

[0046] Step S302: If the positional relationship indicates that the pixel region is located within the detection frame, determine that the sensing data of the signal receiver and the sensing data of the image sensor belong to the same detection object.

[0047] In implementation, if the pixel region of the object to be detected in the sensor data of the signal receiver is not within the detection box corresponding to the first position of the object to be detected in the sensor data of the image sensor, then the sensor data of the signal receiver and the sensor data of the image sensor do not belong to the same object to be detected. Conversely, if the pixel region of the object to be detected in the sensor data of the signal receiver is within the detection box corresponding to the first position of the object to be detected in the sensor data of the image sensor, then the sensor data of the signal receiver and the sensor data of the image sensor belong to the same object to be detected, thereby realizing the fusion of the sensor data of the signal receiver and the sensor data of the image sensor.

[0048] In this embodiment, the positional transformation relationship between the target sensing device and other sensing devices is utilized to transform the first position of the detected object in the sensing data collected by other sensing devices to the target sensing device, thereby obtaining the second position of the detected object detected by other sensing devices in the target sensing device. Then, based on the judgment result of whether the second position and the first position of the detected object detected by the target sensing device are within a preset area, it is determined that the sensing data of other sensing devices and the sensing data of the target sensing device belong to the same detected object, thereby realizing the data fusion of the sensing data of the target sensing device and the sensing data of other sensing devices.

[0049] In some embodiments, reference Figure 4 The signal receiver detects at least two detection objects; the implementation of "determining the detection object based on the judgment result of whether the pixel region and the first position of the detection object detected by the image sensor are in a preset region" in step S201 may include the following steps S401 to S404, wherein:

[0050] Step S401: When at least two pixel regions corresponding to at least two detected objects are respectively located in the corresponding detection boxes, the current type of the detected objects detected by the image sensor is determined based on the sensing data collected by the image sensor.

[0051] Here, the detection object can be any two of the following in a vehicle-road cooperative scenario: cars, buses, non-motorized vehicles, and pedestrians. The current type of the detection object can be the car model, the car license plate number, or the car color; this embodiment does not limit the current type of the detection object.

[0052] In implementation, it is first determined whether the multiple pixel regions corresponding to the multiple detected objects detected in the signal receiver are all within the detection frame corresponding to the first position of the detected object in the sensing data of the corresponding image sensor. Then, if at least two pixel regions corresponding to the multiple detected objects are respectively located in the corresponding detection frame, the current type of the detected object detected by the image sensor is determined by the sensing data collected by the image sensor.

[0053] Step S402: Determine the critical dimensions of each detected object in the signal receiver based on the sensing data collected by the signal receiver;

[0054] Here, the key dimensions of the object being inspected refer to its length, width, and height.

[0055] Step S403: Determine a target detection object from the at least two detection objects; wherein the key dimensions of the target detection object match the current type;

[0056] In practice, if the two objects detected by the signal receiver are a car and a pedestrian, and the object detected by the image sensor is a car, then the car detected by the signal receiver is determined to be the target object. The length, width and height of the target object match the current type of the object detected by the image sensor. Thus, it is determined that the sensing data corresponding to the target object and the sensing data of the image sensor belong to the same object, thereby realizing the fusion of the sensing data corresponding to the target object and the sensing data of the image sensor.

[0057] Step S404: Determine that the sensing data corresponding to the target detection object and the sensing data of the image sensor belong to the same detection object.

[0058] In some embodiments, the method further includes the following steps S111 and S112, wherein:

[0059] Step S111: Obtain the location information of the target sensing device and the location information of the other sensing devices;

[0060] Here, the location information of the target sensing device refers to the installation location of the target sensing device; the location information of other sensing devices refers to the installation location of other sensing devices.

[0061] Step S112: Based on the location information of the target sensing device and the location information of the other sensing devices, perform joint calibration on the target sensing device and the other sensing devices to obtain the position transformation relationship between the other sensing devices and the target sensing device.

[0062] Here, given the installation locations of the target sensor and other sensors, by jointly calibrating the target sensor and other sensors, a transformation matrix can be obtained to convert the first position of the detected object in the sensing data of the other sensors into the pixel region in the target sensor, thereby obtaining the position transformation relationship between the other sensors and the target sensor.

[0063] In some embodiments, the method further includes the following steps S121 and S122, wherein:

[0064] Step S121: The sensing data of the signal receiver and the sensing data of the image sensor are fused to obtain the fused data of the same detection object in the current time period;

[0065] In practice, when multiple pixel regions corresponding to the target object detected by the signal receiver are located within the detection box corresponding to the first position of the target object in the sensing data of the image sensor, and the key size of the target object matches the current type, the sensing data of the signal receiver and the sensing data of the image sensor are fused to obtain the fused data of the same target object in the current time period.

[0066] Step S122: Based on the fused data from at least two time periods, determine the structured data of the same detection object.

[0067] Here, if the same detection object is a vehicle, the structured data of the same detection object can be highly organized and neatly formatted data such as the vehicle model, license plate number, and color.

[0068] In some embodiments, reference Figure 5 The detection object is equipped with a positioning unit, and the coordinate system corresponding to the positioning unit has a coordinate transformation relationship with the coordinate system corresponding to the target sensing device; the implementation of step S103, "determining the detection object based on the judgment result of whether the second position and the first position of the detection object detected by the target sensing device are within a preset area", may include the following steps S501 to S503, wherein:

[0069] Step S501: Obtain the third position of the detected object from the positioning unit;

[0070] Here, the positioning unit can be an On-Board Unit (OBU), which uses Dedicated Short Range Communications (DSRC) technology to communicate with Roadside Units (RSUs) via microwave. In an Electronic Toll Collection (ETC) system, the OBU is placed on the vehicle, and Roadside Units are installed on the roadside. The OBU and Roadside Units communicate with each other via microwave. When a vehicle passes through a Roadside Unit at high speed, the OBU and Roadside Unit use microwave communication to identify the vehicle, obtain the vehicle type, calculate the toll, and deduct the toll.

[0071] Step S502: Based on the coordinate transformation relationship, determine the fourth position of the detected object in the target sensing device;

[0072] During implementation, given the known position of the object being detected (i.e., the third position), the position of the object detected by the positioning unit in the target sensing device (i.e., the fourth position) is determined by the coordinate transformation relationship between the coordinate system corresponding to the object detected by the positioning unit and the coordinate system corresponding to the object detected by the target sensing device.

[0073] Step S503: When the second position, the fourth position and the first position of the detected object detected by the target sensing device are in a preset area, it is determined that the sensing data of the other sensing devices and the sensing data of the target sensing device belong to the same detected object.

[0074] During implementation, if the fourth position of the detected object detected by the positioning unit in the target sensing device and the first position of the detected object detected by the target sensing device are both within the preset area, it can be determined that the sensing data of other sensing devices and the sensing data of the target sensing device belong to the same detected object, thereby realizing the data fusion of the sensing data of the target sensing device and the sensing data of other sensing devices.

[0075] In vehicle-road cooperative scenarios, each sensor has its own relatively accurate attribute data. For example, radar (i.e., the other sensing devices or signal receivers mentioned above) describes speed and position information more accurately, while cameras (i.e., the target sensing devices or image sensors mentioned above) describe some visual attributes (e.g., license plates, vehicle colors, and vehicle types) more accurately.

[0076] To provide more comprehensive information on traffic participants, data collected from different sensors can be fused. However, due to the low similarity in attributes between objects detected by cameras and other sensors, accurately fusing descriptive information of traffic participants (e.g., vehicles, pedestrians, and non-motorized vehicles) in camera footage with information collected from other sensors presents a challenge.

[0077] To address the aforementioned problems, this application proposes a data fusion method, which includes the following two parts:

[0078] The first part involves linking the camera's pixel location information with the real-world Global Positioning System (GPS) location information after the camera calibration is completed. This allows the camera to determine its pixel location information within the camera's view (i.e., the second location mentioned above) through the GPS location information.

[0079] The second part involves data fusion between the camera and other sensors (such as radar):

[0080] The fusion of cameras with other sensors is based on the fusion of location information. GPS location information (i.e., the aforementioned sensor data) collected by radar, OBU, etc., is converted into camera pixel coordinates (represented as points) regions (i.e., the aforementioned pixel regions) through a calibration matrix (i.e., the aforementioned transformation matrix). When the camera (detecting vehicles on the road, with backend servers / cloud servers processing the data to achieve fusion) detects a target, it draws a bounding box (BBox) around the detected object (i.e., the aforementioned detection box), and also obtains structured attribute information data such as the object's color and vehicle type (i.e., the aforementioned structured data). At this point, it determines which specific identity the converted pixel belongs to. In the bounding box of the target detection (Document ID), if a pixel is within the bounding box of a traffic participant, it can be preliminarily determined that the camera target and the target detected by the sensor are fused together. If multiple traffic participants are within the same bounding box of the same sensor, the appropriate target ID (i.e., the target detection object mentioned above) can be selected for fusion based on information such as the vehicle type detected by the camera and the length of vehicles detected by other sensors (if a detection box includes the first vehicle and the second vehicle, it can also be determined whether the pixel belongs to the first vehicle or the second vehicle). At this time, the target detected by the camera (i.e., the sensor data mentioned above) is fused together with the targets of other sensors.

[0081] As can be seen from the above, the data fusion method provided in this application firstly completes camera calibration, associating the camera pixel position information with the GPS position information of the real scene; secondly, the pixel position information in the camera image can be obtained through the GPS position information; then, the GPS position information of traffic participants output by other sensors is converted into corresponding pixel position information (represented in the form of points) through the calibration matrix, and it is determined whether the pixel point corresponding to the GPS position of a traffic participant with a specific ID is in the pixel box detected by the camera target (i.e., the above-mentioned pixel area), thereby associating the target tracking ID of other sensors with the target tracking ID in the camera image, and completing the fusion of multi-sensor data and camera data.

[0082] Figure 6 A schematic diagram illustrating the implementation flow of another data fusion method provided in this application embodiment is shown below. Figure 6 As shown, the method includes the following steps:

[0083] Step 601: Obtain sensor data reported by other sensing devices (e.g., signal receivers);

[0084] Here, the first position of the detected object is determined based on the sensing data from other sensing devices;

[0085] Step 602: Obtain the sensing data reported by the camera (i.e., the aforementioned target sensing device or image sensor);

[0086] Here, the initial location of the detected object is determined based on the sensor data reported by the camera. It should be noted that the detected object detected by other sensing devices may be the same as or different from the detected object detected by the camera; that is, the objects within the detection range of different other sensing devices may be the same or different.

[0087] Step 603: Convert the first position of the detected object detected by other sensing devices into pixel coordinates (i.e., the aforementioned pixel region) using a transformation matrix;

[0088] Step 604: Calculate whether the pixel coordinates after position transformation are within the rectangular box (i.e., the detection box mentioned above) where the camera detected the target;

[0089] For example, the images captured by the camera at different times are shown in Figure 7(a) and Figure 7(b). Figure 7(a) includes vehicle detection boxes 20 to 23, and Figure 7(b) also includes vehicle detection boxes 20 to 23. The difference is that the position of the vehicle is different at different times. The 1 in the detection box on vehicle 20 to 23 represents the pixel position corresponding to the OBU GPS, and the 2 in the detection box on vehicle 20 to 23 represents the pixel position corresponding to the radar GPS.

[0090] Step 605: Determine if the pixel coordinates are within the rectangle;

[0091] If the pixel coordinates are not within the rectangle, proceed to step 606; if the pixel coordinates are within the rectangle, proceed to step 607. For example, continuing to refer to Figures 7(a) and 7(b), the fused pixel coordinates 1 and 2 are both within the detection box, therefore they are determined to belong to the same detection object, namely vehicles 20 to 23.

[0092] Step 606: Determine that the detection object detected by other sensing devices and the detection object detected by the camera belong to different targets (i.e., the aforementioned detection object).

[0093] Step 607: Calculate whether there are multiple targets for the same other sensing device;

[0094] Step 608: Determine whether multiple targets are within the bounding box;

[0095] If multiple targets are within the rectangle, proceed to step 609; if multiple targets are not within the rectangle, proceed to step 610.

[0096] Step 609: Select a more accurate target for fusion based on the attributes of type and vehicle length (i.e., the structured data mentioned above);

[0097] Step 610: Output: Fusion successful.

[0098] Compared with the prior art, the embodiments of this application have the following advantages: 1. Data fusion of camera information and information from other sensing devices is realized; 2. The comprehensiveness of traffic participant information is improved.

[0099] Based on the foregoing embodiments, this application provides a data fusion device, which includes various modules, sub-modules included in each module, units included in each sub-module, and sub-units included in each unit. This device can be implemented using a processor in a computer device; alternatively, it can be implemented using specific logic circuits. During implementation, the processor can be a Central Processing Unit (CPU), a Microprocessor Unit (MPU), a Digital Signal Processor (DSP), or a Field Programmable Gate Array (FPGA), etc.

[0100] Figure 8 This is a schematic diagram of the composition structure of a data fusion device provided in an embodiment of this application, as shown below. Figure 8 As shown, the device 800 includes: a first acquisition module 810, a conversion module 820, and a first determination module 830, wherein:

[0101] The first acquisition module 810 is used to acquire sensing data collected by the target sensing device and other sensing devices respectively; wherein, the sensing data includes at least the first position of the detected object;

[0102] The conversion module 820 is used to convert the first position of the detected object detected by the other sensing devices based on the position conversion relationship between the target sensing device and the other sensing devices, so as to obtain the second position of the detected object detected by the other sensing devices in the target sensing device.

[0103] The first determining module 830 is used to determine the detection object based on the judgment result of whether the second position and the first position of the detection object detected by the target sensing device are in a preset area; wherein, the detection object is an object whose sensing data of the other sensing devices and the sensing data of the target sensing device are the same.

[0104] In some embodiments, the device 800 further includes:

[0105] The second acquisition module is used to acquire the location information of the target sensing device and the location information of the other sensing devices;

[0106] The joint calibration module is used to perform joint calibration on the target sensing device and the other sensing devices based on the location information of the target sensing device and the location information of the other sensing devices, so as to obtain the position transformation relationship between the other sensing devices and the target sensing device.

[0107] In some embodiments, the target sensing device is an image sensor, and the other sensing device is a signal receiver. The first determining module 830 is used to determine the detection object based on the judgment result of whether the pixel region and the first position of the detection object detected by the image sensor are within a preset area when the detection object in the sensing data of the signal receiver is represented as a pixel region in the second position of the image sensor.

[0108] In some embodiments, the first determining module 830 includes:

[0109] The first determining submodule is used to determine the positional relationship between the pixel region and the detection box when the detection box is used to characterize the first position of the detected object detected by the image sensor;

[0110] The second determining submodule is used to determine, when the positional relationship indicates that the pixel region is located in the detection frame, that the sensing data of the signal receiver and the sensing data of the image sensor belong to the same detection object.

[0111] In some embodiments, at least two detection objects are detected in the signal receiver; the first determining module 830 includes:

[0112] The third determining submodule is used to determine the current type of the detected object detected by the image sensor based on the sensing data collected by the image sensor when the at least two pixel regions corresponding to the at least two detected objects are respectively located in the corresponding detection boxes;

[0113] The fourth determination submodule is used to determine the critical dimensions of each detected object in the signal receiver based on the sensing data collected by the signal receiver;

[0114] The fifth determination submodule is used to determine a target detection object from the at least two detection objects; wherein the key dimensions of the target detection object match the current type;

[0115] The sixth determination submodule is used to determine that the sensing data corresponding to the target detection object and the sensing data of the image sensor belong to the same detection object.

[0116] In some embodiments, the device 800 further includes:

[0117] The fusion module is used to fuse the sensing data of the signal receiver and the sensing data of the image sensor to obtain the fused data of the same detection object in the current time period.

[0118] The second determining module is used to determine the structured data of the same detection object based on the fused data from at least two time periods.

[0119] In some embodiments, a positioning unit is installed on the detection object, and the coordinate system corresponding to the positioning unit has a coordinate transformation relationship with the coordinate system corresponding to the target sensing device; the first determining module 530 includes:

[0120] The first acquisition subunit is used to acquire the third position of the detected object from the positioning unit;

[0121] The seventh determining subunit is used to determine the fourth position of the detected object in the target sensing device based on the coordinate transformation relationship;

[0122] The eighth determining subunit is used to determine that the sensing data of the other sensing devices and the sensing data of the target sensing device belong to the same detection object when the second position, the fourth position and the first position of the detection object detected by the target sensing device are in a preset area.

[0123] In some embodiments, the first position of the detected object in the other sensing device is characterized as the centroid of the detected object in the other sensing device; the conversion unit 830 is used to determine the second position of the detected object detected by the other sensing device in the target sensing device based on the spatial coordinates of the centroid and the conversion matrix.

[0124] The descriptions of the apparatus embodiments above are similar to those of the method embodiments above, and have similar beneficial effects. In some embodiments, the functions or modules included in the apparatus provided in this application can be used to perform the methods described in the method embodiments above. For technical details not disclosed in the apparatus embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0125] It should be noted that, in the embodiments of this application, if the above-described data fusion method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the related technology, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware, software, or firmware, or any combination of hardware, software, and firmware.

[0126] This application also provides a computer device, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements some or all of the steps in the above-described method.

[0127] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements some or all of the steps in the above-described method. The computer-readable storage medium may be transient or non-transient.

[0128] This application also provides a computer program including computer-readable code, wherein when the computer-readable code is run in a computing device, a processor in the computing device performs some or all of the steps in the above-described method.

[0129] This application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, it implements some or all of the steps in the above-described method. This computer program product can be implemented specifically through hardware, software, or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium; in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc.

[0130] It should be noted that the descriptions of the various embodiments above tend to emphasize the differences between them, while their similarities or commonalities can be referred to interchangeably. The descriptions of the above embodiments of the device, storage medium, computer program, and computer program product are similar to the descriptions of the above method embodiments and have similar beneficial effects. For technical details not disclosed in the embodiments of the device, storage medium, computer program, and computer program product of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0131] It should be noted that, Figure 9 This is a schematic diagram of a hardware entity of a computer device in an embodiment of this application, such as... Figure 9 As shown, the hardware entity of the computer device 900 includes: a processor 901, a communication interface 902, and a memory 903, wherein:

[0132] Processor 901 typically controls the overall operation of computer device 900.

[0133] Communication interface 902 enables computer devices to communicate with other terminals or servers over a network.

[0134] The memory 903 is configured to store instructions and applications executable by the processor 901, and can also cache data to be processed or already processed (e.g., image data, audio data, voice communication data, and video communication data) in the processor 901 and various modules in the computer device 900. It can be implemented using flash memory or random access memory (RAM). Data transfer between the processor 901, the communication interface 902, and the memory 903 can be performed via bus 904.

[0135] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above steps / processes do not imply a sequential order of execution; the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above embodiments of this application are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0136] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0137] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0138] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0139] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0140] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0141] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence or the part that contributes to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, magnetic disks, or optical disks.

[0142] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A data fusion method, characterized in that, The method includes: Acquire sensing data collected by the target sensing device and other sensing devices respectively; wherein, the sensing data includes at least the first position of the detected object; Based on the position transformation relationship between the target sensing device and the other sensing devices, the first position of the detected object detected by the other sensing devices is transformed to obtain the second position of the detected object in the target sensing device; the target sensing device is an image sensor, and the other sensing devices are signal receivers; the second position is a pixel region; When at least two pixel regions corresponding to at least two detected objects detected by the signal receiver are respectively located in the corresponding detection frame, the current type of the detected objects detected by the image sensor is determined based on the sensing data collected by the image sensor; The key dimensions of each detected object in the signal receiver are determined based on the sensor data collected by the signal receiver; A target detection object is determined from the at least two detection objects; wherein the key dimensions of the target detection object match the current type; The sensor data corresponding to the target detection object and the sensor data of the image sensor are determined to belong to the same detection object. Then, the sensor data of the target sensor device belonging to the same detection object is fused with the sensor data of the other sensor devices.

2. The method according to claim 1, characterized in that, The method further includes: Obtain the location information of the target sensing device and the location information of the other sensing devices; Based on the location information of the target sensing device and the location information of the other sensing devices, the target sensing device and the other sensing devices are jointly calibrated to obtain the position transformation relationship between the other sensing devices and the target sensing device.

3. The method according to claim 1, characterized in that, The method further includes: In the case of using a detection box to characterize the first position of the detected object detected by the image sensor, the positional relationship between the pixel region and the detection box is determined; If the positional relationship indicates that the pixel region is located within the detection frame, it is determined that the sensing data of the signal receiver and the sensing data of the image sensor belong to the same detection object.

4. The method according to claim 1, characterized in that, The method further includes: The sensing data from the signal receiver and the sensing data from the image sensor are fused to obtain fused data of the same detected object in the current time period; Based on fused data from at least two time periods, the structured data of the same detection object is determined.

5. The method according to claim 1, characterized in that, The detection object is equipped with a positioning unit, and the coordinate system corresponding to the positioning unit has a coordinate transformation relationship with the coordinate system corresponding to the target sensing device; the method further includes: Obtain the third position of the detected object from the positioning unit; Based on the coordinate transformation relationship, the fourth position of the detected object in the target sensing device is determined; If the second position, the fourth position, and the first position of the detected object detected by the target sensing device are within a preset area, it is determined that the sensing data of the other sensing devices and the sensing data of the target sensing device belong to the same detected object.

6. The method according to any one of claims 1 to 5, characterized in that, The first position of the detected object detected by the other sensing device is characterized as the center of gravity of the detected object detected by the other sensing device; The step of converting the first position of the detected object detected by the other sensing devices based on the position conversion relationship between the target sensing device and the other sensing devices to obtain the second position of the detected object detected by the other sensing devices in the target sensing device includes: Based on the spatial coordinates of the center of gravity and the transformation matrix, the second position of the detected object detected by the other sensing devices in the target sensing device is determined.

7. A data fusion device, characterized in that, The device includes: The first acquisition module is used to acquire sensing data collected by the target sensing device and other sensing devices respectively; wherein, the sensing data includes at least the first position of the detected object; A conversion module is used to convert the first position of the detected object detected by the other sensing devices based on the position conversion relationship between the target sensing device and the other sensing devices, to obtain the second position of the detected object detected by the other sensing devices in the target sensing device; the target sensing device is an image sensor, and the other sensing devices are signal receivers; the second position is a pixel region; The first determining module is configured to: determine the current type of the detected objects detected by the image sensor based on the sensing data collected by the image sensor, when at least two pixel regions corresponding to at least two detected objects detected by the signal receiver are respectively located in corresponding detection frames; determine the key dimensions of each detected object in the signal receiver based on the sensing data collected by the signal receiver; determine a target detected object from the at least two detected objects; wherein the key dimensions of the target detected object match the current type; determine that the sensing data corresponding to the target detected object and the sensing data of the image sensor belong to the same detected object, and then perform data fusion on the sensing data of the target sensing device belonging to the same detected object and the sensing data of the other sensing devices.

8. A computer device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 6.

Images