An information fusion method, apparatus, and electronic device

Abstract

An embodiment of the present invention provides an information fusion method, apparatus and electronic device, which are applied to the field of vision processing technology. The method includes: obtaining multiple frames of images continuously collected by each camera in a plurality of cameras for a target area during the movement of a target object in the target area; for each image, determining the time dimension offset of the same object in the image and its first adjacent image on the imaging plane of the main camera; for each image, determining the spatial dimension offset of the same object in the image and its second adjacent image on the imaging plane of the main camera; based on the time dimension offset and spatial dimension offset of each image, fusing the object information about the target object included in each image to obtain the complete object information of the target object. Through this solution, the complete object information of the target object can be automatically obtained, thereby improving the information acquisition efficiency of the target object.

Description

Technical Field

[0001] The present invention relates to the technical field of visual processing, and in particular, to an information fusion method, apparatus, and electronic device. Background Art

[0002] When the volume of a target object is large, it is difficult to obtain a clear and complete image of the target object through a single camera. For example, for the goods in and out of an automated factory, in the case of ensuring clarity, the field of view of a single camera is difficult to cover the entire goods, making it impossible to obtain a clear and complete image of the goods.

[0003] Since a clear and complete image of the target object cannot be obtained, it is impossible to automatically obtain information about the target object, such as the batch information of the goods marked on the goods, the entrance and exit time information, etc., through the analysis of the complete image. In related technologies, it mostly relies on manual copying of the information marked on the target object, with low efficiency. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide an information fusion method, apparatus, and electronic device to automatically obtain the complete object information of the target object, thereby improving the information acquisition efficiency of the target object.

[0005] In a first aspect, the embodiments of the present invention provide an information fusion method, which includes:

[0006] Obtain multiple frames of images continuously collected by each camera in a plurality of cameras for a target area during the movement of a target object in the target area; wherein, the overall picture content of each obtained image includes the target object;

[0007] For each image, determine the position offset of the same object in the first adjacent image of the image on the imaging plane of the main camera as the time dimension offset of the image; wherein, the first adjacent image of each image is: among the multiple frames of images collected by the camera to which the image belongs, the image whose acquisition time is adjacent to that of the image; the main camera is one of the multiple cameras;

[0008] For each image, determine the position offset of the same object in the second adjacent image of the image on the imaging plane of the main camera as the space dimension offset of the image; wherein, the second adjacent image of each image is: the image collected by the camera adjacent to the camera to which the image belongs at the acquisition time of the image;

[0009] Based on the time dimension offset and space dimension offset of each image, fuse the object information about the target object included in each image to obtain the complete object information of the target object.

[0010] Optionally, determining the position offset of the same object in the image and its first adjacent image on the imaging plane of the main camera as the time dimension offset of the image includes:

[0011] Determining the position offset of the same object in the image and its first adjacent image on the imaging plane of the camera to which the image belongs as the first offset of the image;

[0012] Calculating the ratio of the first average offset corresponding to the main camera to the first average offset corresponding to the designated camera as the first ratio; wherein, the designated camera is the camera to which the image belongs, and the first average offset corresponding to each camera is the mean value of the first offsets of the images collected by the camera;

[0013] Calculating the product of the first offset and the first ratio as the position offset of the same object in the image and its first adjacent image on the imaging plane of the main camera.

[0014] Optionally, determining the position offset of the same object in the image and its first adjacent image on the imaging plane of the camera to which the image belongs as the first offset of the image includes:

[0015] Determining the same object included in the image and its first adjacent image as the target object;

[0016] Calculating the difference between the position of the target object in the image and the position of the target object in the first adjacent image of the image as the first offset of the image.

[0017] Optionally, before determining, for each image, the position offset of the same object in the image and its second adjacent image on the imaging plane of the main camera as the spatial dimension offset of the image, the method further includes:

[0018] Determining the mapped image of each image on the imaging plane of the main camera;

[0019] Determining the position offset of the same object in the mapped image of the image and the mapped image of its second adjacent image on the imaging plane of the main camera as the spatial dimension offset of the image includes:

[0020] Calculating the position offset of the same object in the mapped image of the image and the mapped image of its second adjacent image on the imaging plane of the main camera as the spatial dimension offset of the image.

[0021] Optionally, determining the mapped image of each image on the imaging plane of the main camera includes:

[0022] For each image, calculate the ratio of the second offset of the image to the time - dimension offset of the image, and scale the image according to the ratio to obtain a scaled image, which is used as the mapped image of the image on the imaging plane of the main camera;

[0023] Among them, the second offset of each image is: the position offset of the same object in the image and its first adjacent image on the imaging plane of the camera to which the image belongs.

[0024] Optionally, the method of fusing the object information about the target object included in each image based on the time - dimension offset and the space - dimension offset of each image to obtain the complete object information of the target object includes:

[0025] Based on the space - dimension offset of each image collected by each camera, determine the homography matrix between the imaging plane of each camera and the imaging plane of the main camera as the homography matrix of each camera; among them, the homography matrix of each camera indicates: the mapping relationship between the imaging plane of the camera and the splicing area in the imaging plane of the main camera; the splicing area is: the area used for splicing the images collected by each camera at the same acquisition moment;

[0026] For each image, based on the homography matrix of the camera to which the image belongs, determine the first position information of the object information included in the image in the splicing area, and based on the time - dimension offset of the image and the first position information, determine the second position information of the object information included in the image on the imaging plane of the main camera after fusion;

[0027] Based on the second position information of the object information included in each image, fuse the object information about the target object included in each image to obtain the complete object information of the target object.

[0028] Optionally, the method of fusing the object information about the target object included in each image based on the second position information of the object information included in each image to obtain the complete object information of the target object includes:

[0029] Splice the object information about the target object included in each image on the imaging plane of the main camera according to the second position information of the object information included in each image to obtain the complete object information of the target object.

[0030] Optionally, the main camera is: the camera among all cameras that has the largest number of common images with other cameras; the existence of a common image between two cameras means: the two cameras simultaneously collect valid images at the same acquisition moment, and the valid image is: an image whose time - dimension offset meets a preset condition.

[0031] Optionally, the main camera is determined from the multiple cameras in the following manner, including:

[0032] Determine a target camera from the multiple cameras;

[0033] From each of the first cameras, determine a candidate camera that has the largest number of common images with the clustering cameras; wherein, the first cameras are the cameras among the multiple cameras except the clustering cameras, and the clustering cameras include the target camera;

[0034] If the ratio of the time - dimension offset of the candidate camera to the time - dimension offset of the clustering camera is within the specified ratio range, add the candidate camera to the clustering cameras; otherwise, remove the candidate camera from the first cameras;

[0035] Return to execute the step of determining a candidate camera that has the largest number of common images with the clustering cameras from the first cameras until the number of the first cameras is zero;

[0036] Determine the clustering score of the clustering cameras as the clustering score corresponding to the target camera; wherein, the clustering score of the clustering cameras is: the sum of the number of common images between each camera in the clustering cameras and other cameras;

[0037] Return to execute the step of determining a target camera from the multiple cameras until the clustering scores of each camera in the multiple cameras are determined;

[0038] Take the camera with the largest clustering score as the main camera.

[0039] In a second aspect, an information fusion device is provided according to an embodiment of the present invention. The device includes:

[0040] An image acquisition module, configured to acquire multiple frames of images continuously collected by each of the multiple cameras for a target area during the movement of a target object in the target area; wherein, the overall picture content of each acquired image includes the target object;

[0041] A first offset determination module, configured to, for each image, determine the position offset on the imaging plane of the main camera of the same object in the image and its first adjacent image as the time - dimension offset of the image; wherein, the first adjacent image of each image is: the image adjacent in acquisition time among the multiple frames of images collected by the camera to which the image belongs; the main camera is one of the multiple cameras;

[0042] A second offset determination module, configured to determine, for each image, the position offset of the same object in the image and its second adjacent image on the imaging plane of the main camera as the spatial dimension offset of the image; wherein, the second adjacent image of each image is the image captured by the camera adjacent to the camera to which the image belongs at the acquisition moment of the image.

[0043] An information fusion module, configured to fuse the object information about the target object included in the images based on the time dimension offset and the spatial dimension offset of the images, so as to obtain the complete object information of the target object.

[0044] In a third aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete communication with each other through the communication bus.

[0045] The memory is used for storing a computer program.

[0046] The processor is configured to implement the method steps of any one of the first aspects when executing the program stored on the memory.

[0047] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the method steps of any one of the first aspects are implemented.

[0048] Advantageous effects of the embodiments of the present invention:

[0049] An information fusion method, apparatus, and electronic device provided by an embodiment of the present invention can obtain multiple frames of images continuously collected by each camera in multiple cameras for a target area during the movement of a target object in the target area. And for each image, determine the position offset of the same object in the image and its first adjacent image on the imaging plane of the main camera as the time dimension offset of the image, and for each image, determine the position offset of the same object in the image and its second adjacent image on the imaging plane of the main camera as the spatial dimension offset of the image. Then, based on the time dimension offset and spatial dimension offset of each image, fuse the object information about the target object included in each image to obtain the complete object information of the target object. Since the overall picture content of each obtained image contains the target object, it means that the complete object information of the target object is included in each obtained image. Further, determine the time dimension offset of each image, which indicates the offset amount in the movement direction of the target object between each image and its adjacent image taken by the same camera, and determine the spatial dimension offset of each image, which indicates the offset amount between each image and the image taken by the adjacent camera at the same acquisition time. Thus, the object information included in each image can be fused based on the time dimension offset and spatial dimension offset of each image to obtain the complete object information of the target object. It can be seen that by adopting the solution provided by the embodiment of the present invention, the complete object information of the target object can be automatically obtained, thereby improving the information acquisition efficiency of the target object.

[0050] Of course, it is not necessary for any product or method implementing the present invention to achieve all the above-mentioned advantages simultaneously. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention, and those of ordinary skill in the art can also obtain other embodiments based on these drawings.

[0052] FIG. 1(a) is a schematic diagram of a cargo transportation scenario provided by an embodiment of the present invention;

[0053] FIG. 1(b) is a schematic diagram of the target object areas included in each image at the same acquisition time provided by an embodiment of the present invention;

[0054] Figure 2 is a flowchart of the information fusion method provided by an embodiment of the present invention;

[0055] Figure 3 is a schematic diagram of the camera arrangement provided by an embodiment of the present invention;

[0056] Figure 4 A schematic diagram of an image provided by an embodiment of the present invention;

[0057] Figure 5 Another flowchart of the information fusion method provided by an embodiment of the present invention;

[0058] Figure 6 Another flowchart of the information fusion method provided by an embodiment of the present invention;

[0059] Figure 7 Another flowchart of the information fusion method provided by an embodiment of the present invention

[0060] Figure 8 A schematic structural diagram of the information fusion device provided by an embodiment of the present invention;

[0061] Figure 9 A schematic structural diagram of the electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0062] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art based on the present invention belong to the scope of protection of the present invention.

[0063] For a target object with a large volume, in the case of ensuring clarity, a single camera can often only collect a partial image of the target object, and the partial image of the target object often only contains partial object information of the target object, making it impossible to automatically obtain the complete object information of the target object in the related art.

[0064] In the embodiments of the present invention, in order to be able to automatically obtain the complete object information of the target object, a multi-camera method is adopted to collect images during the movement of the target object.

[0065] As shown in Figure 1(a), taking the goods transportation scenario as an example for illustration, in Figure 1(a), the position of the left transportation trolley is the starting point of goods transportation, the position of the transportation trolley formed by the right dotted line is the ending point of goods transportation, and the middle rectangular area contains multiple cameras, and each camera is used to collect images of the front view. During the process of the transportation trolley transporting the goods from the starting point of transportation to the ending point of transportation, multiple cameras can continuously collect images.

[0066] At each acquisition moment during the transportation of goods, each camera can acquire a partial image of the goods, and the partial areas included in the images acquired by each camera at the same acquisition moment cover at least all areas of the target object in the direction perpendicular to its movement direction. Thus, during the entire movement process of the target object, by continuously acquiring multiple images with each camera among multiple cameras, the overall picture content of the images acquired by the multiple cameras during the entire movement process can include the target object.

[0067] Exemplarily, as shown in FIG. 1(b), area A in the figure is the partial area included in the images acquired by multiple cameras at the first acquisition moment, which is the area located on the target object; area B in the figure is the partial area included in the images acquired by multiple cameras at the second acquisition moment, which is the area located on the target object; area C in the figure is the partial area included in the images acquired by multiple cameras at the third acquisition moment, which is the area located on the target object. Thus, during the entire movement process, the overall picture content of the images acquired by the multiple cameras includes area A, area B, and area C, including the complete target object.

[0068] On this basis, in order to automatically obtain the complete object information of the target object and thus improve the information acquisition efficiency of the target object, the embodiments of the present invention provide an information fusion method, device, and electronic device.

[0069] It should be noted that in specific applications, the embodiments of the present invention can be applied to various electronic devices, such as personal computers, servers, mobile phones, and other devices with data processing capabilities. Moreover, the information fusion method provided by the embodiments of the present invention can be implemented in a software, hardware, or software-hardware combination manner.

[0070] Among them, the information fusion method provided by the embodiments of the present invention may include:

[0071] Acquire multiple frames of images continuously acquired by each camera among multiple cameras for a target area during the movement of the target object in the target area; wherein, the overall picture content of the acquired images includes the target object;

[0072] For each image, determine the position offset of the same object in the image and its first adjacent image on the imaging plane of the main camera as the time dimension offset of the image; wherein, the first adjacent image of each image is: among the multiple frames of images acquired by the camera to which the image belongs, the image whose acquisition moment is adjacent to that of the image; the main camera is one of the multiple cameras;

[0073] For each image, determine the position offset on the imaging plane of the main camera of the same object in the image and the second adjacent image of the image as the spatial dimension offset of the image; wherein, the second adjacent image of each image is: the image collected by the camera adjacent to the camera to which the image belongs at the acquisition time of the image.

[0074] Based on the time dimension offset and spatial dimension offset of each image, fuse the object information about the target object included in each image to obtain the complete object information of the target object.

[0075] In the above solution of the embodiment of the present invention, since the overall picture content of each acquired image includes the target object, it means that the complete object information of the target object is included in each acquired image. Further, determine the time dimension offset of each image, which indicates the offset amount between each image and the adjacent image of the same camera in the moving direction of the target object, and determine the spatial dimension offset of each image, which indicates the offset amount between each image and the image collected by the adjacent camera at the same acquisition time, so that the object information included in each image can be fused based on the time dimension offset and spatial dimension offset of each image to obtain the complete object information of the target object. It can be seen that by using the solution provided by the embodiment of the present invention, the complete object information of the target object can be automatically obtained, thereby improving the information acquisition efficiency of the target object.

[0076] Next, the information fusion method provided by the embodiment of the present invention will be elaborated in detail with reference to the accompanying drawings of the specification.

[0077] As Figure 2 shown, the embodiment of the present invention provides an information fusion method, including steps S201-S204, wherein:

[0078] S201, acquire multiple frames of images continuously collected by each camera among multiple cameras for the target area during the movement of the target object in the target area; wherein, the overall picture content of each acquired image includes the target object.

[0079] Wherein, the above target object is an object for which object information needs to be acquired, such as goods for outbound / inbound, etc. The above target area is the area passed by the target object during movement, such as the lane passed by the outbound / inbound. The camera fields of view of each of the multiple cameras face the target area, and thus, during the movement of the target object in the target area, each camera can continuously acquire multiple frames of images for the target area.

[0080] To ensure that the overall picture content of each acquired image includes the target object, the multiple cameras can be arranged in a linear array or a planar array and installed on at least one side of the moving direction of the target object during the movement in the target area. Exemplarily, as Figure 3As shown in the figure, an embodiment of the present invention provides a schematic diagram of camera arrangement. The object in the middle position of the front view and the top view is the target object, and multiple cameras are arranged on both sides. It can be seen from the front view and the top view that multiple cameras are installed on both sides of the target object, so the overall picture content of the images obtained by the multiple cameras on each side includes the target object.

[0081] Optionally, in order to obtain multiple frames of images continuously collected by each camera among the multiple cameras during the movement of the target object in the target area, when it is detected that the target object enters the target area, a start signal can be sent to each camera to enable each camera to start image acquisition. When it is detected that the target object leaves the target area, an end signal can be sent to each camera to enable each camera to end image acquisition, and during this process, the images collected by each camera can be obtained in real time, or after sending the end signal to each camera, the multiple images collected by each camera can be uniformly obtained from each camera.

[0082] Exemplarily, as shown in Table 1:

[0083] Table 1

[0084] T1 T2 T3 T4 Camera 1 Image 11 Image 12 Image 13 Image 14 Camera 2 Image 21 Image 22 Image 23 Image 24 Camera 3 Image 31 Image 32 Image 33 Image 34

[0085] In the above table, the multiple cameras include Camera 1, Camera 2, and Camera 3; T1, T2, T3, and T4 are 4 consecutive acquisition times during the movement of the target object in the target area; Image 11, Image 12, Image 13, and Image 14 are multiple frames of images continuously collected by Camera 1 for the target area during the movement of the target object in the target area; Image 21, Image 22, Image 23, and Image 24 are multiple frames of images continuously collected by Camera 2 for the target area during the movement of the target object in the target area; Image 31, Image 32, Image 33, and Image 34 are multiple frames of images continuously collected by Camera 3 for the target area during the movement of the target object in the target area.

[0086] S202. For each image, determine the position offset of the same object in the imaging plane of the main camera between this image and its first adjacent image, as the time dimension offset of this image;

[0087] Among them, the first adjacent image of each image is: among the multiple frames of images collected by the camera to which this image belongs, the image whose acquisition time is adjacent to this image.

[0088] Optionally, the first adjacent image of each image may be the image captured by the camera to which the image belongs at the previous capture moment and / or the next capture moment of capturing this image. Exemplarily, taking Table 1 as an example, Camera 1 continuously captures four images: Image 11, Image 12, Image 13, and Image 14. The first adjacent image of Image 11 is Image 12, the first adjacent image of Image 12 is Image 11 and Image 13, the first adjacent image of Image 13 is Image 12 and Image 14, and the first adjacent image of Image 14 is Image 3.

[0089] The same object in the image and the first adjacent image of the image may be the same image texture or the same fixed pattern. Thus, the position offset between the two images mentioned in the embodiments of the present invention can be understood as the position offset of the same image texture or the same fixed pattern in the two images.

[0090] Optionally, in order to determine the position offset of the same object in the image and the first adjacent image of the image on the imaging plane of the main camera, the following steps may be adopted:

[0091] Step A1: Determine the position offset of the same object in the image and the first adjacent image of the image on the imaging plane of the camera to which the image belongs as the first offset of the image.

[0092] Since the image and the first adjacent image of the image are two frames of images captured by the same camera, the position offset of the same object in the image and the first adjacent image of the image on the imaging plane of the camera to which the image belongs can be understood as the pixel position difference of the same object in the image and the first adjacent image of the image.

[0093] In one implementation, the same object included in the image and the first adjacent image of the image may be first determined as the target object, and then the difference between the position of the target object in the image and the position of the target object in the first adjacent image of the image is calculated as the position offset of the same object in the image and the first adjacent image of the image on the imaging plane of the main camera.

[0094] Exemplarily, as Figure 4 shown, the embodiments of the present invention provide a schematic diagram of an image. Figure 4In it, the left box represents image 11, the right box represents image 12, and the black squares in the left box and the right box represent the same object in image 11 and image 12. Then, the black square can be determined as the target object, and further, the pixel coordinates of the target object in image 11 are determined to be (x1, y1), and the pixel coordinates of the target object in image 11 are determined to be (x2, y2). Further, calculate the difference in the position of the target object between image 11 and image 12 as: Δx = x1 - x2, Δy = y1 - y2. Δx represents the position offset in the horizontal coordinate direction, and Δy represents the position offset in the vertical coordinate direction. Then, the first offset of image 11 is (Δx, Δy).

[0095] Step A2, calculate the ratio of the first average offset corresponding to the main camera to the first average offset corresponding to the specified camera as the first ratio;

[0096] After determining the first offset of the image, the ratio of the first average offset corresponding to the main camera to the first average offset corresponding to the specified camera can be calculated. Here, the specified camera is the camera to which the image belongs.

[0097] The first average offset corresponding to each camera is: the average value of the first offsets of the images collected by the camera.

[0098] For example, among images 11, 12, 13, and 14 collected by camera 1, the first offset of image 11 is Δ11, the first offset of image 12 is Δ12, the first offset of image 13 is Δ13, and the first offset of image 14 is Δ14. Then, the first average offset corresponding to camera 1 is Δ1 = (Δ11 + Δ12 + Δ13 + Δ14) / 4. Camera 2 is the main camera. Among images 21, 22, 23, and 24 collected by it, the first offset of image 21 is Δ21, the first offset of image 22 is Δ22, the first offset of image 23 is Δ23, and the first offset of image 24 is Δ24. Then, the first average offset corresponding to camera 2 is Δ2 = (Δ21 + Δ22 + Δ23 + Δ24) / 4. Then, the ratio φ of the first average offset corresponding to the main camera to the first average offset corresponding to the specified camera is φ = Δ2 / Δ1 as the first ratio.

[0099] Step A3, calculate the product of the first offset and the first ratio as the position offset of the same object in the image and its first adjacent image on the imaging plane of the main camera.

[0100] After calculating the first ratio, the product of the first offset and the first ratio can be calculated as the position offset of the same object in the image and its first adjacent image on the imaging plane of the main camera.

[0101] For example, the first offset corresponding to Image 1 is Δ11, and the first ratio corresponding to Camera 1 to which Image 1 belongs is φ. Then, calculate the product of the first offset and the first ratio = Δ11 × φ, which is used as the position offset of the same object in the image and its first adjacent image on the imaging plane of the main camera.

[0102] S203. For each image, determine the position offset of the same object in the image and its second adjacent image on the imaging plane of the main camera, and use it as the spatial dimension offset of the image.

[0103] Among them, the second adjacent image of each image is: the image collected by the camera adjacent to the camera to which the image belongs at the acquisition moment of the image.

[0104] Exemplarily, taking Table 1 as an example for illustration, Camera 1 continuously captures four images: Image 11, Image 12, Image 13, and Image 14; Camera 2 continuously captures four images: Image 21, Image 22, Image 23, and Image 24; Camera 3 continuously captures four images: Image 31, Image 32, Image 33, and Image 34. Among them, the second adjacent image of Image 11 is Image 21, the second adjacent images of Image 21 are Image 11 and Image 31, and the second adjacent image of Image 31 is Image 21.

[0105] In this step, the specific implementation method for determining the position offset of the same object in the image and its second adjacent image on the imaging plane of the main camera will be elaborated in detail in the subsequent embodiments and will not be elaborated here.

[0106] S204. Based on the time dimension offset and spatial dimension offset of each image, fuse the object information about the target object included in each image to obtain the complete object information of the target object.

[0107] After determining the time dimension offset and spatial dimension offset of each image, the time dimension offset and spatial dimension offset can be combined to map each image onto the imaging plane of the main camera. Then, the object information mapped to the same area is merged and fused, and for the same type of object information merged together, calculate the information offset between the image to which the object information belongs, its first adjacent image, and its second adjacent image. Furthermore, correct the time dimension offset of the image to which the object information belongs, and then fuse the object information about the target object included in each image again according to the corrected time dimension offset.

[0108] In one implementation, in order to improve the accuracy of fusion and eliminate the pixel position deviation caused by different depths of field between each camera and the main camera, further, the depth of field coefficient of each frame of image can be calculated, which represents the ratio of the depth of field of the imaging plane of the camera to which the image belongs to the depth of field of the imaging plane of the main camera.

[0109] Optionally, the depth of field coefficient of each frame of image can be calculated using the following formula:

[0110] Homoscale = scale speed *scale offset

[0111] where Homoscale is the depth of field coefficient of the image, and scale speed is the ratio of the first offset of the frame image to the time dimension offset, and scale offset is the ratio of the spatial dimension offset of the frame image to the offset of the preset standard plane camera.

[0112] After calculating the depth of field coefficient of each frame of image, the image can be scaled according to the depth of field coefficient, and then based on the time dimension offset and spatial dimension offset of the frame image, the position of the object information contained in the frame image on the imaging plane of the main camera can be determined. Furthermore, according to the positions of the object information contained in each frame of image on the imaging plane of the main camera, the object information about the target object contained in each image is fused to obtain the complete object information of the target object.

[0113] In the above solution of the embodiment of the present invention, since the overall picture content of each acquired image contains the target object, it means that the complete object information of the target object is contained in each acquired image. Further, the time dimension offset of each image is determined, which indicates the offset amount in the moving direction of the target object between each image and the adjacent image of the same camera, and the spatial dimension offset of each image is determined, which indicates the offset amount between each image and the image acquired by the adjacent camera at the same time. Thus, the object information contained in each image can be fused based on the time dimension offset and spatial dimension offset of each image to obtain the complete object information of the target object. It can be seen that by adopting the solution provided by the embodiment of the present invention, the complete object information of the target object can be automatically acquired, thereby improving the information acquisition efficiency of the target object.

[0114] As Figure 5 shown, the embodiment of the present invention provides an information fusion method, including steps S501 - S505, where:

[0115] S501, acquiring multiple frames of images continuously collected by each camera in a plurality of cameras for a target area during the movement of a target object in the target area; wherein, the overall picture content of each acquired image contains the target object;

[0116] Among them, this step is the same as or similar to step S201, and the specific implementation manner can be referred to step S201, which will not be elaborated in the embodiment of the present invention here.

[0117] S502. For each image, determine the position offset on the imaging plane of the main camera of the same object in this image and the first adjacent image of this image as the time dimension offset of this image.

[0118] Among them, the first adjacent image of each image is: the image adjacent to this image in terms of acquisition time among the multiple frames of images collected by the camera to which this image belongs; the main camera is one of the multiple cameras; this step is the same as or similar to step S202, and the specific implementation method can refer to step S202, which will not be elaborated in this embodiment of the present invention.

[0119] S503. Determine the mapped image of each image on the imaging plane of the main camera.

[0120] After determining the time dimension offset of each image, in order to improve the accuracy of fusion, each image can be first mapped to the imaging plane of the main camera, which is convenient for calculating the position offset on the imaging plane of the main camera of the same object in this image and its second adjacent image.

[0121] Optionally, in one implementation, for each image, calculate the ratio of the second offset of this image to the time dimension offset of this image, and scale this image according to the ratio to obtain a scaled image as the mapped image of this image on the imaging plane of the main camera.

[0122] Among them, the second offset of each image is: the position offset on the imaging plane of the camera to which this image belongs of the same object in this image and the first adjacent image of this image. Optionally, first determine the same object included in this image and the first adjacent image of this image as the target object, and then calculate the difference between the position of the target object in this image and the position of the target object in the first adjacent image of this image as the position offset on the imaging plane of the main camera of the same object in this image and the first adjacent image of this image. The specific implementation process is the same as the calculation process of the first offset involved in the above embodiments of the present invention. The specific calculation method can refer to the process of calculating the first offset described above and will not be elaborated here.

[0123] Since the time dimension offset of each image reflects the position offset on the imaging plane of the main camera, and the above second offset is the position offset on the imaging plane of the camera to which this image belongs, the ratio of the second offset of this image to the time dimension offset of this image is the mapping coefficient for mapping this image to the imaging plane of the main camera. Therefore, after calculating the ratio of the second offset of this image to the time dimension offset of this image, this image can be scaled according to the calculated ratio to obtain a scaled image as the mapped image of this image on the imaging plane of the main camera.

[0124] If the image size of the scaled image is larger than that of the original image, the scaled image can be cropped to obtain a scaled image with the same size as the original image. If the image size of the scaled image is smaller than that of the original image, pixel filling can be performed on the periphery of the scaled image to make the size of the scaled image the same as that of the original image.

[0125] S504. For each image, calculate the position offset on the imaging plane of the main camera between the mapped image of this image and the mapped images of the second adjacent images of this image for the same object, as the spatial dimension offset of this image.

[0126] After mapping each image to the imaging plane of the main camera, the position offset on the imaging plane of the main camera between the mapped image of this image and the mapped images of the second adjacent images of this image for the same object can be further calculated as the spatial dimension offset of this image. For the specific calculation process, reference can be made to the calculation of the time dimension offset described above, which will not be elaborated here.

[0127] S505. Based on the time dimension offset and spatial dimension offset of each image, fuse the object information about the target object included in each image to obtain the complete object information of the target object.

[0128] Among them, this step is the same as or similar to step S204. For the specific implementation method, reference can be made to step S204, which will not be elaborated in the embodiments of the present invention here.

[0129] In the above solution of the embodiments of the present invention, the complete object information of the target object can be automatically obtained, thereby improving the information acquisition efficiency of the target object. Further, by mapping each image to the imaging plane of the main camera, and then calculating the spatial dimension offset of each image through the mapped images of each image, it provides an implementation basis for automatically obtaining the complete object information of the target object and further improving the information acquisition efficiency of the target object.

[0130] Based on Figure 2 the embodiments shown, as Figure 6 shown, the embodiments of the present invention provide an information fusion method. The above S204 may include steps S204A - S204C:

[0131] S204A. Based on the spatial dimension offset of each image collected by each camera, determine the homography matrix between the imaging plane of each camera and the imaging plane of the main camera as the homography matrix of each camera.

[0132] Among them, the homography matrix of each camera indicates the mapping relationship between the imaging plane of this camera and the splicing area in the imaging plane of the main camera; the splicing area is the area used for splicing the images collected by each camera at the same acquisition moment.

[0133] Among them, the above splicing area may include the area mapped on the imaging plane of the main camera of the images collected by the central camera in each camera. Optionally, the above splicing area may also be determined in combination with the size of the images collected by the camera and the preset size of the fused image.

[0134] For each of the images collected at the same acquisition moment, for each image among its images, based on the spatial dimension offset of the image and the spatial position offsets of other images in the images, the offset between the image and the image collected by the central camera can be determined. Furthermore, based on this offset, the offset amount for mapping each pixel point in the image to the area where the image collected by the central camera is located can be determined as the corresponding offset coefficient for each pixel point, so as to determine the homography matrix of the image.

[0135] S204B, for each image, based on the homography matrix of the camera to which the image belongs, determine the first position information of the object information included in the image in the splicing area, and based on the time dimension offset and the first position information of the image, determine the second position information of the object information included in the image on the imaging plane of the main camera after fusion;

[0136] After determining the homography matrix of the camera to which the image belongs, the position information of the object information included in the image can be determined first, and then based on the unit matrix, the first position information of the object information included in the image in the splicing area can be determined. Then, using the time dimension offset of the image, the second position information of the object information included in the image on the imaging plane of the main camera after fusion can be determined.

[0137] S204C, based on the second position information of the object information included in each image, fuse the object information about the target object included in each image to obtain the complete object information of the target object.

[0138] Optionally, in one implementation manner, the object information about the target object included in each image can be spliced on the imaging plane of the main camera according to the second position information of the object information included in each image to obtain the complete object information of the target object.

[0139] In the above solution of the embodiment of the present invention, the complete object information of the target object can be automatically obtained, thereby improving the information acquisition efficiency of the target object. Further, by fusing the object information included in each image through the homography matrix of each camera, it provides an implementation basis for automatically obtaining the complete object information of the target object and further improving the information acquisition efficiency of the target object.

[0140] Optionally, in another embodiment of the present invention, the main camera may be: the camera among the cameras that has the largest number of common images with other cameras; wherein, the existence of a common image between two cameras means that the two cameras simultaneously capture valid images at the same acquisition moment, and the valid image is an image whose time - dimension offset satisfies a preset condition.

[0141] As shown in Table 2 below:

[0142] Table 2

[0143] T1 T2 T3 T4 Camera 1 Image 12 Image 12 Camera 2 Image 21 Image 22 Camera 3 Image 31 Image 32 Image 33 Image 34

[0144] In Table 2 above, Images 12, 12, 21, 22, 31, 32, 33, and 34 are valid images. Then the number of common images between Camera 1 and other cameras is 2 + 1 = 3, the number of common images between Camera 2 and other cameras is 1 + 2 = 3, and the number of common images between Camera 3 and other cameras is 1 + 2 + 1 = 4. Therefore, Camera 3 is the main camera.

[0145] In order to determine the main camera with the largest number of common images with other cameras, based on the Figure 2 provided embodiment, as Figure 7 shown, the embodiment of the present invention also provides another information fusion method, which may further include steps S701 - S709:

[0146] S701, determine a target camera from the multiple cameras;

[0147] In this step, a target camera can be randomly determined from the multiple cameras. Or, when the clustering scores of the existing cameras have been determined, a target camera is randomly selected from the remaining cameras for which the clustering scores have not been determined.

[0148] S702, determine a candidate camera with the largest number of common images with the clustering camera from each first camera;

[0149] Wherein, the first camera is a camera among the multiple cameras except the clustering camera, and the clustering camera includes the target camera. In the first clustering, the clustering camera is the target camera. At this time, the first camera is a camera among the multiple cameras except the target camera. After subsequent loop - adding clustering cameras, the number of clustering cameras gradually increases, and the number of first cameras continuously decreases.

[0150] S703, determine whether the ratio of the time - dimension offset of the candidate camera to the time - dimension offset of the clustering camera is within a specified ratio range; if it is, execute step S704; if not, execute step S705;

[0151] In this step, the above-specified ratio range can be determined according to needs and actual requirements. For example, it can be 0.9 to 1.1. If the ratio of the time dimension offset of the candidate camera to the time dimension offset of the clustering camera is within the specified ratio range, then step S704 is executed. If the ratio of the time dimension offset of the candidate camera to the time dimension offset of the clustering camera is not within the specified ratio range, then step S705 is executed.

[0152] S704. Add the candidate camera to the clustering camera.

[0153] In this step, when the ratio of the time dimension offset of the candidate camera to the time dimension offset of the clustering camera is within the specified ratio range, it indicates that the images captured by the candidate camera are normal. At this time, the candidate camera can be added to the clustering camera.

[0154] S705. Remove the candidate camera from the first cameras.

[0155] In this step, when the ratio of the time dimension offset of the candidate camera to the time dimension offset of the clustering camera is not within the specified ratio range, it can be explained that the images captured by the candidate camera are abnormal. At this time, the candidate camera can be removed from the first cameras.

[0156] S706. Determine whether the number of first cameras is zero. If it is zero, then step S707 is executed. If it is not zero, then return to execute step S702.

[0157] In this step, if the number of first cameras is zero, then step S707 is executed. If the number of first cameras is not zero, then step S702 is executed.

[0158] S707. Determine the clustering score of the clustering camera as the clustering score corresponding to the target camera.

[0159] Among them, the clustering score of the clustering camera is: the sum of the number of common images of each camera in the clustering camera with other cameras.

[0160] Exemplarily, the clustering camera includes camera 2 and camera 3, and the number of common images of camera 2 with other cameras is 3, and the number of common images of camera 3 with other cameras is 4. Then the clustering score corresponding to the target camera is 7.

[0161] S708. Determine whether the clustering score has been determined for each camera among the multiple cameras. If so, then step S709 is executed. If not, then return to execute step S701.

[0162] In this step, if each of the multiple cameras determines a clustering score, step S709 is executed. If there are still cameras among the multiple cameras that have not determined a clustering score, step S701 is executed.

[0163] S709. Use the camera with the maximum clustering score as the main camera.

[0164] Optionally, the maximum clustering score and the corresponding target camera are updated in real time. Then, after each of the multiple cameras determines a clustering score, the maximum clustering score and the corresponding target camera can be output.

[0165] In the above solution of the embodiment of the present invention, the complete object information of the target object can be automatically obtained, thereby improving the information acquisition efficiency of the target object. Optionally, by determining whether the ratio of the time dimension offset of the candidate camera to the time dimension offset of the clustering camera is within the specified ratio range, the camera with abnormal images can be identified, thereby further improving the accuracy of subsequent fusion.

[0166] Corresponding to the information fusion method provided in the above embodiment of the present invention, as Figure 8 shown, the embodiment of the present invention further provides an information fusion device, including:

[0167] An image acquisition module 801, configured to acquire multiple frames of images continuously collected by each of the multiple cameras for the target area during the movement of the target object in the target area; wherein, the overall picture content of each acquired image includes the target object;

[0168] A first offset determination module 802, configured to, for each image, determine the position offset of the same object in the image and its first adjacent image on the imaging plane of the main camera as the time dimension offset of the image; wherein, the first adjacent image of each image is: among the multiple frames of images collected by the camera to which the image belongs, the image whose acquisition time is adjacent to that of the image; the main camera is one of the multiple cameras;

[0169] A second offset determination module 803, configured to, for each image, determine the position offset of the same object in the image and its second adjacent image on the imaging plane of the main camera as the space dimension offset of the image; wherein, the second adjacent image of each image is: the image collected by the camera adjacent to the camera to which the image belongs at the acquisition time of the image;

[0170] An information fusion module 804, configured to fuse the object information about the target object included in each image based on the time dimension offset and the space dimension offset of each image to obtain the complete object information of the target object.

[0171] Optionally, the first offset determination module includes:

[0172] A first offset determination sub-module, configured to determine the position offset of the same object in the image and the first adjacent image of the image on the imaging plane of the camera to which the image belongs as the first offset of the image;

[0173] A ratio calculation sub-module, configured to calculate the ratio of the first average offset corresponding to the main camera to the first average offset corresponding to the specified camera as the first ratio; wherein, the specified camera is the camera to which the image belongs, and the first average offset corresponding to each camera is: the average value of the first offsets of the images collected by the camera;

[0174] An offset calculation sub-module, configured to calculate the product of the first offset and the first ratio as the position offset of the same object in the image and the first adjacent image of the image on the imaging plane of the main camera.

[0175] Optionally, the first offset determination sub-module is specifically configured to determine the same object included in the image and the first adjacent image of the image as the target object;

[0176] Calculate the difference between the position of the target object in the image and the position of the target object in the first adjacent image of the image as the position offset of the same object in the image and the first adjacent image of the image on the imaging plane of the main camera.

[0177] Optionally, the apparatus further includes:

[0178] A mapping module, configured to determine the mapped image of each image on the imaging plane of the main camera before the second offset determination module executes, for each image, determining the position offset of the same object in the image and the second adjacent image of the image on the imaging plane of the main camera as the spatial dimension offset of the image;

[0179] The second offset determination module is specifically configured to calculate the position offset of the same object in the mapped image of the image and the mapped image of the second adjacent image of the image on the imaging plane of the main camera as the spatial dimension offset of the image.

[0180] Optionally, the mapping module is specifically configured to, for each image, calculate the ratio of the second offset of the image to the time dimension offset of the image, and scale the image according to the ratio to obtain a scaled image as the mapped image of the image on the imaging plane of the main camera; wherein, the second offset of each image is: the position offset of the same object in the image and the first adjacent image of the image on the imaging plane of the camera to which the image belongs.

[0181] Optionally, the information fusion module includes:

[0182] A homography matrix determination sub-module, configured to determine a homography matrix between the imaging plane of each camera and the imaging plane of the main camera based on the spatial dimension offset of each image collected by each camera, as the homography matrix of each camera; wherein, the homography matrix of each camera indicates: the mapping relationship between the imaging plane of this camera and the splicing area in the imaging plane of the main camera; the splicing area is: the area used for splicing the images collected by each camera at the same acquisition moment;

[0183] A position information determination sub-module, configured to, for each image, determine the first position information of the object information included in this image in the splicing area based on the homography matrix of the camera to which this image belongs, and determine the second position information of the object information included in this image on the imaging plane of the main camera after fusion based on the time dimension offset of this image and the first position information;

[0184] An information fusion sub-module, configured to fuse the object information about the target object included in each image based on the second position information of the object information included in each image, to obtain the complete object information of the target object.

[0185] Optionally, the information fusion sub-module is specifically configured to splice the object information about the target object included in each image on the imaging plane of the main camera according to the second position information of the object information included in each image, to obtain the complete object information of the target object.

[0186] Optionally, the main camera is: the camera among the cameras that has the largest number of common images with other cameras; there is a common image between two cameras, which means: the two cameras simultaneously collect valid images at the same acquisition moment, and the valid image is: an image whose time dimension offset meets a preset condition.

[0187] Optionally, the device further includes:

[0188] The main camera determination module is used to determine a target camera from the multiple cameras; determine, from each of the first cameras, a candidate camera that has the largest number of common images with the clustering cameras; if the ratio of the time - dimension offset of the candidate camera to the time - dimension offset of the clustering camera is within the specified ratio range, add the candidate camera to the clustering cameras; otherwise, remove the candidate camera from the first cameras; return to execute the step of determining, from each of the first cameras, a candidate camera that has the largest number of common images with the clustering cameras until the number of the first cameras is zero; determine the clustering score of the clustering cameras as the clustering score corresponding to the target camera; wherein, the clustering score of the clustering cameras is: the sum of the number of common images between each camera in the clustering cameras and other cameras; return to execute the step of determining a target camera from the multiple cameras until the clustering score of each camera in the multiple cameras is determined; take the camera with the largest clustering score as the main camera; wherein, the first cameras are the cameras among the multiple cameras except the clustering cameras, and the clustering cameras include the target camera.

[0189] In the above - mentioned solution of the embodiment of the present invention, since the overall picture content of each acquired image includes the target object, it means that the complete object information of the target object is included in each acquired image. Further, determine the time - dimension offset of each image, which indicates the offset amount in the moving direction of the target object between each image and the adjacent image of the same camera, and determine the space - dimension offset of each image, which indicates the offset amount between each image and the image acquired by the adjacent camera at the same acquisition moment. Thus, the object information included in each image can be fused based on the time - dimension offset and space - dimension offset of each image to obtain the complete object information of the target object. It can be seen that by adopting the solution provided by the embodiment of the present invention, the complete object information of the target object can be automatically obtained, thereby improving the information acquisition efficiency of the target object.

[0190] The embodiment of the present invention also provides an electronic device, as Figure 9 shown, including a processor 901, a communication interface 902, a memory 903, and a communication bus 904. Among them, the processor 901, the communication interface 902, and the memory 903 complete communication with each other through the communication bus 904.

[0191] The memory 903 is used to store a computer program.

[0192] When the processor 901 is used to execute the program stored in the memory 903, the following steps are implemented:

[0193] Acquire multiple frames of images continuously collected by each camera in multiple cameras for a target area during the movement of a target object in the target area; wherein, the overall picture content of each acquired image includes the target object.

[0194] For each image, determine the position offset on the imaging plane of the main camera of the same object in the image and the first adjacent image of the image as the time dimension offset of the image; wherein, the first adjacent image of each image is: among the multiple frames of images collected by the camera to which the image belongs, the image whose acquisition time is adjacent to that of the image; the main camera is one of the multiple cameras.

[0195] For each image, determine the position offset on the imaging plane of the main camera of the same object in the image and the second adjacent image of the image as the spatial dimension offset of the image; wherein, the second adjacent image of each image is: the image collected by the camera adjacent to the camera to which the image belongs at the acquisition time of the image.

[0196] Based on the time dimension offset and spatial dimension offset of each image, fuse the object information about the target object included in each image to obtain the complete object information of the target object.

[0197] In the electronic device provided in the embodiment of the present invention, since the overall picture content of each acquired image includes the target object, it means that the complete object information of the target object is included in each acquired image. Further, determine the time dimension offset of each image, which indicates the offset amount in the moving direction of the target object between each image and the adjacent image of the same camera, and determine the spatial dimension offset of each image, which indicates the offset amount between each image and the image collected by the adjacent camera at the same acquisition time, so that the object information included in each image can be fused based on the time dimension offset and spatial dimension offset of each image to obtain the complete object information of the target object. It can be seen that by adopting the solution provided in the embodiment of the present invention, the complete object information of the target object can be automatically acquired, thereby improving the information acquisition efficiency of the target object.

[0198] The communication bus mentioned in the above electronic device may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity, only a thick line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus.

[0199] The communication interface is used for communication between the above electronic device and other devices.

[0200] The memory may include a Random Access Memory (RAM), or may also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0201] The aforementioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0202] In another embodiment provided by the present invention, there is also provided a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the steps of any of the above information fusion methods are implemented.

[0203] In another embodiment provided by the present invention, there is also provided a computer program product containing instructions, which when run on a computer, causes the computer to execute any of the information fusion methods in the above embodiments.

[0204] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that the computer can access or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)).

[0205] It should be noted that, in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise", or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device that includes a series of elements includes not only those elements but also other elements that are not explicitly listed, or also includes elements that are inherent to such process, method, article, or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article, or device that includes the element.

[0206] Each embodiment in this specification is described in a related manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment.

[0207] The above are only the preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention are all included within the protection scope of the present invention.

Claims

1. An information fusion method, characterized in that, The method includes: Obtaining multiple frames of images continuously collected by each camera among multiple cameras for the target area during the movement of the target object in the target area; wherein, the overall picture content of each obtained image includes the target object; For each image, determining the position offset on the imaging plane of the main camera of the same object in this image and its first adjacent image as the time - dimension offset of this image; wherein, the first adjacent image of each image is: among the multiple frames of images collected by the camera to which this image belongs, the image whose acquisition time is adjacent to this image; the main camera is one of the multiple cameras; For each image, determining the position offset on the imaging plane of the main camera of the same object in this image and its second adjacent image as the space - dimension offset of this image; wherein, the second adjacent image of each image is: the image collected by the camera adjacent to the camera to which this image belongs at the acquisition time of this image; Based on the time - dimension offset and space - dimension offset of each image, fusing the object information about the target object included in each image to obtain the complete object information of the target object.

2. The method according to claim 1, wherein The determining the position offset on the imaging plane of the main camera of the same object in this image and its first adjacent image as the time - dimension offset of this image includes: Determining the position offset on the imaging plane of the camera to which this image belongs of the same object in this image and its first adjacent image as the first offset of this image; Calculating the ratio of the first average offset corresponding to the main camera to the first average offset corresponding to the designated camera as the first ratio; wherein, the designated camera is the camera to which this image belongs, and the first average offset corresponding to each camera is: the mean value of the first offsets of each image collected by this camera; Calculating the product of the first offset and the first ratio as the position offset on the imaging plane of the main camera of the same object in this image and its first adjacent image.

3. The method according to claim 2, wherein The determining the position offset on the imaging plane of the camera to which this image belongs of the same object in this image and its first adjacent image as the first offset of this image includes: Determining the same object included in this image and its first adjacent image as the target object; Calculating the difference between the position of the target object in this image and the position of the target object in the first adjacent image of this image as the first offset of this image.

4. The method according to any one of claims 1 to 3, characterized in that, Before the step of, for each image, determining the position offset on the imaging plane of the main camera of the same object in this image and its second adjacent image as the space - dimension offset of this image, the method further includes: Determining the mapped image of each image on the imaging plane of the main camera; The determining the position offset on the imaging plane of the main camera of the same object in this image and its second adjacent image as the space - dimension offset of this image includes: Calculating the position offset on the imaging plane of the main camera of the same object in the mapped image of this image and the mapped image of its second adjacent image as the space - dimension offset of this image.

5. The method according to claim 4, characterized in that, The determination of the mapped image of each image on the imaging plane of the main camera includes: For each image, calculate the ratio of the second offset of the image to the time - dimension offset of the image, and scale the image according to the ratio to obtain a scaled image, which is used as the mapped image of the image on the imaging plane of the main camera; Among them, the second offset of each image is: the position offset of the same object in the image and its first adjacent image on the imaging plane of the camera to which the image belongs.

6. The method according to any one of claims 1 to 3, characterized in that, The fusion of the object information about the target object included in the images based on the time - dimension offset and the space - dimension offset of the images to obtain the complete object information of the target object includes: Based on the space - dimension offset of each image collected by each camera, determine the homography matrix between the imaging plane of each camera and the imaging plane of the main camera as the homography matrix of each camera; among them, the homography matrix of each camera indicates: the mapping relationship between the imaging plane of the camera and the stitching area in the imaging plane of the main camera; the stitching area is: the area used for stitching the images collected by each camera at the same acquisition moment; For each image, based on the homography matrix of the camera to which the image belongs, determine the first position information of the object information included in the image in the stitching area, and based on the time - dimension offset of the image and the first position information, determine the second position information of the object information included in the image on the imaging plane of the main camera after fusion; Based on the second position information of the object information included in the images, fuse the object information about the target object included in the images to obtain the complete object information of the target object.

7. The method according to claim 6, wherein The fusion of the object information about the target object included in the images based on the second position information of the object information included in the images to obtain the complete object information of the target object includes: According to the second position information of the object information included in the images, stitch the object information about the target object included in the images on the imaging plane of the main camera to obtain the complete object information of the target object.

8. The method according to any one of claims 1 to 3, characterized in that The main camera is: the camera among the cameras that has the largest number of common images with other cameras; the existence of a common image between two cameras means: the two cameras simultaneously collect valid images at the same acquisition moment, and the valid image is: an image whose time - dimension offset meets the preset conditions.

9. The method according to claim 8, wherein The main camera is determined from the multiple cameras in the following way, including: Determine a target camera from the multiple cameras; Among the first cameras, determine the candidate camera with the largest number of common images with the clustering cameras; where the first cameras are the cameras among the multiple cameras except the clustering cameras, and the clustering cameras include the target camera; If the ratio of the time - dimension offset of the candidate camera to the time - dimension offset of the clustering camera is within the specified ratio range, add the candidate camera to the clustering cameras; otherwise, remove the candidate camera from the first cameras. Return to execute the step of determining, from each of the first cameras, the candidate camera with the largest number of common images with the clustering camera until the number of the first cameras is zero; Determine the clustering score of the clustering camera as the clustering score corresponding to the target camera; wherein, the clustering score of the clustering camera is the sum of the number of common images of each camera in the clustering camera with other cameras; Return to execute the step of determining a target camera from the multiple cameras until the clustering score of each camera in the multiple cameras is determined; Take the camera with the largest clustering score as the main camera.

10. An information fusion device, characterized in that, The device includes: An image acquisition module, configured to acquire multiple frames of images continuously collected by each of the multiple cameras for the target area during the movement of the target object in the target area; wherein, the overall picture content of each acquired image includes the target object; A first offset determination module, configured to, for each image, determine the position offset on the imaging plane of the main camera of the same object in this image and the first adjacent image of this image as the time dimension offset of this image; wherein, the first adjacent image of each image is the image adjacent to this image in terms of acquisition time among the multiple frames of images collected by the camera to which this image belongs; the main camera is one of the multiple cameras; A second offset determination module, configured to, for each image, determine the position offset on the imaging plane of the main camera of the same object in this image and the second adjacent image of this image as the space dimension offset of this image; wherein, the second adjacent image of each image is the image collected by the camera adjacent to the camera to which this image belongs at the acquisition time of this image; An information fusion module, configured to fuse the object information about the target object included in the respective images based on the time dimension offset and the space dimension offset of the respective images to obtain the complete object information of the target object.

11. An electronic device, characterized in that, Including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus; The memory is used for storing a computer program; The processor, when executing the program stored on the memory, implements the method steps described in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the method steps described in any one of claims 1-9 are implemented.

Images