Double-video target fusion method based on holographic intersection scene

Abstract

The invention discloses a double-video target fusion method based on a holographic intersection scene, and relates to the technical field of video fusion, and the method comprises the steps: determining at least one vehicle 2D target detection frame in a to-be-shot region and a detection frame bottom edge center pixel coordinate corresponding to each vehicle 2D target detection frame; mapping the vehicle 2D target detection frame corresponding to any camera into an image scene collected by another camera at the same moment through a pixel mapping relation and a detection frame bottom edge center pixel coordinate corresponding to each vehicle 2D target detection frame; and carrying out detection frame fusion on the vehicle 2D target detection frames corresponding to the same target vehicle by taking the corresponding vehicle 2D target detection frame in the image scene acquired by the other camera at the same moment as a main body. According to the method, each camera can be effectively assisted to determine more accurate detection frame bottom edge center pixel coordinates corresponding to the vehicle 2D target detection frame, so that the accuracy of subsequent fusion processing is greatly improved.

Description

Background Art

[0002] With the rapid growth of the number of motor vehicles in cities, the congestion problem at intersections during peak hours in the morning and evening has become one of the more difficult problems to solve. The newly built urban intersections have a wider single-direction road surface. It is usually difficult to cover more than three lanes in the same direction with one camera. In order to achieve full coverage of the intersection, it is necessary to install cameras on the left and right sides of an electric police pole. Using the intelligent traffic holographic twin technology, the dynamic scene of the intersection can be reproduced in real time, which is convenient for traffic management personnel to timely control the traffic situation at the intersection, and can effectively alleviate the congestion problem at the intersection during peak hours. The existing technology usually splices the video first and then detects the target. When the video dynamic splicing is good, the detected target does not need target fusion, which reduces the difficulty of target fusion. As the cameras at the intersection are installed more and more densely, the network generally has a certain degree of jamming, and video splicing anomalies usually occur, which increases the difficulty of target detection. The existing methods have high requirements for real-time network transmission. Summary of the invention

[0003] The technical problem to be solved by the present invention is to address the deficiencies of the prior art and specifically provide a dual-video target fusion method based on a holographic intersection scene, which is as follows:

[0004] 1) In the first aspect, the present invention provides a dual-video target fusion method based on a holographic intersection scene, and the specific technical solution is as follows:

[0005] Determine at least one base point in the camera scene of each camera of the two cameras for the area to be photographed and the latitude and longitude coordinates corresponding to each base point, determine at least one common base point in the common scene in the camera scenes of the two cameras and the pixel coordinates of any common base point in the camera scene corresponding to each camera;

[0006] Decode the video streams collected by the two cameras for the area to be photographed in real time, and determine at least one vehicle 2D target detection frame in the area to be photographed and the center pixel coordinates of the bottom edge of the detection frame corresponding to each vehicle 2D target detection frame according to the decoding results, the longitude and latitude coordinates corresponding to each base point, and the pixel coordinates corresponding to each common base point;

[0007] Through the pixel mapping relationship and the bottom center pixel coordinates of the detection frame corresponding to each vehicle 2D target detection frame, the vehicle 2D target detection frame corresponding to any camera is mapped to the image scene collected by another camera at the same time, and the vehicle 2D target detection frame corresponding to the same target vehicle is mainly fused with the corresponding vehicle 2D target detection frame in the image scene collected by the other camera at the same time.

[0008] The beneficial effects of the dual video target fusion method based on the holographic intersection scene provided by the present invention are as follows:

[0009] Through this solution, the targets in the dual video scenes are fused. In addition, the common base point of the two cameras in the common area to be shot is determined, which can effectively assist each camera to determine a more accurate coordinate of the center pixel of the bottom edge of the detection frame corresponding to the vehicle 2D target detection frame, thereby greatly improving the accuracy of subsequent fusion processing, so that the fusion result can greatly improve the twin display effect of the holographic intersection, timely control the traffic situation at the intersection, and effectively alleviate the congestion problem at the intersection during peak hours.

[0010] Based on the above solution, the present invention can also be improved as follows.

[0011] Further, the same target vehicle is:

[0012] Mapping all vehicle 2D object detection frames corresponding to any one of the cameras to an image scene captured by another camera at the same time, and determining whether any vehicle 2D object detection frame in the image scene captured by the other camera at the same time has an intersecting vehicle 2D object detection frame;

[0013] If so, determine whether the intersecting area exceeds the preset area. When the intersecting area exceeds the preset area, determine that the target vehicle corresponding to any vehicle 2D target detection frame and the intersecting vehicle 2D target detection frame is the same target vehicle.

[0014] Furthermore, it also includes:

[0015] Determine a right turn area in a camera scene for each of the two cameras when targeting the area to be photographed;

[0016] When any vehicle 2D target detection frame in the image scene captured by the other camera at the same time is in the right turn area, the vehicle 2D target detection frame corresponding to the same target vehicle is fused with the vehicle 2D target detection frame corresponding to any camera as the main detection frame.

[0017] Furthermore, it also includes:

[0018] Determine a right turn area in a camera scene for each of the two cameras when targeting the area to be photographed;

[0019] When any vehicle 2D target detection frame in the image scene captured by the other camera at the same time is in the right turn area, the vehicle 2D target detection frame corresponding to the same target vehicle is fused with the vehicle 2D target detection frame corresponding to any camera as the main detection frame.

[0020] 2) In the second aspect, the present invention also provides a dual-video target fusion system based on a holographic intersection scene, and the specific technical solution is as follows:

[0021] The determination module is used to: determine at least one base point in the camera scene of each camera of the two cameras for the area to be photographed and the latitude and longitude coordinates corresponding to each base point, determine at least one common base point in the common scene in the camera scenes of the two cameras and the pixel coordinates of any common base point in the camera scene corresponding to each camera;

[0022] The detection module is used to: decode the video streams collected by the two cameras for the area to be photographed in real time, and determine at least one vehicle 2D target detection frame in the area to be photographed and the pixel coordinates of the bottom edge center of the detection frame corresponding to each vehicle 2D target detection frame according to the decoding processing results, the latitude and longitude coordinates corresponding to each base point, and the pixel coordinates corresponding to each common base point;

[0023] The fusion module is used to: map the vehicle 2D target detection frame corresponding to any camera to the image scene collected by another camera at the same time through the pixel mapping relationship and the bottom center pixel coordinates of the detection frame corresponding to each vehicle 2D target detection frame, and fuse the vehicle 2D target detection frame corresponding to the same target vehicle with the corresponding vehicle 2D target detection frame in the image scene collected by the other camera at the same time as the main detection frame.

[0024] Based on the above solution, the present invention can also be improved as follows.

[0025] Further, the same target vehicle is:

[0026] Mapping all vehicle 2D object detection frames corresponding to any one of the cameras to an image scene captured by another camera at the same time, and determining whether any vehicle 2D object detection frame in the image scene captured by the other camera at the same time has an intersecting vehicle 2D object detection frame;

[0027] If so, determine whether the intersecting area exceeds the preset area. When the intersecting area exceeds the preset area, determine that the target vehicle corresponding to any vehicle 2D target detection frame and the intersecting vehicle 2D target detection frame is the same target vehicle.

[0028] Furthermore, it also includes:

[0029] The adjustment module is used to: determine a right turn area in a camera scene of each camera of the two cameras when the camera is directed to the area to be photographed;

[0030] When any vehicle 2D target detection frame in the image scene captured by the other camera at the same time is in the right turn area, the vehicle 2D target detection frame corresponding to the same target vehicle is fused with the vehicle 2D target detection frame corresponding to any camera as the main detection frame.

[0031] Furthermore, it also includes:

[0032] The processing module is used for: if it does not exist, treating the vehicle 2D target detection frame corresponding to any camera without the same target vehicle as an independent detection frame without performing fusion processing.

[0033] 3) In a third aspect, the present invention further provides an electronic device, comprising a processor, wherein the processor is coupled to a memory, wherein at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor so that the electronic device implements any of the above methods.

[0034] 4) In a fourth aspect, the present invention further provides a computer-readable storage medium, wherein at least one computer program is stored in the computer-readable storage medium, and the at least one computer program is loaded and executed by a processor so that a computer implements any of the above methods.

[0035] It should be noted that the beneficial effects achieved by the technical solutions of the second to fourth aspects of the present invention and the corresponding possible implementation methods can be found in the above-mentioned technical effects of the first aspect and its corresponding possible implementation methods, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0037] Figure 1 A schematic diagram of a flow chart of a dual-video target fusion method based on a holographic intersection scene according to an embodiment of the present invention;

[0038] Figure 2 The figure is a structural framework diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0040] like Figure 1 As shown, a dual-video target fusion method based on a holographic intersection scene according to an embodiment of the present invention comprises the following steps:

[0041] S1, determining at least one base point in the camera scene of each camera of the two cameras for the area to be photographed and the latitude and longitude coordinates corresponding to each base point, and determining at least one common base point in the common scene in the camera scenes of the two cameras and the pixel coordinates of any common base point in the camera scene corresponding to each camera;

[0042] S2, decoding the video streams collected by the two cameras for the area to be photographed in real time, and determining at least one vehicle 2D target detection frame in the area to be photographed and the bottom edge center pixel coordinates of the detection frame corresponding to each vehicle 2D target detection frame according to the decoding results, the longitude and latitude coordinates corresponding to each base point, and the pixel coordinates corresponding to each common base point;

[0043] S3, through the pixel mapping relationship and the bottom center pixel coordinates of the detection frame corresponding to each vehicle 2D target detection frame, the vehicle 2D target detection frame corresponding to any camera is mapped to the image scene collected by another camera at the same time, and the vehicle 2D target detection frame corresponding to the same target vehicle is mainly fused with the corresponding vehicle 2D target detection frame in the image scene collected by the other camera at the same time.

[0044] The beneficial effects of the dual video target fusion method based on the holographic intersection scene provided by the present invention are as follows:

[0045] Through this solution, the targets in the dual video scenes are fused. In addition, the common base point of the two cameras in the common area to be shot is determined, which can effectively assist each camera to determine a more accurate coordinate of the center pixel of the bottom edge of the detection frame corresponding to the vehicle 2D target detection frame, thereby greatly improving the accuracy of subsequent fusion processing, so that the fusion result can greatly improve the twin display effect of the holographic intersection, timely control the traffic situation at the intersection, and effectively alleviate the congestion problem at the intersection during peak hours.

[0046] S1, determining at least one base point in the camera scene of each camera of the two cameras for the area to be photographed and the latitude and longitude coordinates corresponding to each base point, and determining at least one common base point in the common scene of the camera scenes of the two cameras and the pixel coordinates of any common base point in the camera scene corresponding to each camera. Wherein:

[0047] The area to be photographed refers to any traffic intersection or any road section where the traffic volume exceeds the preset amount. The preset amount is set according to actual needs or the processing capacity of the processor, and no specific amount is limited here.

[0048] Before determining the common scene in the camera scenes of the two cameras, the camera scenes collected by the two cameras need to be preprocessed, including:

[0049] Camera calibration: This step involves determining the internal and external parameters of the camera to ensure that images taken from different angles can be accurately stitched together. Based on the depth information of the target object and the internal parameters of each camera device, the object point cloud acquired by each camera device is determined, and then the position and angle of the camera device are adjusted so that the images of each camera device can be correctly aligned. The specific process is:

[0050] Determine the geometric model of camera imaging: First, in order to determine the relationship between the three-dimensional geometric position of a point on the surface of a spatial object and its corresponding point in the image, the geometric model of the camera must be established. These geometric model parameters include the internal parameters and external parameters of the camera. The internal parameters include focal length, principal point coordinates, distortion coefficient, etc., while the external parameters involve the position, posture and direction of the camera.

[0051] The specific process is: select the reference point corresponding to any camera, the number of reference points is not specifically limited, and determine the coordinate information of each reference point in the world coordinate system, camera coordinate system, image coordinate system and pixel coordinate system. The world coordinate system is used to describe the position of an object in the real world, the camera coordinate system takes the camera as the origin, the image coordinate system coincides with the imaging plane, and the pixel coordinate system takes the upper left corner of the image as the origin. Determine the rotation matrix R and the translation vector t to convert the coordinate information of the reference point in the world coordinate system to the corresponding target coordinate information in the camera coordinate system. Further, the target coordinate information is converted to the image coordinate system through the perspective projection relationship, completing the process of projecting the coordinate information corresponding to the reference point in the three-dimensional space onto the two-dimensional image plane. The coordinate information in the image coordinate system is converted to the pixel coordinate system through the position and size information of the photosensitive element in each camera. The conversion from the image coordinate system to the pixel coordinate system involves converting the points on the image plane into pixel coordinates in the digital image, taking into account the position and size of the photosensitive element.

[0052] 1) Selection of calibration board for camera calibration:

[0053] The types of calibration plates for camera calibration mainly include solid circle array patterns and chessboard patterns.

[0054] Among them, the solid circle array pattern, also called the equally spaced solid circle array pattern, has high stability and is easy to identify.

[0055] The chessboard pattern, which is characterized by black and white squares, is similar to a chess board. This pattern is also suitable for camera calibration because it can provide clear corners and edges for accurate measurement and calibration.

[0056] According to the shooting results of the camera to be calibrated for the experimental collection area, the calibration plate for camera calibration is determined, that is, according to the boundary definition feature information in the shooting results, the calibration plate for camera calibration is determined. When the boundary definition feature information is an irregular pattern, a calibration plate with a solid circle array pattern is selected, and when the boundary definition feature information is a regular pattern, a chessboard pattern is selected.

[0057] 2) Placement and shooting of calibration plate for camera calibration:

[0058] After taking multiple shots of the experimental collection area, extract the images obtained after each shot, and select the placement of the calibration plate by determining the clarity of the images corresponding to different batches, specifically:

[0059] Before each shooting of the experimental collection area, the calibration plate is moved a fixed distance in the specified direction, and the captured image is associated with the position of the calibration plate. The clarity of the corresponding image after each movement of the fixed plate is determined in batches, and the clarity of all batches of images is compared. The calibration plate position corresponding to the image with the highest clarity is used as the calibration plate placement position for camera calibration.

[0060] Another embodiment of this solution may also include:

[0061] After the calibration plate is positioned, image acquisition is performed again for the experimental acquisition area, feature recognition is performed on the acquired images, and the placement of the calibration plate is fine-tuned by comparing the recognition results with the manual annotation results.

[0062] 3) Extraction of corner points of calibration plate for camera calibration:

[0063] Through image processing technology (such as grayscale, binarization, contour detection, etc.), the corner point positions on the calibration plate are extracted.

[0064] According to the position of the corner points and the geometric model of camera imaging determined above, the position information of any pixel point in the captured image converted to the three-dimensional space can be further determined.

[0065] 4) Verification and adjustment: By capturing multiple valid images, calculating the relationship matrix, and verifying the calibration results. If the calibration is qualified, these parameters will be used in subsequent image processing and computer vision tasks, such as image reconstruction, 3D reconstruction, target tracking, etc.

[0066] 5) Optimization of camera calibration

[0067] Distortion correction: Since the camera uses the pinhole imaging principle, the lens will inevitably introduce radial distortion and tangential distortion during the imaging process. Therefore, it is an important step to correct the distortion of the camera to ensure that the image in reality can be truly presented in the picture, which is beneficial to subsequent tasks such as robotic arm grasping.

[0068] In one embodiment of the present scheme, in order to improve the accuracy of camera calibration, the basic parameters corresponding to different cameras and the subsequent processing targets for the camera acquisition area are first extracted, the recognition scores corresponding to the current two cameras are determined, and the parameter adjustment direction of the corresponding camera calibration is determined based on the recognition scores. The basic parameters include: internal parameters and distortion parameters of the camera, and the internal parameters include: focal length, pixel size, and the position of the optical axis relative to the image plane. The distortion parameters are used to describe the distortion phenomenon caused by the non-ideal optical system during the imaging process of the camera, such as radial distortion and tangential distortion. Radial distortion is the distortion caused by the different optical paths of the light rays from the center of the lens to the imaging plane at different radii, while tangential distortion is the distortion caused by the incomplete central symmetry of the lens. Subsequent processing targets include: target position determination or target type recognition.

[0069] The process of determining the corresponding recognition scores of two cameras is as follows:

[0070] A scoring system for the camera's internal parameters, distortion parameters, and processing targets is pre-constructed. According to the scoring system, a target recognition score corresponding to each camera is determined. In the correspondence between the recognition score and the parameter adjustment direction, a target parameter adjustment direction corresponding to the target recognition score is determined. Based on the target parameter adjustment direction corresponding to each camera, the camera calibration of the corresponding camera is adjusted. The parameter adjustment directions include but are not limited to: internal parameter downward adjustment, internal parameter upward adjustment, distortion parameter downward adjustment, and distortion parameter upward adjustment.

[0071] S2, decoding the video streams collected by the two cameras for the area to be photographed in real time, and determining at least one vehicle 2D target detection frame in the area to be photographed and the bottom edge center pixel coordinates of the detection frame corresponding to each vehicle 2D target detection frame according to the decoding results, the longitude and latitude coordinates corresponding to each base point, and the pixel coordinates corresponding to each common base point;

[0072] S3, through the pixel mapping relationship and the bottom center pixel coordinates of the detection frame corresponding to each vehicle 2D target detection frame, the vehicle 2D target detection frame corresponding to any camera is mapped to the image scene collected by another camera at the same time, and the vehicle 2D target detection frame corresponding to the same target vehicle is mainly fused with the corresponding vehicle 2D target detection frame in the image scene collected by the other camera at the same time.

[0073] Further, the same target vehicle is:

[0074] Mapping all vehicle 2D object detection frames corresponding to any one of the cameras to an image scene captured by another camera at the same time, and determining whether any vehicle 2D object detection frame in the image scene captured by the other camera at the same time has an intersecting vehicle 2D object detection frame;

[0075] If so, determine whether the intersecting area exceeds the preset area. When the intersecting area exceeds the preset area, determine that the target vehicle corresponding to any vehicle 2D target detection frame and the intersecting vehicle 2D target detection frame is the same target vehicle.

[0076] Furthermore, it also includes:

[0077] Determine a right turn area in a camera scene for each of the two cameras when targeting the area to be photographed;

[0078] When any vehicle 2D target detection frame in the image scene captured by the other camera at the same time is in the right turn area, the vehicle 2D target detection frame corresponding to the same target vehicle is fused with the vehicle 2D target detection frame corresponding to any camera as the main detection frame.

[0079] Furthermore, it also includes:

[0080] Determine a right turn area in a camera scene for each of the two cameras when targeting the area to be photographed;

[0081] When any vehicle 2D target detection frame in the image scene captured by the other camera at the same time is in the right turn area, the vehicle 2D target detection frame corresponding to the same target vehicle is fused with the vehicle 2D target detection frame corresponding to any camera as the main detection frame.

[0082] In another embodiment of the present invention, the implementation of the present invention includes the following steps:

[0083] Latitude and longitude coordinate calibration, select 4 position points from the left and right scenes respectively (the 4 position points are selected randomly), record their pixel coordinates and longitude and latitude coordinates; send them to the configuration files of each video detection program respectively; used to calculate the mapping of the detected target frame to the longitude and latitude coordinates; it should be noted that the left scene is the scene captured by the left camera placed side by side, and similarly the right camera is the scene captured by the right camera placed side by side.

[0084] Pixel coordinate calibration: select 4 common base points in the left and right scenes (the 4 points are randomly selected), record their corresponding pixel coordinates, and send them to the configuration file of the fusion program; during fusion, the right camera detection frame is mapped to the left scene through pixels through the mapping relationship for detection frame filtering;

[0085] After the detection algorithm program is started, the video of the two camera streams is decoded respectively, and the target in each frame is detected in real time. The tracking algorithm is used to ensure the continuity and uniqueness of the target. Through the algorithm mapping relationship, the center pixel of the bottom edge of the detection frame is mapped to the longitude and latitude coordinates. Due to the fluctuation of the detection frame, the mapped longitude and latitude coordinates need to be smoothed to a certain extent.

[0086] This description uses the post-fusion method. The two-way target detection results and longitude and latitude coordinates are sent to the fusion program. Through the pixel mapping relationship, each target detection frame of the right scene is mapped in the left scene. The matrix intersection relationship is used to determine whether they are the same target. Usually, if the intersection area is greater than the ratio of the area of ​​one of the detection frames (70%), they can be regarded as the same target.

[0087] Since the field of view of the left scene is wider, for the fused targets in the non-right-turn area, the longitude and latitude of the left camera target are mainly used during fusion filtering, which can effectively reduce the longitude and latitude coordinate fluctuations caused by the fluctuations of the detection frame; since the right-turn area has some overlapping boundaries in the two pictures, for the targets whose detection frames are in the right-turn area, the longitude and latitude coordinates calculated by the right camera target are mainly used as the longitude and latitude of the fused targets, which ensures a larger right-turn range while reducing the longitude and latitude coordinate fluctuations of right-turn vehicles and making the trajectory smoother; when determining whether the target detection frame is in the right-turn area, it is determined by whether the center point of the bottom edge of the detection frame is within the polygonal area; for target vehicles in the non-right-turn area, the detection frame determined by the left camera is taken as the fused detection frame, and the longitude and latitude of the target of the left camera are the longitude and latitude of the fused target; this method can ensure that each target has a unique and continuous target ID from entering the scene to leaving the scene.

[0088] It should be noted that the specific processing process based on the longitude and latitude of the target of a camera on one side is as follows: when fusion is performed, the target frame of the same car is present in both the left and right camera images. The target detection frame of the left camera is taken as the detection frame after fusion. If the center point of the bottom edge of the frame is located in the non-right turn area of ​​the left camera, the longitude and latitude mapped by the target on the left camera are taken as the longitude and latitude after fusion; otherwise, the longitude and latitude mapped by the target on the right camera are taken as the longitude and latitude after fusion.

[0089] If the target frame of the car is only in the left picture, the longitude and latitude of the target in the left picture can be used after fusion;

[0090] If the target frame of the car is only in the right picture, the longitude and latitude of the target in the right picture can be taken as the fused target.

[0091] After the fusion,the fusion program sends the target to the server at a fixed frame rate, and the,terminal or twin GIS platform displays the target information in real time.

[0092] In the above embodiments, although the steps are numbered S1, S2, etc., these are only specific embodiments given by the present invention. Those skilled in the art may adjust the execution order of S1, S2, etc. according to actual conditions, which is also within the protection scope of the present invention. It can be understood that in some embodiments, some or all of the above embodiments may be included.

[0093] The present invention also provides a dual-video target fusion system based on a holographic intersection scene, and the specific technical solution is as follows:

[0094] The determination module is used to: determine at least one base point in the camera scene of each camera of the two cameras for the area to be photographed and the latitude and longitude coordinates corresponding to each base point, determine at least one common base point in the common scene in the camera scenes of the two cameras and the pixel coordinates of any common base point in the camera scene corresponding to each camera;

[0095] The detection module is used to: decode the video streams collected by the two cameras for the area to be photographed in real time, and determine at least one vehicle 2D target detection frame in the area to be photographed and the pixel coordinates of the bottom edge center of the detection frame corresponding to each vehicle 2D target detection frame according to the decoding processing results, the latitude and longitude coordinates corresponding to each base point, and the pixel coordinates corresponding to each common base point;

[0096] The fusion module is used to: map the vehicle 2D target detection frame corresponding to any camera to the image scene collected by another camera at the same time through the pixel mapping relationship and the bottom center pixel coordinates of the detection frame corresponding to each vehicle 2D target detection frame, and fuse the vehicle 2D target detection frame corresponding to the same target vehicle with the corresponding vehicle 2D target detection frame in the image scene collected by the other camera at the same time as the main detection frame.

[0097] Based on the above solution, the present invention can also be improved as follows.

[0098] Further, the same target vehicle is:

[0099] Mapping all vehicle 2D object detection frames corresponding to any one of the cameras to an image scene captured by another camera at the same time, and determining whether any vehicle 2D object detection frame in the image scene captured by the other camera at the same time has an intersecting vehicle 2D object detection frame;

[0100] If so, determine whether the intersecting area exceeds the preset area. When the intersecting area exceeds the preset area, determine that the target vehicle corresponding to any vehicle 2D target detection frame and the intersecting vehicle 2D target detection frame is the same target vehicle.

[0101] Furthermore, it also includes:

[0102] The adjustment module is used to: determine a right turn area in a camera scene of each camera of the two cameras when the camera is directed to the area to be photographed;

[0103] When any vehicle 2D target detection frame in the image scene captured by the other camera at the same time is in the right turn area, the vehicle 2D target detection frame corresponding to the same target vehicle is fused with the vehicle 2D target detection frame corresponding to any camera as the main detection frame.

[0104] Furthermore, it also includes:

[0105] The processing module is used for: if it does not exist, treating the vehicle 2D target detection frame corresponding to any camera without the same target vehicle as an independent detection frame without performing fusion processing.

[0106] It should be noted that the beneficial effects of the dual-video target fusion system based on the holographic intersection scene provided by the above embodiment are the same as the beneficial effects of the dual-video target fusion method based on the holographic intersection scene, which will not be repeated here. In addition, when the system provided by the above embodiment realizes its functions, it only takes the division of the above-mentioned functional modules as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to actual conditions to complete all or part of the functions described above. In addition, the system and method embodiments provided by the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0107] like Figure 2 As shown, an electronic device 300 according to an embodiment of the present invention includes a processor 320, the processor 320 is coupled to a memory 310, and the memory 310 stores at least one computer program 330, and the at least one computer program 330 is loaded and executed by the processor 320, so that the electronic device 300 implements any of the above methods, specifically:

[0108] The electronic device 300 may have relatively large differences due to different configurations or performances, and may include one or more processors 320 (Central Processing Units, CPU) and one or more memories 310, wherein the one or more memories 310 store at least one computer program 330, and the at least one computer program 330 is loaded and executed by the one or more processors 320, so that the electronic device 300 implements a dual video target fusion method based on a holographic intersection scene provided in the above embodiment. Of course, the electronic device 300 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The electronic device 300 may also include other components for implementing device functions, which will not be repeated here.

[0109] A computer-readable storage medium according to an embodiment of the present invention stores at least one computer program, and the at least one computer program is loaded and executed by a processor so that a computer implements any one of the above methods.

[0110] Optionally, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0111] In an exemplary embodiment, a computer program product or a computer program is also provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device performs any of the above methods.

[0112] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects and represent the definition of a specific order or sequence. The order of use of similar objects can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than the order shown or described.

[0113] Those skilled in the art know that the present invention can be implemented as a system, method or computer program product. Therefore, the present disclosure can be specifically implemented in the following forms, namely: it can be complete hardware, it can be complete software (including firmware, resident software, microcode, etc.), or it can be a combination of hardware and software, which is generally referred to as "circuit", "module" or "system" herein. In addition, in some embodiments, the present invention can also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable medium contains computer-readable program code.

[0114] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.

[0115] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A dual-video target fusion method based on a holographic intersection scene, characterized in that: include: Determine at least one base point in the camera scene of each camera of the two cameras for the area to be photographed and the latitude and longitude coordinates corresponding to each base point, determine at least one common base point in the common scene in the camera scenes of the two cameras and the pixel coordinates of any common base point in the camera scene corresponding to each camera; Decode the video streams collected by the two cameras for the area to be photographed in real time, and determine at least one vehicle 2D target detection frame in the area to be photographed and the center pixel coordinates of the bottom edge of the detection frame corresponding to each vehicle 2D target detection frame according to the decoding results, the longitude and latitude coordinates corresponding to each base point, and the pixel coordinates corresponding to each common base point; Through the pixel mapping relationship and the bottom center pixel coordinates of the detection frame corresponding to each vehicle 2D target detection frame, the vehicle 2D target detection frame corresponding to any camera is mapped to the image scene collected by another camera at the same time, and the vehicle 2D target detection frame corresponding to the same target vehicle is mainly fused with the corresponding vehicle 2D target detection frame in the image scene collected by the other camera at the same time.

2. According to the dual video target fusion method based on holographic intersection scene of claim 1, it is characterized in that: The same target vehicle is: Mapping all vehicle 2D object detection frames corresponding to any one of the cameras to an image scene captured by another camera at the same time, and determining whether any vehicle 2D object detection frame in the image scene captured by the other camera at the same time has an intersecting vehicle 2D object detection frame; If so, determine whether the intersecting area exceeds the preset area. When the intersecting area exceeds the preset area, determine that the target vehicle corresponding to any vehicle 2D target detection frame and the intersecting vehicle 2D target detection frame is the same target vehicle.

3. According to claim 2, a dual-video target fusion method based on a holographic intersection scene is characterized in that: Also includes: Determine a right turn area in a camera scene for each of the two cameras when targeting the area to be photographed; When any vehicle 2D target detection frame in the image scene captured by the other camera at the same time is in the right turn area, the vehicle 2D target detection frame corresponding to the same target vehicle is fused with the vehicle 2D target detection frame corresponding to any camera as the main detection frame.

4. According to the dual video target fusion method based on holographic intersection scene of claim 2, it is characterized in that: Also includes: If not, the vehicle 2D target detection frame corresponding to any of the cameras that does not have the same target vehicle is used as an independent detection frame without fusion processing.

5. A dual-video target fusion system based on holographic intersection scene, characterized in that: include: The determination module is used to: determine at least one base point in the camera scene of each camera of the two cameras for the area to be photographed and the latitude and longitude coordinates corresponding to each base point, determine at least one common base point in the common scene in the camera scenes of the two cameras and the pixel coordinates of any common base point in the camera scene corresponding to each camera; The detection module is used to: decode the video streams collected by the two cameras for the area to be photographed in real time, and determine at least one vehicle 2D target detection frame in the area to be photographed and the pixel coordinates of the bottom edge center of the detection frame corresponding to each vehicle 2D target detection frame according to the decoding processing results, the latitude and longitude coordinates corresponding to each base point, and the pixel coordinates corresponding to each common base point; The fusion module is used to: map the vehicle 2D target detection frame corresponding to any camera to the image scene collected by another camera at the same time through the pixel mapping relationship and the bottom center pixel coordinates of the detection frame corresponding to each vehicle 2D target detection frame, and fuse the vehicle 2D target detection frame corresponding to the same target vehicle with the corresponding vehicle 2D target detection frame in the image scene collected by the other camera at the same time as the main detection frame.

6. A dual video target fusion system based on holographic intersection scene according to claim 5, characterized in that: The same target vehicle is: Mapping all vehicle 2D object detection frames corresponding to any one of the cameras to an image scene captured by another camera at the same time, and determining whether any vehicle 2D object detection frame in the image scene captured by the other camera at the same time has an intersecting vehicle 2D object detection frame; If so, determine whether the intersecting area exceeds the preset area. When the intersecting area exceeds the preset area, determine that the target vehicle corresponding to any vehicle 2D target detection frame and the intersecting vehicle 2D target detection frame is the same target vehicle.

7. The dual-video target fusion system based on holographic intersection scene according to claim 6 is characterized in that: Also includes: The adjustment module is used to: determine a right turn area in a camera scene of each camera of the two cameras when the camera is directed to the area to be photographed; When any vehicle 2D target detection frame in the image scene captured by the other camera at the same time is in the right turn area, the vehicle 2D target detection frame corresponding to the same target vehicle is fused with the vehicle 2D target detection frame corresponding to any camera as the main detection frame.

8. The dual-video target fusion system based on holographic intersection scene according to claim 6 is characterized in that: Also includes: The processing module is used for: if it does not exist, treating the vehicle 2D target detection frame corresponding to any camera without the same target vehicle as an independent detection frame without performing fusion processing.

9. An electronic device, characterized in that: The electronic device comprises a processor, the processor is coupled to a memory, at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor so that the electronic device implements the method according to any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by a processor so that a computer implements the method according to any one of claims 1 to 4.

Images