Target detection method and system for multi-channel video stitching

Abstract

The invention discloses a multi-channel video stitching target detection method and system, and the method comprises the following steps: S1, decoding the original compressed code stream of each camera, and outputting a decoding frame comprising a timestamp; s2, carrying out frame synchronization on the decoded frame to obtain a synchronization frame of each path of camera; s3, the synchronization frames of the cameras are preprocessed and then spliced into a to-be-detected picture; and S4, inputting a to-be-detected picture into the single model instance to execute one-time detection, and outputting global coordinates of a detection box of the to-be-detected picture and a first detection result. According to the invention, through a multi-source video synchronization mechanism, the time consistency of each path of camera decoding frame in the spliced image is ensured; batch detection of multi-channel videos is realized in a splicing mode, GPU occupation is remarkably reduced, throughput is greatly improved, and overall processing efficiency is remarkably improved.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, and in particular to a target detection method and system for multi-channel video stitching. Background Technology

[0002] In practical applications such as smart parks, traffic monitoring, and industrial security, it is often necessary to process video data from multiple cameras simultaneously. The relevant processing methods mainly fall into two categories, both of which have significant drawbacks: Related Method 1: Each camera starts an independent detection instance (multiple instances for single detection). The core problem is that when the number of cameras is massive, it consumes a lot of GPU memory, resulting in high hardware costs. Moreover, the performance scalability is poor. N cameras need to correspond to N detection instances, and resource consumption increases linearly.

[0003] Related Method 2: Each camera is fed into the detection queue frame by frame and detected one by one (multiple detections per single instance). The core problem is that the model input switches frequently, the detection process is not continuous, the GPU utilization fluctuates greatly, and the resources are wasted. The detection latency is unstable and cannot meet the real-time requirements.

[0004] The relevant technologies cannot effectively reduce the resource consumption of parallel detection of multiple video streams, and the computational consumption increases non-linearly as the number of cameras increases; in addition, when processing the stitched image, the relevant models may misjudge targets that cross sub-regions (such as a car appearing at the boundary of the stitched image of two cameras) as a large target. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to address at least one defect of the related technologies mentioned in the background: the related technologies cannot effectively reduce the resource consumption of parallel detection of multiple video streams, and the computational consumption increases non-linearly when the number of cameras increases. The present invention provides a target detection method and system for multi-channel video stitching.

[0006] The technical solution adopted by this invention to solve its technical problem is: to construct a target detection method for multi-channel video stitching, comprising the following steps: S1: Decode the raw compressed bitstreams from each camera and output decoded frames including timestamps; S2: Perform frame synchronization on the decoded frames to obtain the synchronization frames of each camera. S3: Preprocess the synchronization frames from each camera and stitch them together to form the image to be detected; S4: Input the image to be detected into a single model instance and perform a detection once. Output the global coordinates of the detection box of the image to be detected and the first detection result.

[0007] In some embodiments, the first detection result includes at least one of the following: the target category, confidence level, and timestamp.

[0008] In some embodiments, step S1 includes: The raw compressed bitstreams from each camera are decoded in parallel using multi-threading, and an independent frame buffer queue is established for each camera. The decoded frames, including timestamps, are then stored in the frame buffer queue.

[0009] In some embodiments, step S2 includes: The maximum time difference of the current decoded frames of each camera is calculated based on the timestamps; If the maximum time difference is not greater than the preset difference, the current decoded frame is used as the output synchronization frame; or, if the maximum time difference is greater than the preset difference, the last available decoded frame from the camera is used as the output synchronization frame; or, if the camera malfunctions, a blank frame is used as the output synchronization frame.

[0010] In some embodiments, the maximum time difference of the current decoded frames of each camera is calculated based on the timestamp, including: After subtracting the timestamps of the last frame of the current decoded frame from each camera to obtain multiple decoding frame time differences, the maximum time difference of the current decoded frame is obtained by comparing the multiple decoding frame time differences.

[0011] In some embodiments, synchronization frame preprocessing includes scaling the synchronization frame to a uniform size and normalizing the pixel values.

[0012] In some embodiments, step S3 further includes adding a boundary line of preset pixels between each synchronous frame of the image to be detected.

[0013] In some embodiments, the decoded frame also includes a camera ID; the method further includes the following steps: S5: Query the stitching offset matrix based on the global coordinates. The stitching offset matrix includes the IDs of each camera and the corresponding stitching offset parameters. Determine the camera ID to which the detection box of the image to be detected belongs by using the stitching offset matrix and the global coordinates. Calculate the corresponding local coordinates using the global coordinates and the stitching offset parameters. S6: Output the second detection result according to the preset output data format; The second detection result includes: the first detection result, the camera ID, and the local coordinates.

[0014] In some embodiments, global coordinates and local coordinates include: their respective horizontal coordinates, vertical coordinates, detection box width, and detection box height; Calculate the corresponding local coordinates using global coordinate splicing offset parameters, including: Subtract the horizontal offset parameter from the global horizontal coordinate to obtain the local horizontal coordinate; Subtract the vertical offset parameter from the global vertical coordinate to obtain the local vertical coordinate; The width and height of the detection box in local coordinates remain unchanged from those in global coordinates; the local coordinates are obtained by combining them after calculation.

[0015] Some embodiments of the present invention also disclose a target detection system for multi-channel video stitching, the system comprising: The video decoding thread pool is used to decode the raw compressed bitstreams from various cameras and output decoded frames including timestamps. The frame synchronization module is used to synchronize the decoded frames to obtain synchronized frames from each camera. The image stitching module is used to preprocess and stitch together the synchronized frames from various cameras into an image to be detected; and, The detection module is used to input the image to be detected into a single model instance to perform a single detection, and output the global coordinates of the detection box of the image to be detected and the first detection result.

[0016] By implementing this invention, the following beneficial effects are achieved: This invention ensures the time consistency of decoding frames from various cameras in the stitched image through a multi-source video synchronization mechanism; it enables batch detection of multiple video streams through stitching, significantly reducing GPU usage, greatly increasing throughput, and significantly improving overall processing efficiency. Attached Figure Description

[0017] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings: Figure 1 A flowchart illustrating the steps of an embodiment of the target detection method for multi-channel video stitching of the present invention is shown. Figure 2 The diagram shows an overall framework of an embodiment of the target detection system for multi-channel video stitching of the present invention. Detailed Implementation

[0018] To provide a clearer understanding of the technical features, objectives, and effects of the present invention, specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0019] It should be noted that the flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0020] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0021] It should be noted that "at least two" refers to at least two, which can be two, three, or any number. "At least one" can be one, two, or any number.

[0022] like Figure 1 As shown, some embodiments of the present invention disclose a target detection method for multi-channel video stitching, including the following steps: S1: Decode the raw compressed bitstreams from each camera and output decoded frames including timestamps; S2: Perform frame synchronization on the decoded frames to obtain the synchronization frames of each camera. S3: Preprocess the synchronization frames from each camera and stitch them together to form the image to be detected; S4: Input the image to be detected into a single model instance and perform a detection once. Output the global coordinates of the detection box of the image to be detected and the first detection result.

[0023] Specifically, the image to be detected is input into a single-look object detection model to perform one forward propagation inference. The single-look object detection model is specifically the YOLO (You Only Look Once) model, which outputs the global coordinates (x, y, w, h) of the detection box of the image to be detected and the first detection result. Here, x is the global horizontal coordinate, y is the global vertical coordinate, w is the width of the detection box, and h is the height of the detection box.

[0024] This method eliminates the need to create independent model instances for each camera, reducing instance management overhead; it significantly reduces GPU memory usage (it only needs to support one stitched image instead of N independent frames); and its throughput is 2 to 5 times higher than traditional methods (depending on the number of cameras stitched).

[0025] This embodiment ensures the time consistency of decoding frames from various cameras in the stitched image through a multi-source video synchronization mechanism; it achieves batch detection of multiple video streams through stitching, significantly reducing GPU usage, greatly increasing throughput, and significantly improving overall processing efficiency.

[0026] In some embodiments, the first detection result includes at least one of the following: the target category, confidence level, and timestamp.

[0027] In some embodiments, step S1 includes: The raw compressed bitstreams from each camera are decoded in parallel using multi-threading, and an independent frame buffer queue is established for each camera. The decoded frames, including timestamps, are output to the frame buffer queue for storage, with timestamps accurate to the millisecond level.

[0028] In some embodiments, the multi-threaded parallel decoding scheme includes at least one of FFmpeg software decoding, GStreamer software decoding, and GPU hardware decoding. Using multi-threaded parallel decoding can avoid single-threaded blocking, improve frame recovery speed, and ensure real-time performance.

[0029] In some embodiments, step S2 includes: The maximum time difference of the current decoded frames of each camera is calculated based on the timestamps; If the maximum time difference is not greater than the preset difference, the current decoded frame is used as the output synchronization frame; or, if the maximum time difference is greater than the preset difference, the last available decoded frame from the camera is used as the output synchronization frame; or, if the camera malfunctions, a blank frame is used as the output synchronization frame.

[0030] If the maximum time difference exceeds a preset difference, the last available decoded frame from that camera is used as the synchronization frame output. Specifically, if the maximum time difference exceeds the preset difference, indicating network latency or bitstream loss, the last available (even slightly older) decoded frame from the camera's frame buffer queue is used as the synchronization frame output to prevent pipeline stalls due to waiting. For example, the preset difference is 80ms by default and can be configured. If the maximum time difference exceeds 80ms, indicating temporary network jitter, the last available decoded frame from that camera is used as the synchronization frame output.

[0031] If the camera malfunctions, blank frames are used as synchronization frames in the output. Specifically, a camera malfunction manifests as a continuous lack of bitstream input, indicating the camera is offline. Blank frames are then used as synchronization frames to maintain the stability of the stitched image structure and prevent disruption to the overall stitching process. The blank frames can be one of black, white, or gray. This soft synchronization strategy does not require strict waiting for all frames to align, making it suitable for real-time video scenarios and preventing overall blockage due to single-channel anomalies.

[0032] In some embodiments, the maximum time difference of the current decoded frames of each camera is calculated based on the timestamps, including: obtaining multiple decoding frame time differences by subtracting the timestamps of the last frame of the current decoded frames of each camera pairwise, and then comparing the multiple decoding frame time differences to obtain the maximum time difference of the current decoded frames.

[0033] In some embodiments, step S2 further includes: recording the synchronization failure event to a log for operation and maintenance monitoring, but without affecting other normal detection processes.

[0034] In some embodiments, synchronization frame preprocessing includes scaling the synchronization frame to a uniform size and normalizing the pixel values.

[0035] Specifically, the synchronization frames of all cameras are scaled to a uniform size (e.g., 640×360, configurable) and the pixel values ​​are normalized according to the YOLO model input requirements (e.g., the pixel values ​​are mapped to the [0,1] range).

[0036] In some embodiments, the stitching method uses a Mosaic layout; after stitching, the size of the image to be detected conforms to the input of the single-instance YOLO detection model.

[0037] In some embodiments, step S3 further includes adding a boundary line of preset pixels between each synchronized frame of the image to be detected. For example, adding a boundary line with a black border of 1 to 5 pixels between each synchronized frame of the image to be detected. This suppresses false detections across regions, prevents the YOLO model from misidentifying cross-region objects as continuous entities, and improves detection accuracy.

[0038] In some embodiments, the decoded frame also includes a camera ID, and the method further includes the following steps: S5: Query the stitching offset matrix based on the global coordinates. The stitching offset matrix includes the IDs of each camera and the corresponding stitching offset parameters. Determine the camera ID to which the detection box of the image to be detected belongs by using the stitching offset matrix and the global coordinates, and calculate the corresponding local coordinates using the stitching offset parameters. Specifically, the image to be detected is input into a single YOLO model instance, and forward propagation inference is performed once. After outputting the global coordinates (x, y, w, h) of the detection box of the image to be detected, the stitching offset matrix is ​​queried based on (x, y) in the global coordinates. The stitching offset matrix contains the IDs of each camera and the corresponding stitching offset parameters. The camera ID to which the detection box of the image to be detected belongs is determined by the stitching offset matrix and the global coordinates. The corresponding local coordinates are calculated using the stitching offset parameters.

[0039] For example, when there are 4 cameras, the image to be detected is input into a single YOLO model instance and a forward propagation inference is performed. The YOLO model detects a target and outputs the global coordinates (720, 350, 20, 10) of the detection box of the image to be detected. Then, based on the (720, 350) in the global coordinates, the stitching offset matrix is ​​queried to determine that the camera ID to which the detection box of the image to be detected belongs is C2. An example of the stitching offset matrix is ​​shown in Table 1 below: Table 1 S6: Store and output the second detection result according to the preset output data format; The second detection result includes: the first detection result, the camera ID, and the local coordinates. The default output data format is fields and descriptions. Specifically, the default output data format is shown in Table 2 below: Table 2 In some embodiments, global coordinates and local coordinates include their respective horizontal coordinates, vertical coordinates, detection box width, and detection box height; Calculate the corresponding local coordinates using the splicing offset parameter, including: The local horizontal coordinate is obtained by subtracting the horizontal offset parameter from the global horizontal coordinate; specifically, local_x = x - region[camera_id].x_offset, where local_x is the local horizontal coordinate, x is the global horizontal coordinate, and region[camera_id].x_offset is the horizontal offset parameter.

[0040] The local vertical coordinate is obtained by subtracting the vertical offset parameter from the global vertical coordinate; specifically, local_y = y - region[camera_id].y_offset, where local_y is the local vertical coordinate, y is the global horizontal coordinate, and region[camera_id].y_offset is the vertical offset parameter.

[0041] The width and height of the detection box in local coordinates remain unchanged from those in global coordinates; these are then combined to obtain the local coordinates. The detected bounding box (BBox) can be offset and recalculated back to the original camera coordinate system to ensure coordinate accuracy.

[0042] Specifically, the image to be detected is input into a single YOLO model instance to perform one forward propagation inference, outputting the global coordinates (x, y, w, h) of the detection box for the image and the first detection result. Here, x, y, w, and h represent the global horizontal coordinate, global vertical coordinate, detection box width, and detection box height, respectively.

[0043] For example, after outputting the global coordinates (720, 350, 20, 10) of the detection box of the image to be detected, the stitching offset matrix table 1 is queried based on the global coordinates (720, 350) to determine that the camera ID to which the detection box of the image to be detected belongs is C2, its horizontal offset parameter is 640, its vertical offset parameter is 0, its global horizontal coordinate is 720, its global vertical coordinate is 350, the detection box width is 20 and the detection box height is 10. After calculation, the local coordinates are obtained as (80, 350, 20, 10).

[0044] In some embodiments, the method also supports interfacing with mainstream protocols such as Message Queuing Telemetry Transport (MQTT), WebSocket, and Representational State Transfer Application Programming Interface (REST API), and can be directly integrated into existing video surveillance platforms.

[0045] like Figure 2 As shown, some embodiments of the present invention also disclose a target detection system for multi-channel video stitching, the system comprising: The video decoding thread pool is used to decode the raw compressed bitstreams from various cameras and output decoded frames including timestamps. The frame synchronization module is used to synchronize the decoded frames to obtain synchronized frames from each camera. The image stitching module is used to preprocess and stitch together the synchronized frames from various cameras into an image to be detected; and, The detection module is used to input the image to be detected into a single model instance to perform a single detection, and output the global coordinates of the detection box of the image to be detected and the first detection result.

[0046] Specifically, the image to be detected is input into a single-look object detection model to perform one forward propagation inference. The single-look object detection model is specifically the YOLO (You Only Look Once) model, which outputs the global coordinates (x, y, w, h) of the detection box of the image to be detected and the first detection result. Here, x is the global horizontal coordinate, y is the global vertical coordinate, w is the width of the detection box, and h is the height of the detection box.

[0047] This system eliminates the need to create independent model instances for each camera, reducing instance management overhead; GPU memory usage is significantly reduced (it only needs to support one stitched image instead of N independent frames); and throughput is increased by 2 to 5 times compared to traditional methods (depending on the number of cameras stitched).

[0048] This embodiment ensures the time consistency of decoding frames from various cameras in the stitched image through a multi-source video synchronization mechanism; it achieves batch detection of multiple video streams through stitching, significantly reducing GPU usage, greatly increasing throughput, and significantly improving overall processing efficiency.

[0049] In some embodiments, the video decoding thread pool is used to decode the raw compressed bitstreams of each camera in parallel using multiple threads, and to establish an independent frame buffer queue for each camera, outputting decoded frames including timestamps to the frame buffer queue for storage, with the timestamps accurate to the millisecond level.

[0050] In some embodiments, the multi-threaded parallel decoding scheme includes at least one of FFmpeg software decoding, GStreamer software decoding, and GPU hardware decoding. Using multi-threaded parallel decoding can avoid single-threaded blocking, improve frame recovery speed, and ensure real-time performance.

[0051] In some embodiments, frame synchronization is performed on the decoded frames to obtain synchronized frames from each camera, specifically including: The maximum time difference of the current decoded frames of each camera is calculated based on the timestamps; If the maximum time difference is not greater than the preset difference, the current decoded frame is used as the output synchronization frame; or, if the maximum time difference is greater than the preset difference, the last available decoded frame from the camera is used as the output synchronization frame; or, if the camera malfunctions, a blank frame is used as the output synchronization frame.

[0052] If the maximum time difference exceeds a preset difference, the last available decoded frame from that camera is used as the synchronization frame output. Specifically, if the maximum time difference exceeds the preset difference, indicating network latency or bitstream loss, the last available (even slightly older) decoded frame from the camera's frame buffer queue is used as the synchronization frame output to prevent pipeline stalls due to waiting. For example, the preset difference is 80ms by default and can be configured. If the maximum time difference exceeds 80ms, indicating temporary network jitter, the last available decoded frame from that camera is used as the synchronization frame output.

[0053] If the camera malfunctions, blank frames are used as synchronization frames in the output. Specifically, a camera malfunction manifests as a continuous lack of bitstream input, indicating the camera is offline. Blank frames are then used as synchronization frames to maintain the stability of the stitched image structure and prevent disruption to the overall stitching process. The blank frames can be one of black, white, or gray. This soft synchronization strategy does not require strict waiting for all frames to align, making it suitable for real-time video scenarios and preventing overall blockage due to single-channel anomalies.

[0054] In some embodiments, the maximum time difference of the current decoded frames of each camera is calculated based on the timestamps, including: obtaining multiple decoding frame time differences by subtracting the timestamps of the last frame of the current decoded frames of each camera pairwise, and then comparing the multiple decoding frame time differences to obtain the maximum time difference of the current decoded frames.

[0055] In some embodiments, the frame synchronization module is also used to record synchronization failure events to a log for operation and maintenance monitoring, but does not affect other normal detection processes.

[0056] In some embodiments, synchronization frame preprocessing includes scaling the synchronization frame to a uniform size and normalizing the pixel values.

[0057] Specifically, the synchronization frames of all cameras are scaled to a uniform size (e.g., 640×360, configurable) and the pixel values ​​are normalized according to the YOLO model input requirements (e.g., the pixel values ​​are mapped to the [0,1] range).

[0058] In some embodiments, the image stitching module uses a Mosaic layout for stitching; after stitching, the size of the image to be detected conforms to the input of the single-instance YOLO detection model.

[0059] In some embodiments, the image stitching module is also used to add a boundary line of preset pixels between each synchronous frame of the image to be detected. For example, a boundary line with a black border of 1 to 5 pixels is added between each synchronous frame of the image to be detected. This suppresses false detections across regions, prevents the YOLO model from misidentifying cross-region objects as continuous entities, and improves detection accuracy.

[0060] In some embodiments, the decoded frame also includes a camera ID. The system also includes an offset calculation module and an output module, specifically: The offset calculation module is used to query the stitching offset matrix based on global coordinates. The stitching offset matrix contains the IDs of each camera and the corresponding stitching offset parameters. The camera ID to which the detection box of the image to be detected belongs is determined by the stitching offset matrix and global coordinates. The camera ID contains the stitching offset parameters. The corresponding local coordinates are calculated using global coordinates and stitching offset parameters. Specifically, the image to be detected is input into a single YOLO model instance, which performs forward propagation inference once. After outputting the global coordinates (x, y, w, h) of the detection box of the image to be detected, the offset calculation module queries the stitching offset matrix based on the (x, y) in the global coordinates. The stitching offset matrix contains the IDs of each camera and the corresponding stitching offset parameters. The camera ID to which the detection box of the image to be detected belongs is determined by the stitching offset matrix and the global coordinates, and the corresponding local coordinates are calculated using the stitching offset parameters.

[0061] For example, when there are 4 cameras, the image to be detected is input into a single YOLO model instance and a forward propagation inference is performed. The YOLO model detects a target and outputs the global coordinates (720, 350, 20, 10) of the detection box of the image to be detected. Then, based on the (720, 350) in the global coordinates, the stitching offset matrix is ​​queried to determine that the camera ID to which the detection box of the image to be detected belongs is C2. An example of the stitching offset matrix is ​​shown in Table 1 below: Table 1 The output module is used to store and output the second detection result according to a preset output data format; The second detection result includes: detection result, camera ID, and local coordinates. The default output data format is fields and descriptions. Specifically, the default output data format is shown in Table 2 below: Table 2 In some embodiments, global coordinates and local coordinates include their respective horizontal coordinates, vertical coordinates, detection box width, and detection box height; Calculate the corresponding local coordinates using the splicing offset parameter, including: The local horizontal coordinate is obtained by subtracting the horizontal offset parameter from the global horizontal coordinate; specifically, local_x = x - region[camera_id].x_offset, where local_x is the local horizontal coordinate, x is the global horizontal coordinate, and region[camera_id].x_offset is the horizontal offset parameter.

[0062] The local vertical coordinate is obtained by subtracting the vertical offset parameter from the global vertical coordinate; specifically, local_y = y - region[camera_id].y_offset, where local_y is the local vertical coordinate, y is the global horizontal coordinate, and region[camera_id].y_offset is the vertical offset parameter.

[0063] The width and height of the detection bounding box in local coordinates remain unchanged from those in global coordinates; these are then combined to obtain the local coordinates. The detected bounding box (bbox) can be offset and recalculated back to the original camera coordinate system to ensure coordinate accuracy.

[0064] Specifically, the detection module inputs the image to be detected into a single YOLO model instance and performs forward propagation inference once, outputting the global coordinates (x, y, w, h) of the detection box for the image and the first detection result. Here, x, y, w, and h represent the global horizontal coordinate, global vertical coordinate, detection box width, and detection box height, respectively.

[0065] For example, after the detection module outputs the global coordinates (720, 350, 20, 10) of the detection box of the image to be detected, the offset calculation module queries the splicing offset matrix table 1 based on the global coordinates (720, 350) to determine that the camera ID to which the detection box of the image to be detected belongs is C2, its horizontal offset parameter is 640, its vertical offset parameter is 0, its global horizontal coordinate is 720, its global vertical coordinate is 350, the detection box width is 20 and the detection box height is 10. After calculation, the local coordinates are obtained as (80, 350, 20, 10).

[0066] In some embodiments, the system also supports interfacing with mainstream protocols such as Message Queuing Telemetry Transport (MQTT), WebSocket, and Representational State Transfer Application Programming Interface (REST API), and can be directly integrated into existing video surveillance platforms.

[0067] By implementing this invention, the following beneficial effects are achieved: This embodiment employs a multi-source video synchronization mechanism. Synchronization is considered successful if the maximum time difference between the timestamps of the decoded frames does not exceed a preset value, ensuring time consistency of decoded frames from various cameras in the stitched image. Frames that are not synchronized are filled by retrieving the last frame or a blank frame from the frame buffer queue of that camera. This soft synchronization strategy does not require strict waiting for all frames to align, making it suitable for real-time video scenarios and preventing overall blocking due to single-channel anomalies. By inserting black pixel isolation bands between sub-images, visual breakpoints are artificially created, preventing the model from forming cross-region feature connections and avoiding the YOLO model misidentifying cross-region objects as continuous entities, thus improving detection accuracy. Batch detection of multiple video streams is achieved through stitching, significantly reducing GPU usage, greatly increasing throughput, and significantly improving overall processing efficiency. The position of each camera sub-image is recorded using a stitching offset matrix; the detected bounding boxes (BBoxes) can be calculated back to the original camera's local coordinate system through offset, ensuring coordinate accuracy.

[0068] It is understood that the above embodiments only illustrate some implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can freely combine the above embodiments or technical features without departing from the concept of the present invention, and can also make several modifications and improvements, all of which fall within the protection scope of the present invention. That is, the embodiments described "in some embodiments" can be freely combined with any of the preceding and following embodiments. Therefore, all equivalent transformations and modifications made within the scope of the claims of the present invention should be covered by the claims of the present invention.

Claims

1. A target detection method for multi-channel video stitching, characterized in that, Includes the following steps: S1: Decode the raw compressed bitstreams from each camera and output decoded frames including timestamps; S2: Perform frame synchronization on the decoded frames to obtain the synchronization frames of each of the cameras; S3: The synchronization frames of each camera are preprocessed and stitched together to form the image to be detected; S4: Input the image to be detected into a single model instance to perform a detection once, and output the global coordinates of the detection box of the image to be detected and the first detection result.

2. The target detection method for multi-channel video stitching according to claim 1, characterized in that, The first detection result includes at least one of the following: the target category, confidence level, and timestamp.

3. The target detection method for multi-channel video stitching according to claim 1, characterized in that, Step S1 includes: The original compressed bitstreams from each camera are decoded in parallel using multi-threading, and an independent frame buffer queue is established for each camera. The decoded frames, including timestamps, are then stored in the frame buffer queue.

4. The target detection method for multi-channel video stitching according to claim 1, characterized in that, Step S2 includes: The maximum time difference of the current decoded frames of each of the cameras is calculated based on the timestamps. If the maximum time difference is not greater than a preset difference, then the current decoded frame is used as the synchronization frame for output; or, If the maximum time difference is greater than a preset difference, then the last available decoded frame from that camera is used as the synchronization frame output; or... If the camera malfunctions, blank frames are filled in as synchronization frames for output.

5. The target detection method for multi-channel video stitching according to claim 4, characterized in that, The step of calculating the maximum time difference of the current decoded frames of each camera based on the timestamp includes: After subtracting the timestamps of the last frame of the current decoding frame from each of the cameras to obtain multiple decoding frame time differences, the maximum time difference of the current decoding frame is obtained by comparing the multiple decoding frame time differences.

6. The target detection method for multi-channel video stitching according to claim 1, characterized in that, The synchronization frame preprocessing includes: The synchronization frames are scaled to a uniform size and their pixel values ​​are normalized.

7. The target detection method for multi-channel video stitching according to claim 1, characterized in that, Step S3 also includes: A boundary line of preset pixels is added between each synchronization frame of the image to be detected.

8. The target detection method for multi-channel video stitching according to claim 1, characterized in that, The decoded frame also includes the camera ID; The method also includes the following steps: S5: Query the stitching offset matrix according to the global coordinates. The stitching offset matrix includes the IDs of each camera and the corresponding stitching offset parameters. Determine the camera ID to which the detection box of the image to be detected belongs by the stitching offset matrix and the global coordinates. Calculate the corresponding local coordinates using the global coordinates and the stitching offset parameters. S6: Output the second detection result according to the preset output data format; The second detection result includes: the first detection result, the camera ID, and the local coordinates.

9. The target detection method for multi-channel video stitching according to claim 8, characterized in that, The global coordinates and local coordinates include: their respective horizontal coordinates, vertical coordinates, detection box width, and detection box height; The calculation of the corresponding local coordinates using the splicing offset parameter of the global coordinates includes: Subtract the horizontal offset parameter from the global horizontal coordinate to obtain the local horizontal coordinate; Subtract the vertical offset parameter from the global vertical coordinate to obtain the local vertical coordinate; The detection box width and height for local coordinates remain unchanged from those for global coordinates. The local coordinates are obtained by combining the calculated values.

10. A target detection system for multi-channel video stitching, characterized in that, The system includes: The video decoding thread pool is used to decode the raw compressed bitstreams from various cameras and output decoded frames including timestamps. A frame synchronization module is used to synchronize the decoded frames to obtain synchronized frames from each of the cameras. The image stitching module is used to preprocess and stitch the synchronization frames from each of the cameras into an image to be detected; and, The detection module is used to input the image to be detected into a single model instance to perform a single detection, and output the global coordinates of the detection box of the image to be detected and the first detection result.

Images