High-frame-frequency real-time target area merging method and device
By using the target local contrast characteristics and adaptive thresholds to segment the target on the FPGA module and pre-merge according to the data flow direction, the problem of high time-consuming target feature merging in high-frame-rate video target detection is solved, and efficient target merging and real-time detection are achieved.
Patent Information
- Application Number
- CN202411521792.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-29
- Publication Date
- 2025-03-04
AI Technical Summary
The prior art is difficult to meet the real-time requirements in high-frame-rate video object detection, mainly because the FPGA module cannot extract and merge the target feature by cached the entire image, resulting in high time-consuming merge of target feature.
The high-frame frequency real-time target area merging method is adopted to suppress the background through the target local contrast characteristics, and the target is segmented using adaptive thresholds, and pre-merge is performed according to the data flow direction to generate target information.
It reduces the output information pressure between FPGA and DSP, reduces the number of processing points of the DSP module, improves the target merging efficiency, and realizes real-time processing of high-frame-rate video target detection.
Smart Images
Figure CN119559206A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image detection, and in particular to a high frame rate real-time target area merging method and device. Background Art
[0002] Infrared image target detection methods are usually deployed on a hardware platform of FPGA module and DSP module, where the FPGA module is connected to the DSP module, the FPGA module implements single-frame target information extraction, and sends the target information to the DSP module, which implements post-processing processes such as target detection and tracking algorithms. Under the requirements of high frame rate real-time target detection, limited by the architecture of FPGA image pipeline processing, it is not possible to cache the entire image, extract and merge target features, and then send them to the DSP module for post-processing. The existing processing method transmits the coordinates of the extracted target feature points from FPGA to DSP for target feature merging. The merging process is time-consuming and difficult to meet the real-time requirements of high frame rate video target detection.
[0003] In view of this, how to provide an FPGA to count and merge target information in each column (row) according to the data flow direction and send it to the DSP module, reduce the time consumption of DSP target feature merging, and realize real-time processing of high frame rate video target detection has become a technical problem that needs to be solved urgently. Summary of the invention
[0004] The embodiments of the present application provide a high frame rate real-time target area merging method and a high frame rate real-time target area merging device, which are used to solve the problem that the current architecture for FPGA image pipeline processing and DSP performance are limited and high frame rate image target detection cannot be processed in real time.
[0005] In a first aspect of an embodiment of the present application, a high frame rate real-time target area merging method is provided, comprising:
[0006] The background of the initial image is suppressed by using the local contrast characteristics of the target, and the target is segmented from the background-suppressed image through an adaptive threshold to obtain a background-suppressed segmentation result;
[0007] Pre-merging the background suppression segmentation results according to the data flow direction to obtain pre-merged merged point meta information;
[0008] The merging point meta information is merged to generate target information.
[0009] In a second aspect of an embodiment of the present application, a high frame rate real-time target region merging device is provided, comprising:
[0010] The target extraction module is configured to suppress the background of the initial image by using the local contrast characteristics of the target, and segment the target from the background-suppressed image by an adaptive threshold to obtain a background-suppressed segmentation result;
[0011] A point element pre-merging module is configured to pre-merge the background suppression segmentation results according to the data flow direction to obtain pre-merged merged point element information;
[0012] The target merging module is configured to merge the merging point meta information to generate target information.
[0013] The present application provides a high frame rate real-time target area merging method, comprising: firstly suppressing the background of the initial image by using the local contrast characteristics of the target, and segmenting the target from the background suppressed image through an adaptive threshold to obtain a background suppression segmentation result; then pre-merging the background suppression segmentation results according to the data flow direction to obtain pre-merged merged point element information; finally merging the merged point element information to generate target information.
[0014] The high frame rate real-time target area merging method provided in the embodiment of the present application can reduce the pressure of output information between the FPGA and the DSP on the one hand, and reduce the number of point elements processed by the DSP module on the other hand, thereby improving the target merging efficiency.
[0015] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, which can be implemented in accordance with the contents of the specification, and in order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:
[0017] Figure 1 A flowchart of a high frame rate real-time target area merging method provided in an embodiment of the present application;
[0018] Figure 2 A schematic diagram of a high frame rate real-time target area merging system provided in an embodiment of the present application;
[0019] Figure 3 A schematic diagram of a high frame rate real-time target area merging method provided in an embodiment of the present application taking the column direction of FPGA data stream as an example;
[0020] Figure 4 A schematic diagram of the structure of a high frame rate real-time target area merging system provided in an embodiment of the present application;
[0021] Figure 5 A schematic diagram of the structure of a high frame rate real-time target area merging device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0022] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.
[0023] See also Figure 1 , Figure 1 The following is a flow chart of a high frame rate real-time target area merging method provided in an embodiment of the present application. Figure 1 As shown, the specific steps include the following steps.
[0024] Step S102: using the local contrast characteristics of the target to suppress the background of the initial image, and segmenting the target from the background-suppressed image through an adaptive threshold to obtain a background-suppressed segmentation result;
[0025] Step S104: pre-merge the background suppression segmentation results according to the data flow direction to obtain pre-merged merged point meta information;
[0026] Step S106: Merge the merge point meta information to generate target information.
[0027] In an embodiment of the present application, the background suppression segmentation results are pre-merged according to the data flow direction to obtain the pre-merged target point position, including: merging continuous target point elements according to the background suppression segmentation results to obtain merged point element information, wherein the merged point element information carries continuous point element end coordinates and the number of point elements; and sending the merged point element information to the DSP target merging module for merging processing.
[0028] In an embodiment of the present application, the background suppression segmentation results are pre-merged according to the data flow direction to obtain the pre-merged target point position, including: merging continuous target point elements according to the background suppression segmentation results to obtain merged point element information, wherein the merged point element information carries continuous point element end coordinates and the number of point elements; and sending the merged point element information to the DSP target merging module for merging processing.
[0029] In an embodiment of the present application, the merging of point element information to generate target information includes: parsing the point element terminal coordinates and the number of point elements carried in the merged point element information to obtain a continuously segmented target area; merging each area in the continuously segmented target area, traversing all areas until the margins of all areas meet a preset margin threshold; performing feature calculation on the merged target area to generate a regional feature corresponding to the target area, wherein the regional feature carries the target area centroid position, the target area centroid area and the target area centroid grayscale.
[0030] See also Figure 2 , Figure 2 Schematic diagram of a high frame rate real-time target area merging system provided in an embodiment of the present application. Figure 2 As shown, the high frame rate real-time target area merging system includes three modules: one is the FPGA target extraction module 201, which is used to extract the target position; the second is the FPGA point element pre-merging module 202, which is used to pre-merge the target according to the data flow direction to alleviate the pressure of information sending and subsequent DSP merging calculation; the third is the DSP target merging module 203, which is used to synthesize complete target information.
[0031] Among them, the FPGA target extraction module 201 first uses the local contrast characteristics of the target to suppress the background and enhance the target signal, and then segments the target from the image through an adaptive threshold to achieve target extraction.
[0032] The FPGA point element pre-merger module 202 merges the target point positions output by the target extraction module according to the data flow direction. The module first merges the continuous target point elements according to the background suppression segmentation result. The merged point element information includes the coordinates of the last continuous point element and the number of included point elements. Finally, the FPGA module sends the merged point element information package to the DSP module for the next step of target merging.
[0033] See also Figure 3 , Figure 3 A high frame rate real-time target area merging method provided in an embodiment of the present application is a schematic diagram taking the column direction of FPGA data stream as an example. Taking the column direction of FPGA data stream as an example, the specific steps are as follows:
[0034] Step S1: Initialize the point element table and point element package information for each frame of data, where the point element package records the coordinates of the last point element in the package and the number of point elements included;
[0035] Step S2: Count the FPGA background suppression results of each frame according to the data flow direction;
[0036] Step S3: If the data stream changes columns, the current point element package information is recorded into the point element table, and the point element package is initialized;
[0037] Step S4: If the background suppression result of the current point element is 1, update the point element package information, record the current point element coordinates as the row and column coordinates of the last point element, and add 1 to the number of point elements;
[0038] Step S5: If the background suppression result of the current point element is 0, the current point element packet information is recorded into the point element table, and the point element is initialized;
[0039] Step S6: Repeat steps S2, S3, S4, and S5 until the current frame data stream ends.
[0040] The DSP target merging module 203 merges the point element information merged by the FPGA module. This module is divided into three steps: a point element information recovery unit 2031, a target area merging unit 2032, and a target feature calculation unit 2033.
[0041] Specifically, the point element information recovery unit 2031 is used to analyze the point element terminal coordinates and the number of included point elements sent by the FPGA, and preliminarily obtain the continuous segmentation target area A.
[0042] The target region merging unit 2032 is used to further merge the target region A obtained in the point meta information recovery step. The specific steps are as follows:
[0043] Step S11: setting a region margin threshold T;
[0044] Step S12: traverse the target area A, and record the current traversal area as a;
[0045] Step S13: For each region a, traverse the target region A twice, record the current traversed region as s, if there is a region s whose margin is less than T than the margin threshold of the currently traversed target region, merge the region s into the region a, and update the target region a information.
[0046] Step S14: repeat steps S12 and S13 until all regions do not meet the conditions;
[0047] The target feature calculation unit 2033 is used to calculate the centroid position, area, grayscale and other characteristics of the target area for the target detection and tracking step.
[0048] The high frame rate real-time target area merging method provided in the embodiment of the present application can reduce the pressure of output information between the FPGA and the DSP on the one hand, and reduce the number of point elements processed by the DSP module on the other hand, thereby improving the target merging efficiency.
[0049] See also Figure 4 , Figure 4 The structure diagram of a high frame rate real-time target area merging system provided in the embodiment of the present application is shown in FIG. Figure 4As shown, the high frame rate real-time target area merging system includes an infrared image device 401, an FPGA board 402, a DSP 403 and an infrared image output device 404.
[0050] Among them, the infrared imaging device 401 is used to capture images and send the captured images to the FPGA board 402; the FPGA board 402 is used to process the received images, realize pre-merged point elements, and send the pre-merged point elements to the DSP 403; the DSP 403 is used to process the pre-merged point elements, obtain target information, and display the target information as a target result through the infrared imaging output device 404.
[0051] Corresponding to the above method embodiment, this specification also provides an embodiment of a high frame rate real-time target area merging device, Figure 5 This is a schematic diagram of the structure of a high frame rate real-time target area merging device provided in an embodiment of the present application. Figure 5 As shown, the device comprises:
[0052] The target extraction module 502 is configured to suppress the background of the initial image by using the local contrast characteristics of the target, and segment the target from the background-suppressed image by an adaptive threshold to obtain a background-suppressed segmentation result;
[0053] The point element pre-merging module 504 is configured to pre-merge the background suppression segmentation results according to the data flow direction to obtain pre-merged merged point element information;
[0054] The target merging module 506 is configured to merge the merging point meta information to generate target information.
[0055] In an optional embodiment, the point element pre-merging module 504 is further configured to:
[0056] A merging unit is configured to merge continuous target point elements according to the background suppression segmentation result to obtain merged point element information, wherein the merged point element information carries the continuous point element terminal coordinates and the number of point elements;
[0057] The sending unit is configured to send the merging point meta information to the DSP target merging module for merging processing.
[0058] In an optional embodiment, the target merging module 506 is further configured to:
[0059] A point element information recovery unit is configured to parse and process the point element terminal coordinates and the number of point elements carried in the merged point element information to obtain a continuous segmentation target area;
[0060] A target region merging unit is configured to merge each region in the continuously segmented target region, traversing all regions until the margins of all regions meet a preset margin threshold;
[0061] The target feature calculation unit is configured to perform feature calculation on the target area after the merging process, and generate a region feature corresponding to the target area, wherein the region feature carries the target area centroid position, the target area centroid area and the target area centroid grayscale.
[0062] The high frame rate real-time target area merging device provided in the embodiment of the present application can reduce the pressure of outputting information between the FPGA and the DSP on the one hand, and reduce the number of point elements processed by the DSP module on the other hand, thereby improving the target merging efficiency.
[0063] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the high frame rate real-time target area merging device, since it is basically similar to the high frame rate real-time target area merging method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the high frame rate real-time target area merging method embodiment.
[0064] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0065] It should be noted that the above is a description of a specific embodiment of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of the present specification.
[0066] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0067] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.
Claims
1. A high frame rate real-time target area merging method, characterized in that: include: The background of the initial image is suppressed by using the local contrast characteristics of the target, and the target is segmented from the background-suppressed image through an adaptive threshold to obtain a background-suppressed segmentation result; Pre-merging the background suppression segmentation results according to the data flow direction to obtain pre-merged merged point meta information; The merging point meta information is merged to generate target information.
2. The method according to claim 1, characterized in that The pre-merging of the background suppression segmentation results according to the data flow direction to obtain the pre-merged target point position includes: According to the background suppression segmentation result, continuous target point elements are merged to obtain merged point element information, wherein the merged point element information carries the continuous point element terminal coordinates and the number of point elements; The merge point meta information is sent to the DSP target merge module for merge processing.
3. The method according to claim 1, characterized in that The merging of the merging point meta information to generate target information includes: The point element terminal coordinates and the number of the point elements carried in the merged point element information are analyzed to obtain a continuous segmentation target area; Merging each region in the continuously segmented target region, traversing all regions until the margins of all regions meet a preset margin threshold; The target area after the merging process is subjected to feature calculation to generate a region feature corresponding to the target area, wherein the region feature carries the target area centroid position, the target area centroid area and the target area centroid grayscale.
4. A high frame rate real-time target area merging device, characterized in that: include: The target extraction module is configured to suppress the background of the initial image by using the local contrast characteristics of the target, and segment the target from the background-suppressed image by an adaptive threshold to obtain a background-suppressed segmentation result; A point element pre-merging module is configured to pre-merge the background suppression segmentation results according to the data flow direction to obtain pre-merged merged point element information; The target merging module is configured to merge the merging point meta information to generate target information.
5. The device according to claim 4, characterized in that The point element pre-merging module includes: A merging unit is configured to merge continuous target point elements according to the background suppression segmentation result to obtain merged point element information, wherein the merged point element information carries the continuous point element terminal coordinates and the number of point elements; The sending unit is configured to send the merging point meta information to the DSP target merging module for merging processing.
6. The method according to claim 4, characterized in that The target merging module comprises: A point element information recovery unit is configured to parse and process the point element terminal coordinates and the number of point elements carried in the merged point element information to obtain a continuous segmentation target area; A target region merging unit is configured to merge each region in the continuously segmented target region, traversing all regions until the margins of all regions meet a preset margin threshold; The target feature calculation unit is configured to perform feature calculation on the target area after the merging process, and generate a region feature corresponding to the target area, wherein the region feature carries the target area centroid position, the target area centroid area and the target area centroid grayscale.
7. A computing device comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method described in any one of claims 1 to 3 are implemented.
8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an implementation program for information transmission, and when the program is executed by a processor, the steps of the method according to any one of claims 1 to 3 are implemented.