A target recognition method and device based on motion detection and local search acceleration

Abstract

The application provides a target recognition method and device based on motion detection and local search acceleration, which comprises the following steps: acquiring a target video to be fused by using an RGB camera; recognizing the target to be fused for the initial frame image of the target video to be fused; uniformly dividing a newly acquired frame image into several parts by using a pixel line, so that the target will intersect with the division line when the target moves; determining whether the target to be fused is in a motion state by comparing the RGB difference of the division line; when the target to be fused moves, the target will intersect with the division line of the rectangular region; determining the local search region of the target to be fused according to whether the difference of the RGB pixel value on the horizontal and vertical division lines changes; recognizing the target to be fused in the local search region; and completing the processing of all frame images of the video. The application provides the motion state detection of the target to be fused based on the division line, and simplifies the processing flow when the target to be fused is static.

Description

Technical Field

[0001] This invention relates to the field of target recognition technology, and in particular to a target recognition method and apparatus based on motion detection and local search acceleration. Background Technology

[0002] In traditional virtual-real fusion technology, the search and recognition of the target to be fused requires processing each video frame. If the target remains stationary for a long time, only the first frame of the image at that time needs to be copied, without processing each frame of the video image, thus saving computing power and improving the response speed of the entire system. Therefore, considering that different processing methods are used when the target to be fused is stationary or in motion, this invention proposes a target recognition method based on motion state detection and local acceleration. Before recognizing the target to be fused, the motion state is quickly detected, and a local search strategy is used to improve the recognition speed of the target to be fused.

[0003] Existing virtual-real fusion methods do not adequately consider the motion state detection problem in the recognition of the target to be fused. This leads to excessive redundant computation when the target remains stationary for extended periods, affecting device battery life and reducing the stability of target recognition. Furthermore, existing virtual-real fusion methods lack local search strategies for the target to be fused, thus reducing recognition speed and making it difficult to guarantee the real-time performance of virtual-real fusion. Summary of the Invention

[0004] The technical problem to be solved by the present invention is how to quickly detect the motion state before the target to be fused is identified, and to use a local search strategy to improve the identification speed of the target to be fused; in view of this, the present invention provides a target identification method and apparatus based on motion detection and local search acceleration.

[0005] The technical solution adopted in this invention is that the target recognition method based on motion detection and local search acceleration includes:

[0006] Step S1: Use an RGB camera to acquire the target video to be fused;

[0007] Step S2: For the initial frame image of the target video to be fused, identify the target to be fused;

[0008] Step S3: Based on the identified target to be fused, the latest frame image is evenly divided into several parts using pixel lines, so that the target to be fused will intersect with at least one of the several dividing lines when it moves. By comparing the RGB difference of the dividing lines in the latest two frames, it is determined whether the target to be fused is in motion. If it is in motion, proceed to step S4. If it is not in motion, read a new frame image and repeat step S3.

[0009] Step S4: Several rectangular regions generated when segmenting the current frame image using pixel lines are used. When the target to be fused moves, it will intersect with the segmentation lines of the rectangular regions. Based on whether the difference between the RGB pixel values ​​on the horizontal and vertical segmentation lines changes, the local search area of ​​the target to be fused is determined.

[0010] Step S5: Within the local search area, the target to be fused is identified. If all frames of the video have been processed, the target identification ends; otherwise, a new frame is read and the process proceeds to step 3.

[0011] In one implementation, step S2 includes:

[0012] For the initial frame of the video, a target matching method based on scale-invariant feature transformation is used to identify the target to be fused.

[0013] In one implementation, step S3 includes:

[0014] By determining the sum of RGB pixel values ​​on the horizontal and vertical dividing lines of two frames, the difference between the sum of the RGB pixel values ​​between the two frames is compared with a preset threshold. If the difference is greater than the threshold, the target to be fused is determined to be in motion.

[0015] In one implementation, step S5 includes:

[0016] Within the local search area, the target to be fused is identified using a target matching method based on SIFT features.

[0017] Another aspect of the present invention provides a target recognition device based on motion detection and local search acceleration, comprising:

[0018] The acquisition unit is configured to acquire the target video to be fused using an RGB camera;

[0019] The first identification unit is configured to identify the target to be fused for the initial frame image of the target video to be fused;

[0020] The motion detection unit is configured to divide the most recently acquired frame image into several parts using pixel lines based on the identified target to be fused, so that the target to be fused will intersect with at least one of the several dividing lines when it moves. By comparing the RGB difference of the dividing lines in the two most recently acquired frames, it is determined whether the target to be fused is in motion. If it is in motion, it enters the local search unit. If it is not in motion, it reads a new frame image and repeats the processing of the motion detection unit.

[0021] The local search unit is configured to use pixel lines to segment several rectangular regions generated when the current frame image is segmented. When the target to be fused moves, it will intersect with the segmentation lines of the rectangular regions. Based on whether the difference between the RGB pixel values ​​on the horizontal and vertical segmentation lines changes, the local search region of the target to be fused is determined.

[0022] The second recognition unit is configured to recognize the target to be fused within the local search area. If all frames of the video have been processed, the target recognition ends; otherwise, a new frame is read and the process is transferred to the motion detection unit.

[0023] In one embodiment, the first identification unit is further configured to: identify the target to be fused using a target matching method based on scale-invariant feature transformation features for the initial frame of the video.

[0024] In one embodiment, the motion detection unit is further configured to: determine the sum of RGB pixel values ​​on the horizontal and vertical dividing lines of two frames, compare the difference between the sum of the RGB pixel values ​​between the two frames with a preset threshold, and if the difference is greater than the threshold, the target to be fused is determined to be in motion.

[0025] In one embodiment, the second identification unit is further configured to: identify the target to be fused within the local search area using a target matching method based on SIFT features.

[0026] Another aspect of the present invention provides an electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the target recognition method based on motion detection and local search acceleration as described in any of the preceding claims.

[0027] Another aspect of the present invention provides a computer storage medium storing a computer program that, when executed by a processor, implements the steps of the target recognition method based on motion detection and local search acceleration as described in any of the preceding claims.

[0028] By adopting the above technical solution, the present invention has at least the following advantages:

[0029] 1) This invention proposes motion state detection of the target to be fused based on segmentation lines, which simplifies the processing flow when the target to be fused is stationary;

[0030] 2) The present invention determines the local search area based on the dividing line, and identifies the target to be fused in the local search area, which effectively improves the recognition speed. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the target recognition method based on motion detection and local search acceleration according to an embodiment of the present invention;

[0032] Figure 2 This is a schematic diagram of another target recognition method based on motion detection and local search acceleration according to an embodiment of the present invention;

[0033] Figure 3 This is a schematic diagram of the target recognition device based on motion detection and local search acceleration according to an embodiment of the present invention;

[0034] Figure 4 This is a schematic diagram of an electronic device structure according to an embodiment of the present invention. Detailed Implementation

[0035] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments.

[0036] In the accompanying drawings, the thickness, size, and shape of the objects have been slightly exaggerated for ease of illustration. The drawings are for illustrative purposes only and are not drawn to scale.

[0037] It should also be understood that the terms "comprising," "including," "having," "containing," and / or "comprising," when used in this specification, indicate the presence of the stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or combinations thereof. Furthermore, when expressions such as "at least one of..." appear after a list of listed features, they modify the entire listed feature, not individual elements in the list. Additionally, when describing embodiments of this application, the word "may" is used to mean "one or more embodiments of this application." And the term "exemplary" is intended to refer to an example or illustration.

[0038] As used herein, the terms “basically,” “approximately,” and similar terms are used as terms of approximation rather than terms of degree, and are intended to describe inherent biases in measured or calculated values ​​that will be recognized by those skilled in the art.

[0039] Unless otherwise specified, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms (e.g., those defined in common dictionaries) shall be interpreted as having the meaning consistent with their meaning in the context of the relevant art and shall not be interpreted in an idealized or overly formal sense unless expressly so specified herein.

[0040] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0041] The steps described in the specification and the flowcharts in the accompanying drawings of this invention are not necessarily to be strictly followed according to the step numbers; the execution order of the steps can be changed. Furthermore, certain steps can be omitted, multiple steps can be combined into one step, and / or one step can be broken down into multiple steps.

[0042] The first embodiment of the present invention provides a target recognition method based on motion detection and local search acceleration, such as... Figure 1 As shown, it includes:

[0043] Step S1: Use an RGB camera to acquire the target video to be fused;

[0044] Step S2: For the initial frame image of the target video to be fused, identify the target to be fused;

[0045] Step S3: Based on the identified target to be fused, the latest frame image is evenly divided into several parts using pixel lines, so that the target to be fused will intersect with at least one of the several dividing lines when it moves. By comparing the RGB difference of the dividing lines in the latest two frames, it is determined whether the target to be fused is in motion. If it is in motion, proceed to step S4. If it is not in motion, read a new frame image and repeat step S3.

[0046] Step S4: Several rectangular regions generated when segmenting the current frame image using pixel lines are used. When the target to be fused moves, it will intersect with the segmentation lines of the rectangular regions. Based on whether the difference between the RGB pixel values ​​on the horizontal and vertical segmentation lines changes, the local search area of ​​the target to be fused is determined.

[0047] Step S5: Within the local search area, the target to be fused is identified. If all frames of the video have been processed, the target identification ends; otherwise, a new frame is read and the process proceeds to step 3.

[0048] For reference Figure 1 or Figure 2 The method provided in this embodiment will be described in detail step by step below.

[0049] Step S1: Use an RGB camera to acquire the target video to be fused.

[0050] In this embodiment, an RGB camera is required to capture the environmental target to be fused in real time in order to obtain a video stream including the target to be fused.

[0051] Step S2: For the initial frame image of the target video to be fused, identify the target to be fused.

[0052] In this embodiment, specifically for the initial frame of the video, the target to be fused can be identified using a target matching method based on scale-invariant feature transform (SIFT).

[0053] Step S3: Based on the identified target to be fused, the latest acquired frame image is evenly divided into several parts using pixel lines, so that when the target to be fused moves, it will intersect with at least one of the several dividing lines. By comparing the RGB difference of the dividing lines in the latest two acquired frames, it is determined whether the target to be fused is in motion. If it is in motion, proceed to step S4. If it is not in motion, read a new frame image and repeat step S3.

[0054] In this embodiment, the sum of RGB pixel values ​​on the horizontal and vertical dividing lines of two frames can be determined, and the difference between the sum of the RGB pixel values ​​between the two frames can be compared with a preset threshold. If the difference is greater than the threshold, the target to be fused is determined to be in motion.

[0055] Specifically, assuming the video image size is M×N, when it is uniformly divided, the number of horizontal dividing lines is P, and the number of vertical dividing lines is Q. Let p and q represent the p-th horizontal dividing line and the q-th vertical dividing line, respectively (p = 1, 2, ..., P, q = 1, 2, ..., Q). Then the ordinate of the p-th horizontal dividing line is... The x-coordinate of the qth vertical dividing line is The sum of the RGB pixel values ​​along all horizontal and vertical dividing lines is:

[0056]

[0057] Where R(i,j), G(i,j) and B(i,j) represent the red component, green component and blue component of the image at coordinates (i,j), respectively.

[0058] By calculating the sum of RGB pixel values ​​on the horizontal and vertical dividing lines of the keyframe image, the difference between the sum of these RGB pixel values ​​between two frames is compared with a pre-set threshold T. If the difference is greater than the threshold, the target to be fused is considered to have moved, and the process proceeds to step S4; otherwise, the target to be fused is considered to be still, a new frame of video image is read, and the process proceeds to step S3.

[0059] It should be noted that the terms "motion" and "stillness" here are relative conclusions obtained by comparing with a preset threshold, and do not mean that the target to be fused is in an absolute state of motion or stillness.

[0060] It is understandable that the threshold T can be reasonably configured and set according to the actual situation, and this article will not impose any restrictions on it.

[0061] Step S4: Several rectangular regions generated when segmenting the current frame image using pixel lines are used. When the target to be fused moves, it will intersect with the segmentation lines of the rectangular regions. Based on whether the difference between the RGB pixel values ​​on the horizontal and vertical segmentation lines changes, the local search area of ​​the target to be fused is determined.

[0062] In other words, if the pixel lines in the image segment the moving image to be fused, then the rectangular region where the segmentation line is located is taken as the local search region.

[0063] Step S5: Within the local search area, the target to be fused is identified. If all frames of the video have been processed, the target identification ends; otherwise, a new frame is read and the process proceeds to step 3.

[0064] In this embodiment, within the local search area, a target matching method based on SIFT features can be used to identify the target to be fused.

[0065] Compared with the prior art, this embodiment has at least the following advantages:

[0066] 1) This embodiment uses the difference between the sum of the RGB pixel values ​​of the horizontal and vertical dividing lines to quickly detect the motion state of the target to be fused. Specifically, the motion state detection of the target to be fused can be based on the dividing lines, which simplifies the processing flow when the target to be fused is stationary.

[0067] 2) This embodiment improves the speed of target recognition by determining the local search region of the target to be fused and identifying the target in the local search region.

[0068] The second embodiment of the present invention, corresponding to the first embodiment, introduces a target recognition device based on motion detection and local search acceleration, such as... Figure 3 As shown, it includes:

[0069] The acquisition unit is configured to acquire the target video to be fused using an RGB camera;

[0070] The first identification unit is configured to identify the target to be fused for the initial frame image of the target video to be fused;

[0071] The motion detection unit is configured to divide the most recently acquired frame image into several parts using pixel lines based on the identified target to be fused, so that the target to be fused will intersect with at least one of the several dividing lines when it moves. By comparing the RGB difference of the dividing lines in the two most recently acquired frames, it is determined whether the target to be fused is in motion. If it is in motion, it enters the local search unit. If it is not in motion, it reads a new frame image and repeats the processing of the motion detection unit.

[0072] The local search unit is configured to use pixel lines to segment several rectangular regions generated when the current frame image is segmented. When the target to be fused moves, it will intersect with the segmentation lines of the rectangular regions. Based on whether the difference between the RGB pixel values ​​on the horizontal and vertical segmentation lines changes, the local search region of the target to be fused is determined.

[0073] The second recognition unit is configured to recognize the target to be fused within the local search area. If all frames of the video have been processed, the target recognition ends; otherwise, a new frame is read and the process is transferred to the motion detection unit.

[0074] In this embodiment, the first identification unit is further configured to: identify the target to be fused using a target matching method based on scale-invariant feature transformation features for the initial frame of the video.

[0075] In this embodiment, the motion detection unit is further configured to: determine the sum of RGB pixel values ​​on the horizontal and vertical dividing lines of two frames, compare the difference between the sum of the RGB pixel values ​​between the two frames with a preset threshold, and if it is greater than the threshold, the target to be fused is determined to be in motion.

[0076] In this embodiment, the second identification unit is further configured to: identify the target to be fused using a target matching method based on SIFT features within the local search area.

[0077] A third embodiment of the present invention provides an electronic device, such as... Figure 4 As shown, it can be understood as a physical device, including a processor and a memory storing processor-executable instructions. When the instructions are executed by the processor, the following operations are performed:

[0078] Step S1: Use an RGB camera to acquire the target video to be fused;

[0079] Step S2: For the initial frame image of the target video to be fused, identify the target to be fused;

[0080] Step S3: Based on the identified target to be fused, the latest frame image is evenly divided into several parts using pixel lines, so that the target to be fused will intersect with at least one of the several dividing lines when it moves. By comparing the RGB difference of the dividing lines in the latest two frames, it is determined whether the target to be fused is in motion. If it is in motion, proceed to step S4. If it is not in motion, read a new frame image and repeat step S3.

[0081] Step S4: Several rectangular regions generated when segmenting the current frame image using pixel lines are used. When the target to be fused moves, it will intersect with the segmentation lines of the rectangular regions. Based on whether the difference between the RGB pixel values ​​on the horizontal and vertical segmentation lines changes, the local search area of ​​the target to be fused is determined.

[0082] Step S5: Within the local search area, the target to be fused is identified. If all frames of the video have been processed, the target identification ends; otherwise, a new frame is read and the process proceeds to step 3.

[0083] In the fourth embodiment of the present invention, the flow of the target recognition method based on motion detection and local search acceleration is the same as that of the first, second, or third embodiments. The difference lies in the engineering implementation: this embodiment can be implemented using software plus necessary general-purpose hardware platforms. While hardware implementation is also possible, the former is often a preferred method. Based on this understanding, the method of the present invention can be embodied in the form of a computer software product stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including several instructions to cause a device to execute the method of the embodiments of the present invention.

[0084] Through the description of specific embodiments, a more in-depth and specific understanding should be gained of the technical means and effects adopted by the present invention to achieve the intended purpose. However, the accompanying drawings are only provided for reference and illustration and are not intended to limit the present invention.

Claims

1. A target recognition method based on motion detection and local search acceleration, characterized in that, include: Step S1: Use an RGB camera to acquire the target video to be fused; Step S2: For the initial frame image of the target video to be fused, identify the target to be fused; Step S3: Based on the identified target to be fused, the latest frame image is evenly divided into several parts using pixel lines, so that the target to be fused will intersect with at least one of the several dividing lines when it moves. By comparing the RGB difference of the dividing lines in the latest two frames, it is determined whether the target to be fused is in motion. If it is in motion, proceed to step S4. If it is not in motion, read a new frame image and repeat step S3. Step S4: Utilize pixel lines to segment several rectangular regions generated during the current frame image segmentation. When the target to be fused moves, it will intersect with the segmentation lines of the rectangular regions. Based on whether the difference between the RGB pixel values ​​on the horizontal and vertical segmentation lines changes, determine the local search area of ​​the target to be fused. Step S5: Within the local search area, the target to be fused is identified. If all frames of the video have been processed, the target identification ends. Otherwise, read a new frame of image and proceed to step 3; Step S3 includes: By determining the sum of RGB pixel values ​​on the horizontal and vertical dividing lines of two frames, the difference between the sum of the RGB pixel values ​​between the two frames is compared with a preset threshold. If the difference is greater than the threshold, the target to be fused is determined to be in motion.

2. The target recognition method based on motion detection and local search acceleration according to claim 1, characterized in that, Step S2 includes: For the initial frame of the video, a target matching method based on scale-invariant feature transformation is used to identify the target to be fused.

3. The target recognition method based on motion detection and local search acceleration according to claim 1, characterized in that, Step S5 includes: Within the local search area, the target to be fused is identified using a target matching method based on SIFT features.

4. A target recognition device based on motion detection and local search acceleration, characterized in that, include: The acquisition unit is configured to acquire the target video to be fused using an RGB camera; The first identification unit is configured to identify the target to be fused for the initial frame image of the target video to be fused; The motion detection unit is configured to, based on the identified target to be fused, uniformly divide the most recently acquired frame image into several parts using pixel lines, so that when the target to be fused moves, it will intersect with at least one of the several dividing lines. By comparing the RGB difference between the dividing lines in the most recently acquired two frames, it determines whether the target to be fused is in motion. If it is in motion, it enters the local search unit; if it is not in motion, it reads a new frame image and repeats the processing of the motion detection unit. The motion detection unit is further configured to: determine the sum of RGB pixel values ​​on the horizontal and vertical dividing lines of the two frames, compare the difference between the sum of the RGB pixel values ​​of the two frames with a preset threshold; if it is greater than the threshold, the target to be fused is determined to be in motion. The local search unit is configured to use pixel lines to segment several rectangular regions generated when the current frame image is segmented. When the target to be fused moves, it will intersect with the segmentation lines of the rectangular regions. The local search region of the target to be fused is determined based on whether the difference between the RGB pixel values ​​on the horizontal and vertical segmentation lines changes. The second recognition unit is configured to recognize the target to be fused within the local search area. If all frames of the video have been processed, the target recognition ends; otherwise, a new frame is read and the process is transferred to the motion detection unit.

5. The target recognition device based on motion detection and local search acceleration according to claim 4, characterized in that, The first recognition unit is further configured to: for the initial frame of the video, use a target matching method based on scale-invariant feature transformation features to recognize the target to be fused.

6. The target recognition device based on motion detection and local search acceleration according to claim 4, characterized in that, The second identification unit is further configured to: identify the target to be fused using a target matching method based on SIFT features within the local search area.

7. An electronic device, characterized in that, The electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the target recognition method based on motion detection and local search acceleration as described in any one of claims 1 to 4.

8. A computer storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the target recognition method based on motion detection and local search acceleration as described in any one of claims 1 to 4.

Images