A motion target fusion processing method, device and computer readable storage medium

Abstract

The application discloses a kind of motion target fusion processing method, equipment and computer readable storage medium, wherein, the method comprises: in the process of shooting, the mask of the motion target area of current frame is acquired, and the minimum circumscribed rectangle of the mask is calculated;The feature descriptor of the edge of the minimum circumscribed rectangle is calculated;According to the feature descriptor, feature matching is carried out to obtain the highest similarity of the feature descriptor of motion target in the motion target area in the current frame with the coordinates;In the current frame, the motion target is fused into the highest similarity position according to the coordinates, to determine the freeze target of the motion target.An efficient motion target fusion processing scheme is realized, which effectively improves the extraction accuracy and fusion effect of the motion target, and enhances the user's shooting experience.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a moving target fusion processing method, device, and computer-readable storage medium. Background Technology

[0002] In existing technologies, with the continuous development of smart terminal devices, users' demands for the shooting functions of these devices are becoming increasingly diverse. In particular, for models with a wide range of camera applications, there is significant room for improvement in the accuracy of many specialized shooting applications. For example, a series of specialized shooting applications derived from the basic motion fusion scheme may extract incomplete moving targets in certain complex scenes, resulting in missing moving targets in the final extraction. Therefore, how to effectively improve the accuracy of moving target extraction and fusion effects has become an urgent technical problem to be solved. Summary of the Invention

[0003] To address the aforementioned technical deficiencies in the prior art, this invention proposes a moving target fusion processing method, which includes:

[0004] During the shooting process, the mask of the moving target region in the current frame is obtained, and the minimum bounding rectangle of the mask is calculated.

[0005] Calculate the feature descriptors of the edges of the minimum bounding rectangle;

[0006] Feature matching is performed based on the feature descriptor to obtain the coordinates in the current frame that have the highest similarity to the feature descriptor of the moving target in the moving target region;

[0007] In the current frame, the moving target is fused to the position with the highest similarity based on the coordinates to determine the stationary target of the moving target.

[0008] Optionally, the step of acquiring the mask of the moving target region in the current frame during shooting and calculating the minimum bounding rectangle of the mask includes:

[0009] During the shooting process, a preset model is established based on the acquired video content;

[0010] The mask of the moving target region in the current frame is obtained according to the model.

[0011] Optionally, calculating the feature descriptor of the edge of the minimum bounding rectangle includes:

[0012] The outer background transition region is determined in the mask based on the minimum bounding rectangle;

[0013] Obtain the feature descriptors of the minimum bounding rectangle and the background transition region.

[0014] Optionally, the step of performing feature matching based on the feature descriptor to obtain the coordinates in the current frame that have the highest similarity to the feature descriptor of the moving target region includes:

[0015] Using a preset Hamming distance as a metric, matching is performed between bounding rectangle features based on the feature descriptors;

[0016] The freeze coordinates of the freeze target in the current frame are determined based on the feature descriptor with the highest similarity.

[0017] Optionally, in the current frame, fusing the moving target to the position with the highest similarity based on the coordinates to determine the stationary target of the moving target includes:

[0018] Based on the stationary target of the moving target and the minimum bounding rectangle, the moving target and the background image in the current frame are fused together;

[0019] The fused image is denoted as Image = bg * (1 - mask_min_rect) + fg * mask_min_rect, where bg is the background image, mask_min_rect is the minimum bounding rectangle, and fg is the foreground image in the current frame.

[0020] Optionally, the step of fusing the moving target to the position with the highest similarity based on the coordinates in the current frame to determine the stationary target of the moving target further includes:

[0021] Obtain the intersection of the minimum bounding rectangle and the mask of the moving target in the current frame;

[0022] The intersection is denoted as Lambda_mask = mask_min_rect ∩ mask, where mask is the mask of the moving target.

[0023] Optionally, the step of fusing the moving target to the position with the highest similarity based on the coordinates in the current frame to determine the stationary target of the moving target further includes:

[0024] Detect whether the intersection exists;

[0025] When the intersection is detected, the minimum bounding rectangle is optimized.

[0026] Optionally, the step of fusing the moving target to the position with the highest similarity based on the coordinates in the current frame to determine the stationary target of the moving target further includes:

[0027] When optimizing the minimum bounding rectangle, the intersection of the mask of the moving target and the minimum bounding rectangle is deleted from the minimum bounding rectangle.

[0028] The minimum bounding rectangle after optimization is determined to be Mask_min_rect = mask_min_rect - (mask_min_rect ∩ mask).

[0029] The present invention also proposes a moving target fusion processing device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the moving target fusion processing method as described in any of the preceding claims.

[0030] The present invention also proposes a computer-readable storage medium storing a moving target fusion processing program, which, when executed by a processor, implements the steps of the moving target fusion processing method as described in any of the preceding claims.

[0031] The present invention provides a moving target fusion processing method, apparatus, and computer-readable storage medium. During shooting, a mask of the moving target region in the current frame is acquired, and the minimum bounding rectangle of the mask is calculated. Feature descriptors of the edges of the minimum bounding rectangle are calculated. Feature matching is performed based on the feature descriptors to obtain the coordinates in the current frame that have the highest similarity to the feature descriptors of the moving target region. In the current frame, the moving target is fused to the position with the highest similarity based on the coordinates to determine the stationary target of the moving target. This achieves an efficient moving target fusion processing scheme, effectively improving the extraction accuracy and fusion effect of moving targets, and enhancing the user's shooting experience. Attached Figure Description

[0032] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0033] Figure 1 This is a schematic diagram of the hardware structure of a mobile terminal according to the present invention;

[0034] Figure 2 This is the first flowchart of the moving target fusion processing method of the present invention;

[0035] Figure 3 This is the second flowchart of the moving target fusion processing method of the present invention;

[0036] Figure 4 This is the third flowchart of the motion target fusion processing method of the present invention;

[0037] Figure 5 This is the fourth flowchart of the motion target fusion processing method of the present invention;

[0038] Figure 6 This is the fifth flowchart of the motion target fusion processing method of the present invention;

[0039] Figure 7 This is the sixth flowchart of the motion target fusion processing method of the present invention;

[0040] Figure 8 This is the seventh flowchart of the motion target fusion processing method of the present invention;

[0041] Figure 9 This is the eighth flowchart of the motion target fusion processing method of the present invention;

[0042] Figure 10 It is a complex scene mask image of the moving target fusion processing method of the present invention;

[0043] Figure 11 This is a schematic diagram illustrating the missing motion targets in the motion target fusion processing method of the present invention;

[0044] Figure 12 This is a schematic diagram of the minimum bounding rectangle of the motion target fusion processing method of the present invention;

[0045] Figure 13 This is a schematic diagram of motion target extraction in the motion target fusion processing method of the present invention;

[0046] Figure 14 This is a schematic diagram of the fixed target determination in the motion target fusion processing method of the present invention. Detailed Implementation

[0047] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0048] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustrative purposes and has no specific meaning in itself. Therefore, "module," "part," or "unit" may be used interchangeably.

[0049] Terminals can be implemented in various forms. For example, the terminals described in this invention may include mobile terminals such as mobile phones, tablets, laptops, handheld computers, personal digital assistants (PDAs), portable media players (PMPs), navigation devices, wearable devices, smart bracelets, pedometers, etc., as well as fixed terminals such as digital TVs and desktop computers.

[0050] The following description will use a mobile terminal as an example. Those skilled in the art will understand that, apart from elements specifically designed for mobile purposes, the construction according to embodiments of the present invention can also be applied to fixed-type terminals.

[0051] Please see Figure 1 This is a schematic diagram of the hardware structure of a mobile terminal implementing various embodiments of the present invention. The mobile terminal 100 may include: an RF (Radio Frequency) unit 101, a WiFi module 102, an audio output unit 103, an A / V (Audio / Video) input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, a processor 110, and a power supply 111, etc. Those skilled in the art will understand that... Figure 1 The mobile terminal structure shown does not constitute a limitation on the mobile terminal. The mobile terminal may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0052] The following is combined Figure 1 A detailed introduction to each component of the mobile terminal:

[0053] The radio frequency unit 101 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with the processor 110; additionally, it transmits uplink data to the base station. Typically, the radio frequency unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier, and a duplexer. Furthermore, the radio frequency unit 101 can also communicate wirelessly with networks and other devices. The aforementioned wireless communications may use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution), and TDD-LTE (Time Division Duplexing-Long Term Evolution).

[0054] WiFi is a short-range wireless transmission technology. Mobile terminals using the WiFi module 102 can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 1 WiFi module 102 is shown, but it is understood that it is not a necessary component of a mobile terminal and can be omitted as needed without changing the nature of the invention.

[0055] The audio output unit 103 can convert audio data received by the radio frequency unit 101 or the WiFi module 102 or stored in the memory 109 into audio signals and output them as sound when the mobile terminal 100 is in call signal receiving mode, call mode, recording mode, voice recognition mode, broadcast receiving mode, etc. Furthermore, the audio output unit 103 can also provide audio output related to specific functions performed by the mobile terminal 100 (e.g., call signal receiving sound, message receiving sound, etc.). The audio output unit 103 may include a speaker, a buzzer, etc.

[0056] The A / V input unit 104 is used to receive audio or video signals. The A / V input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042. The GPU 1041 processes image data of still images or videos acquired by an image capture device (such as a camera) in video capture mode or image capture mode. The processed image frames can be displayed on the display unit 106. The image frames processed by the GPU 1041 can be stored in the memory 109 (or other storage medium) or transmitted via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) in operating modes such as telephone call mode, recording mode, and voice recognition mode, and can process such sound into audio data. The processed audio (voice) data can be converted into a format that can be transmitted to a mobile communication base station via the radio frequency unit 101 in telephone call mode. The microphone 1042 can implement various types of noise cancellation (or suppression) algorithms to eliminate (or suppress) noise or interference generated during the reception and transmission of audio signals.

[0057] The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 1061 according to the ambient light level, and the proximity sensor can turn off the display panel 1061 and / or backlight when the mobile terminal 100 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, tapping), etc. Other sensors that may be configured in the phone, such as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0058] The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0059] User input unit 107 can be used to receive input numerical or character information, and generate key signal inputs related to user settings and function control of the mobile terminal. Specifically, user input unit 107 may include touch panel 1071 and other input devices 1072. Touch panel 1071, also known as touch screen, can collect touch operations on or near the user (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near touch panel 1071), and drive corresponding connection devices according to a pre-set program. Touch panel 1071 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, sends it to processor 110, and can receive and execute commands from processor 110. In addition, touch panel 1071 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1071, the user input unit 107 may also include other input devices 1072. Specifically, other input devices 1072 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc., without being limited here.

[0060] Furthermore, the touch panel 1071 may cover the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits the information to the processor 110 to determine the type of touch event. Subsequently, the processor 110 provides corresponding visual output on the display panel 1061 based on the type of touch event. Although in Figure 1 In this embodiment, the touch panel 1071 and the display panel 1061 are two independent components to realize the input and output functions of the mobile terminal. However, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated to realize the input and output functions of the mobile terminal. The specific implementation is not limited here.

[0061] Interface unit 108 serves as an interface through which at least one external device can connect to mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, and so on. Interface unit 108 may be used to receive input (e.g., data, power, etc.) from the external device and transmit the received input to one or more elements within mobile terminal 100, or it may be used to transmit data between mobile terminal 100 and the external device.

[0062] The memory 109 can be used to store software programs and various data. The memory 109 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback, image playback, etc.), etc.; the data storage area may store data created based on the use of the mobile phone (such as audio data, phonebook, etc.). Furthermore, the memory 109 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0063] The processor 110 is the control center of the mobile terminal. It connects various parts of the mobile terminal via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 109, and by calling data stored in the memory 109, it performs various functions and processes data of the mobile terminal, thereby providing overall monitoring of the mobile terminal. The processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 110.

[0064] The mobile terminal 100 may also include a power supply 111 (such as a battery) that supplies power to various components. Preferably, the power supply 111 can be logically connected to the processor 110 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0065] although Figure 1 As not shown, the mobile terminal 100 may also include a Bluetooth module, etc., which will not be described in detail here.

[0066] Based on the above-described mobile terminal hardware structure, various embodiments of the method of the present invention are proposed.

[0067] Figure 2 This is a first flowchart of the moving target fusion processing method of the present invention. This embodiment proposes a moving target fusion processing method, which includes:

[0068] S1. During the shooting process, obtain the mask of the moving target region in the current frame and calculate the minimum bounding rectangle of the mask;

[0069] S2. Calculate the feature descriptors of the edges of the minimum bounding rectangle;

[0070] S3. Perform feature matching based on the feature descriptor to obtain the coordinates with the highest similarity to the feature descriptor of the moving target in the moving target region in the current frame;

[0071] S4. In the current frame, the moving target is fused to the position with the highest similarity according to the coordinates to determine the stationary target of the moving target.

[0072] In this embodiment, please refer to Figure 10 This figure shows the mask image of the extracted moving target in some complex scenes. Please refer to it. Figure 11 Because the accuracy of the extracted moving target mask image is poor, the final extracted human body region shows serious deviations. Therefore, this embodiment will optimize the accuracy of the extracted moving target mask image. Specifically, firstly, by modeling the video content, a moving target mask is obtained, and the minimum bounding rectangle of the mask is calculated. The minimum bounding rectangle corresponds to the image content, such as... Figure 12 The rectangular frame shown; then, the features of the minimum bounding rectangle are extracted, such as... Figure 13 The process involves obtaining the minimum bounding rectangle of the moving target, extracting feature descriptors from the minimum bounding rectangle region and the background transition region, then matching the bounding rectangle features using Hamming distance as a metric to calculate the stationary target's coordinates in the current frame image, and finally optimizing the stationary target mask and the moving target mask in the current frame.

[0073] The beneficial effect of this embodiment is that, during the shooting process, a mask of the moving target region in the current frame is obtained, and the minimum bounding rectangle of the mask is calculated; feature descriptors of the edges of the minimum bounding rectangle are calculated; feature matching is performed based on the feature descriptors to obtain the coordinates with the highest similarity to the feature descriptors of the moving target in the moving target region in the current frame; in the current frame, the moving target is fused to the position with the highest similarity based on the coordinates to determine the stationary target of the moving target. This achieves an efficient moving target fusion processing scheme, effectively improving the extraction accuracy and fusion effect of moving targets, and enhancing the user's shooting experience.

[0074] Alternatively, please refer to Figure 3 The step of acquiring the mask of the moving target region in the current frame during shooting and calculating the minimum bounding rectangle of the mask includes:

[0075] S11. During the shooting process, a preset model is established based on the acquired video content;

[0076] S12. Obtain the mask of the moving target region of the current frame according to the model.

[0077] Optionally, refer to Figure 4 , calculating the feature descriptor of the edge of the minimum bounding rectangle includes:

[0078] S21. Determine the external background transition region in the mask according to the minimum bounding rectangle;

[0079] S22. Obtain the feature descriptors of the minimum bounding rectangle and the background transition region.

[0080] In this embodiment, a specific descriptor is used to describe the edge of the minimum bounding rectangle, that is, the bounding box region of the minimum bounding rectangle; optionally, various types of descriptors can be adopted. Among them, taking the ORB (Oriented Fast and Rotated Brief, an image processing detection method) feature as an example, the ORB feature points first rely on the FAST (Features from accelerated segment test, corner detection) algorithm to detect feature points; in this embodiment, after obtaining the feature points, the attributes of these feature points are described in a certain way. Optionally, in this embodiment, the output of these attributes is called the descriptor of the feature point. In this embodiment, based on ORB and using the BRIEF algorithm to calculate the descriptor of a feature point. Among them, the core idea of the BRIEF algorithm is to select N pairs of points around the key point P in a certain pattern, and combine the comparison results of these N pairs of points as the feature descriptor.

[0081] Specifically, draw a circle O with the key point P as the center and d as the radius. Select N pairs of points in a certain pattern within the circle O. For the convenience of explanation, N = 4, and in practical applications, N can take 512. Suppose the currently selected 4 pairs of points are respectively marked as shown in the above figure: P1(A,B), P2(A,B), P3(A,B), P4(A,B); define the operation (I_A and I_B represent the gray values of points A and B): T(P(A,B)) = 1 when I_A > I_B; T(P(A,B)) = 0 when I_A < I_B. After operating on the selected points, the following result combination is obtained: T(P1(A,B)) = 1, T(P2(A,B)) = 0; T(P3(A,B)) = 1, T(P3(A,B)) = 1, then the final ORB descriptor is: 1011.

[0082] Optionally, refer to Figure 5 , performing feature matching according to the feature descriptor to obtain the coordinate with the highest similarity to the feature descriptor of the moving target in the moving target region in the current frame, including:

[0083] S31. Using the preset Hamming distance as a metric, perform matching between bounding rectangle features based on the feature descriptors;

[0084] S32. Determine the frame coordinates of the framed target in the current frame based on the feature descriptor with the highest similarity.

[0085] In this embodiment, as described in the example above, assuming that the feature of the smallest circumscribed quadrilateral obtained in the second step is F_A, as follows... Figure 14 As shown in the small quadrilateral 1011 on the right, the same quadrilateral features are obtained in the current image using the same method, such as... Figure 14 The small quadrilaterals 1011, 1111, and 1101 on the left are named F_B, F_C, and F_D.

[0086] In this embodiment, Hamming distance is used as the unit of measurement. Hamming distance is the number of different values ​​between two vectors. It is usually used to compare two binary strings of equal length. The nearest features of F_B, F_C, F_D and F_A are calculated, and the position of the nearest feature is used as the coordinates of F_A.

[0087] Alternatively, please refer to Figure 6 The step of fusing the moving target to the position with the highest similarity based on the coordinates in the current frame to determine the stationary target of the moving target includes:

[0088] S41. Based on the stationary target of the moving target and the minimum bounding rectangle, fuse the moving target and the background image in the current frame.

[0089] S42. The fused image is denoted as Image = bg * (1 - mask_min_rect) + fg * mask_min_rect, where bg is the background image, mask_min_rect is the minimum bounding rectangle, and fg is the foreground image in the current frame.

[0090] Alternatively, please refer to Figure 7 The step of fusing the moving target to the position with the highest similarity based on the coordinates in the current frame to determine the stationary target of the moving target further includes:

[0091] S43. Obtain the intersection of the minimum bounding rectangle and the mask of the moving target in the current frame;

[0092] S44. The intersection is denoted as Lambda_mask = mask_min_rect ∩ mask, where mask is the mask of the moving target.

[0093] Alternatively, please refer to Figure 8 The step of fusing the moving target to the position with the highest similarity based on the coordinates in the current frame to determine the stationary target of the moving target further includes:

[0094] S45. Detect whether the intersection exists;

[0095] S46. When the intersection is detected, the minimum bounding rectangle is optimized.

[0096] Alternatively, please refer to Figure 9 The step of fusing the moving target to the position with the highest similarity based on the coordinates in the current frame to determine the stationary target of the moving target further includes:

[0097] S47. When optimizing the minimum bounding rectangle, delete the intersection of the mask of the moving target and the minimum bounding rectangle in the minimum bounding rectangle;

[0098] S48. Determine the minimum bounding rectangle after optimization as Mask_min_rect = mask_min_rect - (mask_min_rect ∩ mask).

[0099] Based on the above embodiments, the present invention also proposes a moving target fusion processing device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the moving target fusion processing method as described in any of the above embodiments.

[0100] It should be noted that the above-described device embodiments and method embodiments belong to the same concept. The specific implementation process can be found in the method embodiments, and the technical features in the method embodiments are also applicable to the device embodiments, which will not be repeated here.

[0101] Based on the above embodiments, the present invention also proposes a computer-readable storage medium storing a moving target fusion processing program, which, when executed by a processor, implements the steps of the moving target fusion processing method as described in any of the above embodiments.

[0102] It should be noted that the above-described medium embodiments and method embodiments belong to the same concept. The specific implementation process can be found in the method embodiments, and the technical features in the method embodiments are also applicable to the medium embodiments, which will not be repeated here.

[0103] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0104] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0105] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0106] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A method for processing fusion of moving targets, characterized in that, The method includes: During the shooting process, the mask of the moving target region in the current frame is obtained, and the minimum bounding rectangle of the mask is calculated. Calculate the feature descriptors of the edges of the minimum bounding rectangle; Feature matching is performed based on the feature descriptor to obtain the coordinates in the current frame that have the highest similarity to the feature descriptor of the moving target in the moving target region; In the current frame, the moving target is fused to the position with the highest similarity based on the coordinates to determine the stationary target of the moving target; in, Based on the stationary target of the moving target and the minimum bounding rectangle, the moving target and the background image in the current frame are fused, and the fused image is denoted as Image=bg*(1-mask_min_rect)+fg*mask_min_rect, where bg is the background image, mask_min_rect is the minimum bounding rectangle, and fg is the foreground image in the current frame; Obtain the intersection of the minimum bounding rectangle and the mask of the moving target in the current frame, and denote the intersection as Lambda_mask=mask_min_rect∩mask, where mask is the mask of the moving target; When the intersection is detected, the minimum bounding rectangle is optimized. In the minimum bounding rectangle, the intersection of the mask of the moving target and the minimum bounding rectangle is deleted, and the minimum bounding rectangle after optimization is determined to be Mask_min_rect = mask_min_rect - (mask_min_rect ∩ mask).

2. The moving target fusion processing method according to claim 1, characterized in that, The step of acquiring the mask of the moving target region in the current frame during shooting and calculating the minimum bounding rectangle of the mask includes: During the shooting process, a preset model is established based on the acquired video content; The mask of the moving target region in the current frame is obtained according to the model.

3. The moving target fusion processing method according to claim 2, characterized in that, The calculation of the feature descriptor of the edge of the minimum bounding rectangle includes: The outer background transition region is determined in the mask based on the minimum bounding rectangle; Obtain the feature descriptors of the minimum bounding rectangle and the background transition region.

4. The moving target fusion processing method according to claim 3, characterized in that, The step of performing feature matching based on the feature descriptor to obtain the coordinates in the current frame that have the highest similarity to the feature descriptor of the moving target in the moving target region includes: Using a preset Hamming distance as a metric, matching is performed between bounding rectangle features based on the feature descriptors; The freeze coordinates of the freeze target in the current frame are determined based on the feature descriptor with the highest similarity.

5. A moving target fusion processing device, characterized in that, The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the moving target fusion processing method as described in any one of claims 1 to 4.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a moving target fusion processing program, which, when executed by a processor, implements the steps of the moving target fusion processing method as described in any one of claims 1 to 4.

Images