Image processing method, device and electronic equipment

Abstract

The embodiment of the present invention discloses an image processing method, device and electronic device. The method comprises: obtaining a key frame image containing a target object, and obtaining a frame image to be processed containing the target object; extracting feature points in the key frame image and feature points in the frame image to be processed respectively; determining the matching relationship between each feature point in the key frame image and each feature point in the frame image to be processed; determining a transformation relationship based on the feature points in the key frame image and the feature points in the frame image to be processed that have a matching relationship; processing the frame image to be processed based on the transformation relationship to obtain a target frame image.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Singapore patent application No. 10201913798W, entitled “IMAGE PROCESSING METHOD AND APPARATUS, AND ELECTRONIC DEVICE”, filed on December 30, 2019, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present invention relates to, but is not limited to, image processing technology, and in particular to an image processing method, device and electronic equipment. Background Art

[0004] In the field of image recognition, the identification and positioning of objects in images is one of the problems. The camera that obtains the image source will inevitably be disturbed by vibration, causing the object to move in the image, which has a great impact on the identification and positioning of the object. Summary of the invention

[0005] Embodiments of the present invention provide an image processing method, an image processing device, and an electronic device.

[0006] The technical solution of the embodiment of the present invention is achieved as follows:

[0007] An embodiment of the present invention provides an image processing method, the method comprising:

[0008] Obtaining a key frame image containing a target object, and obtaining a frame image to be processed containing the target object;

[0009] Respectively extracting one or more feature points in the key frame image and one or more feature points in the frame image to be processed;

[0010] Determine the matching relationship between each feature point in the key frame image and each feature point in the frame image to be processed;

[0011] Determine a transformation relationship based on the feature points in the key frame image and the feature points in the frame image to be processed that have a matching relationship;

[0012] The frame image to be processed is processed based on the transformation relationship to obtain a target frame image.

[0013] In the above solution, the step of determining the matching relationship between each feature point in the key frame image and each feature point in the frame image to be processed includes:

[0014] Obtain a first group of pixel pairs associated with a first feature point in the key frame image, and obtain a second group of pixel pairs associated with a second feature point in the frame image to be processed; wherein the first feature point is a feature point in the key frame image; the second feature point is a feature point in the frame image to be processed; the relative position relationship between the first pixel pair in the first group of pixel pairs and the first feature point is consistent with the relative position relationship between the second pixel pair in the second group of pixel pairs and the second feature point; the first group of pixel pairs includes at least one group of first pixel pairs; the second group of pixel pairs includes at least one group of second pixel pairs;

[0015] Comparing pixel values ​​of two pixels included in the first pixel pair to obtain a first comparison result, and comparing pixel values ​​of two pixels included in the second pixel pair to obtain a second comparison result;

[0016] Determining the similarity between the first group of pixel pairs and the second group of pixel pairs based on at least one first comparison result corresponding to the at least one group of first pixel pairs and at least one second comparison result corresponding to the at least one group of second pixel pairs;

[0017] When the similarity reaches a first preset threshold, it is determined that the first feature point and the second feature point satisfy a matching relationship.

[0018] In the above scheme, the determining the similarity between the first group of pixel pairs and the second group of pixel pairs based on at least one first comparison result corresponding to the at least one group of first pixel pairs and at least one second comparison result corresponding to the at least one group of second pixel pairs includes:

[0019] generating a first character string based on the at least one first comparison result;

[0020] generating a second character string based on the at least one second comparison result;

[0021] A first Hamming distance is determined based on the first character string and the second character string, and a similarity between the first group of pixel pairs and the second group of pixel pairs is determined based on the first Hamming distance.

[0022] In the above scheme, the determining of the transformation relationship based on the feature points in the key frame image with matching relationship and the feature points in the frame image to be processed comprises: screening a feature point pair set to obtain a target feature point pair set; wherein each feature point pair is composed of a feature point in the key frame image with matching relationship and a feature point in the frame image to be processed, and the feature point pair set comprises a plurality of feature point pairs;

[0023] A transformation relationship is determined based on the target feature point pairs in the target feature point pair set.

[0024] In the above scheme, the screening of the feature point pair set to obtain the target feature point pair set includes: obtaining the similarity between the feature points in the key frame image and the feature points in the frame image to be processed contained in each feature point pair in the feature point pair set;

[0025] Determine a maximum value of similarities corresponding to all feature point pairs in the feature point pair set, and determine a first threshold based on the maximum value;

[0026] The feature point pairs whose corresponding similarities are less than the first threshold value are removed from the feature point pair set to obtain a target feature point pair set.

[0027] In the above solution, the extracting the feature points in the key frame image and the feature points in the frame image to be processed respectively includes:

[0028] Determine a plurality of second pixel points within a preset area range of a first pixel point in the key frame image or the frame image to be processed;

[0029] Determining whether the grayscale values ​​of the plurality of second pixels are different from the grayscale value of the first pixel;

[0030] When the grayscale values ​​of the plurality of second pixels are different from the grayscale value of the first pixel, the first pixel is determined to be a feature point in the key frame image or the frame image to be processed.

[0031] In the above scheme, the transformation relationship is determined based on the target feature point pairs in the target feature point pair set, including: determining the transformation matrix based on the coordinates of the feature points in the key frame image contained in the target feature point pairs in the target feature point pair set and the coordinates of the feature points in the frame image to be processed.

[0032] In the above scheme, the processing of the frame image to be processed based on the transformation relationship to obtain the target frame image includes: transforming the target area including the target object in the frame image to be processed based on the transformation matrix to obtain the target frame image.

[0033] In the above scheme, after obtaining the target frame image, the method further includes: extracting feature points in the target frame image;

[0034] Determine a first loss value based on feature points in the key frame image and feature points in the target frame image; the feature points in the key frame image used to determine the first loss value are different from the feature points in the key frame image used to determine the transformation relationship;

[0035] When the first loss value does not exceed the second threshold, the obtained target frame image is retained.

[0036] In the above scheme, the method also includes: when the first loss value exceeds a second threshold, deleting the obtained target frame image.

[0037] The embodiment of the present invention further provides an image processing device, which includes: an image acquisition unit, a feature extraction unit, a matching unit and a transformation unit; wherein:

[0038] The image acquisition unit is used to obtain a key frame image containing a target object, and obtain a frame image to be processed containing the target object;

[0039] The feature extraction unit is used to extract one or more feature points in the key frame image and one or more feature points in the frame image to be processed respectively;

[0040] The matching unit is used to determine the matching relationship between each feature point in the key frame image and each feature point in the frame image to be processed;

[0041] The transformation unit is used to determine a transformation relationship based on feature points in the key frame image having a matching relationship and feature points in the frame image to be processed; and process the frame image to be processed based on the transformation relationship to obtain a target frame image.

[0042] In the above scheme, the matching unit is used to obtain a first group of pixel pairs associated with a first feature point in the key frame image, and to obtain a second group of pixel pairs associated with a second feature point in the frame image to be processed; wherein the first feature point is a feature point in the key frame image; the second feature point is a feature point in the frame image to be processed; the relative position relationship between the first pixel pair in the first group of pixel pairs and the first feature point is consistent with the relative position relationship between the second pixel pair in the second group of pixel pairs and the second feature point; the first group of pixel pairs includes at least one group of first pixel pairs; the second group of pixel pairs includes at least one group of second pixel pairs; compare the pixel values ​​of two pixels included in the first pixel pair to obtain a first comparison result, and compare the pixel values ​​of two pixels included in the second pixel pair to obtain a second comparison result; determine the similarity between the first group of pixel pairs and the second group of pixel pairs based on at least one first comparison result corresponding to the at least one group of first pixel pairs and at least one second comparison result corresponding to the at least one group of second pixel pairs; and determine that the first feature point and the second feature point satisfy a matching relationship when the similarity reaches a first preset threshold.

[0043] In the above scheme, the matching unit is used to determine the similarity between the first group of pixel points and the second group of pixel points based on at least one first comparison result corresponding to the at least one group of first pixel points and at least one second comparison result corresponding to the at least one group of second pixel points, and is used to: generate a first string based on the multiple first comparison results; generate a second string based on the multiple second comparison results; determine a first Hamming distance based on the first string and the second string, and determine the similarity between the first group of pixel points and the second group of pixel points based on the first Hamming distance.

[0044] In the above scheme, the device further includes a screening unit, which is used to screen the feature point pair set to obtain a target feature point pair set; wherein each feature point pair is composed of a feature point in the key frame image and a feature point in the frame image to be processed that have a matching relationship, and the feature point pair set includes multiple groups of feature point pairs;

[0045] The transformation unit is used to determine the transformation relationship based on the feature points in the key frame image with a matching relationship and the feature points in the frame image to be processed, and is used to: determine the transformation relationship based on the target feature point pairs in the target feature point pair set obtained by the screening unit.

[0046] In the above scheme, the screening unit is used to obtain the similarity between the feature points in the key frame image contained in each feature point pair in the feature point pair set and the feature points in the frame image to be processed; determine the maximum value of the similarities corresponding to all feature point pairs in the feature point pair set, and determine a first threshold based on the maximum value; remove the feature point pairs in the feature point pair set whose corresponding similarities are less than the first threshold, and obtain the target feature point pair set.

[0047] In the above scheme, the feature extraction unit is used to determine multiple second pixel points within a preset area range of the first pixel point in the key frame image or the frame image to be processed; judge whether the grayscale values ​​of the multiple second pixel points are different from the grayscale values ​​of the first pixel point; when the grayscale values ​​of the multiple second pixel points are different from the grayscale values ​​of the first pixel point, determine that the first pixel point is a feature point in the key frame image or the frame image to be processed.

[0048] In the above scheme, the transformation unit is used to determine the transformation relationship based on the target feature point pairs in the target feature point pair set, and is used to: determine the transformation matrix based on the coordinates of the feature points in the key frame image contained in the target feature point pairs in the target feature point pair set and the coordinates of the feature points in the frame image to be processed.

[0049] In the above scheme, the transformation unit is used to process the frame image to be processed based on the transformation relationship to obtain the target frame image, and is used to: transform the target area containing the target object in the frame image to be processed based on the transformation matrix to obtain the target frame image.

[0050] In the above solution, the device further includes a determination unit and a processing unit; wherein,

[0051] The feature extraction unit is further used to extract feature points in the target frame image;

[0052] The determining unit is used to determine a first loss value based on feature points in the key frame image and feature points in the target frame image; the feature points in the key frame image used to determine the first loss value are different from the feature points in the key frame image used to determine the transformation relationship;

[0053] The processing unit is used to retain the obtained target frame image when the first loss value does not exceed a second threshold.

[0054] In the above solution, the processing unit is further used to delete the obtained target frame image when the first loss value exceeds a second threshold.

[0055] The embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the method described in the embodiment of the present invention are implemented.

[0056] An embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the embodiment of the present invention when executing the program.

[0057] The image processing method, device and electronic device provided by the embodiment of the present invention include: obtaining a key frame image containing a target object, and obtaining a frame image to be processed containing the target object; extracting feature points in the key frame image and feature points in the frame image to be processed respectively; determining the matching relationship between each feature point in the key frame image and each feature point in the frame image to be processed; determining a transformation relationship based on the feature points in the key frame image and the feature points in the frame image to be processed with the matching relationship; processing the frame image to be processed based on the transformation relationship to obtain the target frame image. The technical solution of the embodiment of the present invention is adopted, and the frame image to be processed is processed by the obtained transformation relationship, so that the viewing angle of the target area of ​​the target object in the obtained target frame image is basically consistent with the viewing angle of the target area of ​​the target object in the key frame image, that is, the relative position relationship between the device for collecting the target frame image and the target object in the target frame image is basically consistent with the relative position relationship between the device for collecting the key frame image and the target object in the key frame image, thereby solving the problem of displacement of the target object in the collected image caused by camera vibration interference or camera displacement, and realizing accurate positioning of the target object. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 FIG. 1 is a flow chart of an image processing method according to an embodiment of the present invention. Figure 1 ;

[0059] Figure 2 The process diagram of the image processing method according to the embodiment of the present invention is as follows: Figure 2 ;

[0060] Figure 3 A schematic diagram of selecting pixel pairs in an image processing method according to an embodiment of the present invention;

[0061] Figure 4 The process diagram of the image processing method according to the embodiment of the present invention is as follows: Figure 3 ;

[0062] Figure 5 The structure of the image processing device according to the embodiment of the present invention is shown in FIG. Figure 1 ;

[0063] Figure 6 The structure of the image processing device according to the embodiment of the present invention is shown in FIG. Figure 2 ;

[0064] Figure 7 The structure of the image processing device according to the embodiment of the present invention is shown in FIG. Figure 3 ;

[0065] Figure 8The figure is a schematic diagram of the hardware structure of the electronic device according to the embodiment of the present invention. DETAILED DESCRIPTION

[0066] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0067] An embodiment of the present invention provides an image processing method. Figure 1 The process diagram of the image processing method according to the embodiment of the present invention is as follows: Figure 1 ;like Figure 1 As shown, the method includes:

[0068] Step 101: obtaining a key frame image containing a target object, and obtaining a frame image to be processed containing the target object;

[0069] Step 102: extracting one or more feature points in the key frame image and one or more feature points in the frame image to be processed respectively;

[0070] Step 103: determining a matching relationship between each feature point in the key frame image and each feature point in the frame image to be processed;

[0071] Step 104: determining a transformation relationship based on the feature points in the key frame image and the feature points in the frame image to be processed that have a matching relationship;

[0072] Step 105: Process the frame image to be processed based on the transformation relationship to obtain a target frame image.

[0073] In this embodiment, the key frame image is an image containing a complete, clear, and unobstructed target object. The frame image to be processed is an image containing the target object. Wherein, both the key frame image and the frame image to be processed may include at least one target object.

[0074] In this embodiment, the key frame image and the frame image to be processed may be images captured by the same image acquisition device. In other embodiments, the key frame image and the frame image to be processed may also be images captured by different image acquisition devices. It can be understood that the frame image to be processed may be multiple frame images; some frame images in the multiple frame images may be captured by the first image acquisition device, and some frame images may be captured by the second image acquisition device, and the layout positions of the first image acquisition device and the second image acquisition device are relatively close, or the relative position relationship between the first image acquisition device and the captured area and the relative position relationship between the second image acquisition device and the captured area are similar.

[0075] In an optional embodiment of the present invention, the key frame image may be a top view of the target object. That is, the image acquisition device for acquiring the key frame image is arranged above the target object, and multiple frames of images are acquired by the image acquisition device arranged above the target object, and an image containing a clear, complete and unobstructed target object is selected from the multiple frames of images as the key frame image. Correspondingly, the frame image to be processed may also be a top view of the target object.

[0076] As an example, a video containing the target object can be captured by an image capture device set above the target object, and a frame image that is clear, unobstructed and has a complete target object is selected from the video as a key frame image, and other frame images in the video except the key frame image are used as images to be processed.

[0077] In an optional embodiment of the present invention, the target object may be a game table. The image recognition method of this embodiment may be used to identify the target object (e.g., the game table) in the key frame image and the frame image to be processed, and align the target object in the frame image to be processed toward the position of the target object in the key frame image, thereby locating the target object, facilitating subsequent image detection and recognition, and being applicable to the research and development of intelligent operation and monitoring of games. It is understandable that the target object is not limited to the game table, and may also be other objects as the target object, which is not limited in this embodiment.

[0078] In this embodiment, the feature points in the key frame image and the feature points in the frame image to be processed can be corner points in the image. The corner points are extreme points, that is, points with particularly prominent attributes in some aspects, such as points with the maximum or minimum intensity in certain attributes. For example, the intersection of the edges of two or more lines, or the points with the largest rate of change of the gradient value and gradient direction in the image, etc. Since the corner points can effectively reduce the amount of data while retaining the important features in the image, this embodiment can improve the calculation speed by extracting the corner points in the image and using the corner points as feature points for subsequent matching and determination of transformation relationships, making subsequent real-time processing possible. It can be understood that the feature points in the image extracted in this embodiment are not limited to corner points, but can also be other types of feature points, such as key points.

[0079] In this embodiment, feature points in the key frame image and feature points in the frame image to be processed may be extracted by methods based on grayscale images, methods based on binary images, methods based on contour curves, and the like. As an implementation, taking the method based on grayscale images to extract feature points in an image (e.g., extracting corner points in an image) as an example, edge feature points in the image may be determined by calculating the gradient in the image, edge curves may be determined based on the edge feature points, and corner points in the image may be determined by calculating the curvature of the edge curve. It is to be understood that this embodiment is not limited to the above-mentioned methods for extracting feature points (e.g., corner points) in an image, and any corner point detection method may be applicable to the embodiments of the present invention.

[0080] In this embodiment, in the key frame image and the frame image to be processed, the feature points in the key frame image and the feature points in the frame image to be processed that have a matching relationship can specifically be that the feature of the first feature point in the key frame image (for example, an attribute in a certain aspect) matches the feature of the second feature point in the frame image to be processed (for example, an attribute in a certain aspect); wherein, the feature point in the key frame image can be recorded as a first feature point (there can be more than one), and the feature point in the frame image to be processed can be recorded as a second feature point (there can be more than one).

[0081] In an optional embodiment of the present invention, a transformation matrix can be determined based on the coordinates of the first feature point and the second feature point having a matching relationship, and the transformation matrix is ​​used to transform the acquisition perspective of the frame image to be processed to be the same as the acquisition perspective of the key frame image. However, due to some reasons, for example, the first feature point and the second feature point obtained do not completely have a matching relationship, that is, the obtained feature point pair has a certain error; for example, the determined transformation relationship cannot strictly transform the acquisition perspective of the frame image to be processed to be the same as the acquisition perspective of the key frame image, that is, the determined transformation relationship has a certain error. Therefore, in this embodiment, it is expected that the acquisition perspective of the frame image to be processed will be transformed to be the same as the acquisition perspective of the key frame image, but it is very likely to be converted into an image that is similar to the acquisition perspective of the key frame image but different, so the converted image is called the target frame image; the acquisition perspective of the target frame image may be consistent with the acquisition perspective of the key frame image, or it may be different from the acquisition perspective of the key frame image.

[0082] By adopting the technical solution of the embodiment of the present invention, the frame image to be processed is processed through the obtained transformation relationship, so that the viewing angle of the target area of ​​the target object in the obtained target frame image is basically consistent with the viewing angle of the target area of ​​the target object in the key frame image, thereby solving the problem of displacement of the target object in the captured image due to camera vibration interference or camera displacement, and realizing accurate positioning of the target object.

[0083] Based on the above embodiments, the embodiments of the present invention further provide an image processing method. Figure 2 The process diagram of the image processing method according to the embodiment of the present invention is as follows: Figure 2 ;like Figure 2 As shown, the method includes:

[0084] Step 201: obtaining a key frame image containing a target object, and obtaining a frame image to be processed containing the target object;

[0085] Step 202: extracting feature points in the key frame image and feature points in the frame image to be processed respectively;

[0086] Step 203: obtaining a first group of pixel pairs associated with a first feature point in the key frame image, and obtaining a second group of pixel pairs associated with a second feature point in the frame image to be processed; wherein the first feature point is a feature point in the key frame image; the second feature point is a feature point in the frame image to be processed; the relative position relationship between the first pixel pair in the first group of pixel pairs and the first feature point is consistent with the relative position relationship between the second pixel pair in the second group of pixel pairs and the second feature point; the first group of pixel pairs includes at least one group of first pixel pairs; the second group of pixel pairs includes at least one group of second pixel pairs;

[0087] Step 204: comparing pixel values ​​of two pixels included in the first pixel pair to obtain a first comparison result, and comparing pixel values ​​of two pixels included in the second pixel pair to obtain a second comparison result;

[0088] Step 205: determining the similarity between the first group of pixel pairs and the second group of pixel pairs based on at least one first comparison result corresponding to the at least one group of first pixel pairs and at least one second comparison result corresponding to the at least one group of second pixel pairs;

[0089] Step 206: When the similarity reaches a first preset threshold, determining that the first feature point and the second feature point satisfy a matching relationship;

[0090] Step 207: Screening the feature point pair set to obtain a target feature point pair set; wherein each feature point pair is composed of a feature point in the key frame image and a feature point in the frame image to be processed that have a matching relationship, and the feature point pair set includes multiple groups of feature point pairs;

[0091] Step 208: determining a transformation relationship based on the target feature point pairs in the target feature point pair set;

[0092] Step 209: Based on the transformation relationship, a transformation process is performed on the target area corresponding to the target object in the frame image to be processed to obtain a target frame image.

[0093] The detailed description of steps 201 to 202 of this embodiment can be specifically referred to the description of steps 101 to 102 in the aforementioned embodiment, which will not be repeated here.

[0094] In an optional embodiment of the present invention, in step 202 of this embodiment, the extracting of feature points in the key frame image and feature points in the frame image to be processed respectively includes: determining a plurality of second pixel points within a preset area range of a first pixel point in the key frame image or the frame image to be processed; judging whether the grayscale values ​​of the plurality of second pixel points are different from the grayscale value of the first pixel point; and, when the grayscale values ​​of the plurality of second pixel points are different from the grayscale value of the first pixel point, determining that the first pixel point is a feature point in the key frame image or the frame image to be processed.

[0095] In this embodiment, the method for extracting feature points in the key frame image and the frame image to be processed can adopt the above method, that is, convert both the key frame image and the frame image to be processed into grayscale images; for each pixel point, within the preset area with the selected pixel point as the center and the preset pixel distance as the radius, determine whether the grayscale values ​​of other pixels within the preset area are different from the grayscale value of the selected pixel point. For example, select any pixel point P in the key frame image, draw a circle with the pixel point P as the center and the 3 pixel distance as the radius to obtain a circular area; if the grayscale value of the pixel point in the circular area is greater than or less than the grayscale value of the pixel point P, the pixel point P can be determined as a feature point. It can be understood that the preset area range in this embodiment is not limited to a circular area range, but can also be an area range of other shapes.

[0096] It can be understood that the feature points in the key frame image and the frame image to be processed can be extracted in the above-mentioned manner, or different feature point extraction methods can be used respectively, which is not limited in this embodiment.

[0097] In this embodiment, each feature point in the frame image to be processed (which can be recorded as the second feature point) is matched with the feature point in the key frame image (which can be recorded as the first feature point), specifically, the second group of pixel pairs associated with each second feature point in the frame image to be processed is selected to match with the first group of pixel pairs associated with each first feature point in the key frame image; wherein, both the first group of pixel pairs and the second group of pixel pairs can include at least one group of pixel pairs, and each pixel pair can include two pixels. The first group of pixel pairs includes at least one group of pixel pairs selected within a preset area centered on the associated first feature point; correspondingly, the second group of pixel pairs includes at least one group of pixel pairs selected within the preset area centered on the associated second feature point.

[0098] Among them, if each first pixel point pair in the first group of pixel point pairs is a pixel point pair randomly selected within the preset area, then the relative position relationship between each second pixel point pair in the second group of pixel point pairs and the feature points associated therewith is consistent with the relative position relationship between each pixel point pair in the first group of pixel point pairs and the feature points associated therewith.

[0099] Figure 3 Schematic diagram of selecting pixel pairs in the image processing method according to an embodiment of the present invention; Figure 3 As shown, Figure 3 Each square in represents a pixel, assuming Figure 3 The pixel point P in the key frame image is a feature point, recorded as the first feature point P in the key frame image, and the first group of pixel pairs associated with the first feature point P is selected according to a preset rule (for example, randomly). Figure 3 The first group of pixel pairs shown in may include three groups of pixel pairs, and the three groups of pixel pairs can be randomly selected in a circular area with the first feature point P as the center and the preset pixel distance as the radius. Accordingly, in order to determine the second feature point that satisfies the matching relationship with the first feature point P, in the frame image to be processed, for each feature point, a second group of pixel pairs that are consistent with the relative position relationship between each group of pixel pairs in the first group of pixel pairs and the associated first feature point is determined, that is, the second pixel pairs are determined according to the relative position relationship between each pixel pair in the first group of pixel pairs and the associated first feature point, that is, for each feature point in the frame image to be processed, a second group of pixel pairs that are consistent with the relative position relationship between each pixel pair in the first group of pixel pairs and the associated first feature point is selected. Figure 3 The pixel pairs shown in and the associated feature points have the same relative position relationship. For example, for each feature point in the frame image to be processed, similarly select Figure 3As shown in , pixel 1 is 3 pixels away from the feature point in the vertical upward direction, and pixel 2 is 1 pixel away from the feature point in the vertical downward direction and 2 pixels away from the feature point in the horizontal left direction, and the pixel pair consisting of pixel 1 and pixel 2 is used as a group of pixel pairs in the frame image to be processed. It can be understood that the corresponding first pixel pair and second pixel pair in the first group of pixel pairs and the second group of pixel pairs adopt the same selection rule, that is, the relative position relationship between the two pixels included in the first pixel pair and the associated first feature point is the same as the relative position relationship between the two pixels included in the corresponding second pixel pair and the associated second feature point (i.e., a feature point in the frame image to be processed).

[0100] In an optional embodiment of the present invention, the first feature point may be a feature point in the target area of ​​the key frame image that includes the target object; correspondingly, the second feature point may also be a feature point in the target area of ​​the frame image to be processed that includes the target object. It can be understood that in this embodiment, matching can be performed on the feature points in the target area of ​​the target object, while ignoring the feature points in other areas except the target area in the key frame image and the frame image to be processed, which can reduce the amount of data processing.

[0101] In step 204 of this embodiment, for at least one first pixel pair in the first group of pixel pairs, the pixel values ​​of two pixels in each pixel pair are compared respectively, and then for at least one first pixel pair in the first group of pixel pairs, at least one first comparison result can be obtained. Correspondingly, for at least one second pixel pair in the second group of pixel pairs, at least one second comparison result can be obtained. In some embodiments, the grayscale values ​​of the two pixels included in the first pixel pair can also be compared to obtain a first comparison result represented by the difference in grayscale values; correspondingly, a second comparison result represented by the difference in grayscale values ​​is obtained.

[0102] For example, from top to bottom and from left to right, the pixel value or grayscale value of the upper or left pixel point can be used as the subtrahend, and the pixel value or grayscale value of the lower or right pixel point can be used as the minuend. It can be understood that the comparison rule of the two pixel points in this embodiment is not limited to the above, and the first comparison result and the second comparison result use the same comparison rule.

[0103] In an optional embodiment of the present invention, in step 205 of this embodiment, the similarity between the first group of pixel pairs and the second group of pixel pairs is determined based on at least one first comparison result corresponding to the at least one group of first pixel pairs and at least one second comparison result corresponding to the at least one group of second pixel pairs, including: generating a first string based on the at least one first comparison result; generating a second string based on the at least one second comparison result; determining a first Hamming distance based on the first string and the second string, and determining the similarity between the first group of pixel pairs and the second group of pixel pairs based on the first Hamming distance.

[0104] In this embodiment, the first group of pixel pairs and the second group of pixel pairs can be used as feature descriptors of the associated feature points, respectively. In this embodiment, whether the corresponding first feature point and the second feature point satisfy the matching relationship can be determined by determining the similarity between the two feature descriptors. It can be understood that by determining whether the pixel pairs around the corresponding two feature points in the key frame image and the frame image to be processed are similar, it is determined whether the corresponding two feature points in the key frame image and the frame image to be processed match.

[0105] As an example, at least one first comparison result corresponding to the first group of pixel pairs is arranged in order and converted into a binary string; correspondingly, at least one second comparison result corresponding to the second group of pixel pairs is arranged in order and converted into another binary string, and the similarity between the first group of pixel pairs and the second group of pixel pairs is determined based on the two obtained binary strings. The arrangement order of the at least one first comparison result is consistent with the arrangement order of the at least one second comparison result. Figure 3 Taking the three groups of pixel pairs shown in FIG. 1 as an example, three first comparison results and three second comparison results can be obtained respectively; the three first comparison results and the three second comparison results are sorted according to the same sorting rule; for example, Figure 3 The comparison result of the two pixels corresponding to "1" shown in the figure is taken as the first comparison result. Figure 3 The comparison result of the two pixels corresponding to "2" shown in the figure is used as the second comparison result. Figure 3 The comparison results of the two pixels corresponding to "3" shown in the figure are used as the third comparison result, the first comparison result is sorted in the high position, the second comparison result is sorted in the middle position, and the third comparison result is sorted in the low position. It can be understood that the sorting rule of the comparison results in this embodiment is not limited to the above, and the at least one first comparison result and the at least one second comparison result use the same sorting rule.

[0106] In this embodiment, as an example, the Hamming distance between two binary strings may be calculated, and the similarity between the first group of pixel pairs and the second group of pixel pairs may be determined based on the Hamming distance. It is understood that other distances may also be used in this embodiment to determine the similarity between the first group of pixel pairs and the second group of pixel pairs, which is not limited in this embodiment.

[0107] In this embodiment, when the calculated similarity between the first set of pixel pairs and the second set of pixel pairs reaches the first preset threshold, it can be determined that the first feature point and the second feature point satisfy the matching relationship. However, the feature point pair formed by the first feature point and the second feature point satisfying the matching relationship may not be correct, and there may be a mismatch, so it is necessary to screen all feature point pairs that satisfy the matching relationship and contain the first feature point and the second feature point in step 207.

[0108] In some optional embodiments, with respect to step 207 of this embodiment, the feature point pair set is screened to obtain a target feature point pair set, including: obtaining the similarity between the feature points in the key frame image contained in each feature point pair in the feature point pair set and the feature points in the frame image to be processed; determining the maximum value of the similarities corresponding to all feature point pairs in the feature point pair set, and determining a first threshold based on the maximum value; and removing the feature point pairs in the feature point pair set whose corresponding similarities are less than the first threshold to obtain a target feature point pair set.

[0109] In this embodiment, the similarity of each group of feature point pairs in the feature point pair set can be determined, that is, the similarity between the feature points in the key frame image and the feature points in the frame image to be processed contained in each group of feature point pairs is determined; wherein, the similarity of each group of feature point pairs is the similarity of the first group of pixel point pairs and the second group of pixel point pairs mentioned above, and the minimum value of the similarity corresponding to each feature point pair in the feature point pair set is determined. For example, the feature point pair set may include three groups of feature point pairs, respectively recorded as: A1 and A2, B1 and B2, C1 and C2; wherein, the similarity of B1 and B2 is the largest, and the maximum value of the similarity is recorded as k, then the first threshold value can be determined based on k, the first threshold value is less than k, that is, the first threshold value is less than the maximum value of the similarity; if the similarity of A1 and A2 is less than the first threshold value, then A1 and A2 can be determined as a wrongly matched feature point pair, and A1 and A2 can be deleted from the feature point pair set; if the similarity of C1 and C2 is greater than the first threshold value, then it can be indicated that C1 and C2 are correctly matched feature point pairs. Based on this, a target feature point pair set is obtained, and the target feature point pair set may include at least one group of target feature point pairs.

[0110] In some embodiments, the similarity corresponding to the feature point pair in this embodiment is also the similarity between the first group of pixel pairs and the second group of pixel pairs corresponding to the feature point pair. When the similarity between the first group of pixel pairs and the second group of pixel pairs is expressed by the Hamming distance (such as the first Hamming distance), if the first Hamming distance is larger, it can be indicated that the similarity between the first group of pixel pairs and the second group of pixel pairs is smaller, and if the first Hamming distance is smaller, it can be indicated that the similarity between the first group of pixel pairs and the second group of pixel pairs is larger. Based on this, in this embodiment, the first threshold value can also be determined based on the first Hamming distance corresponding to the first group of pixel pairs and the second group of pixel pairs. For example, the feature point pair set may include three groups of feature point pairs, which are respectively recorded as: A1 and A2, B1 and B2, C1 and C2; among them, the Hamming distance between B1 and B2 is the smallest, which is recorded as k, then it can be determined that the first threshold value is greater than k, for example, the first threshold value is 2k. If the Hamming distance corresponding to A1 and A2 is greater than the first threshold (i.e. 2k), it can be determined that A1 and A2 are incorrectly matched feature point pairs, and A1 and A2 are deleted from the feature point pair set; if the similarity between C1 and C2 is less than or equal to the first threshold (i.e. 2k), it can be indicated that C1 and C2 are correctly matched feature point pairs.

[0111] In step 208 of this embodiment, the transformation relationship is determined based on the target feature point pairs in the target feature point pair set, including: determining the transformation matrix based on the coordinates of the feature points in the key frame image contained in the target feature point pairs in the target feature point pair set and the coordinates of the feature points in the frame image to be processed.

[0112] In this embodiment, the transformation matrix can be determined based on the first coordinates of the first feature point corresponding to the key frame image and the second coordinates of the second feature point corresponding to the frame image to be processed contained in the target feature point pair set. As an implementation method, the transformation matrix can be a homography matrix, then the transformation matrix is ​​determined based on the first coordinates of the first feature point corresponding to the key frame image and the second coordinates of the second feature point corresponding to the frame image to be processed contained in the target feature point pair set, and the first coordinates of the first feature point corresponding to the key frame image and the second coordinates of the second feature point corresponding to the frame image to be processed contained in the target feature point pair set are homographically transformed to obtain a homography matrix; the homography transformation is used to describe the transformation relationship from one plane to another plane. For example, the coordinates of a feature point of the target object in the key frame image are (x1, y1); the coordinates of the same feature point of the target object in the frame image to be processed are (x2, y2), then through the homography transformation, the coordinates (x2, y2) can be transformed to the coordinates (x1, y1).

[0113] In step 209 of this embodiment, the processing of the frame image to be processed based on the transformation relationship to obtain the target frame image includes: transforming the target area including the target object in the frame image to be processed based on the transformation matrix to obtain the target frame image.

[0114] For example, if the transformation matrix is ​​denoted by H, the frame image to be processed is denoted by q, and the key frame image is denoted by Q, then q = H·Q. Then the target frame image = q·H -1 By processing the changes of the frame image to be processed through the transformation matrix, the difference between the acquisition angle of the frame image to be processed and the acquisition angle of the key frame image caused by the movement of the image acquisition device when acquiring the frame image to be processed is eliminated or weakened, so that the acquisition angle of the target frame image is the same or substantially the same as the acquisition angle of the key frame image.

[0115] By adopting the technical solution of the embodiment of the present invention, the frame image to be processed is processed through the obtained transformation relationship, so that the acquisition angle of the target frame image is the same or substantially the same as the acquisition angle of the key frame image, thereby solving the problem of displacement of the target object in the captured image due to camera vibration interference or camera displacement, and realizing accurate positioning of the target object.

[0116] The embodiment of the present invention also provides an image processing method. Figure 4 The process diagram of the image processing method according to the embodiment of the present invention is as follows: Figure 3 ;like Figure 4 As shown, the method includes:

[0117] Step 301: obtaining a key frame image containing a target object, and obtaining a frame image to be processed containing the target object;

[0118] Step 302: extracting feature points in the key frame image and feature points in the frame image to be processed respectively;

[0119] Step 303: determining a matching relationship between each feature point in the key frame image and each feature point in the frame image to be processed;

[0120] Step 304: determining a transformation relationship based on the feature points in the key frame image and the feature points in the frame image to be processed that have a matching relationship;

[0121] Step 305: Processing the frame image to be processed based on the transformation relationship to obtain a target frame image;

[0122] Step 306: extracting feature points in the target frame image, and determining a first loss value based on the feature points in the key frame image and the feature points in the target frame image; the feature points in the key frame image used to determine the first loss value are different from the feature points in the key frame image used to determine the transformation relationship;

[0123] Step 307: When the first loss value does not exceed the second threshold, retain the obtained target frame image; when the first loss value exceeds the second threshold, delete the obtained target frame image.

[0124] The detailed description of steps 301 to 305 of this embodiment can refer to the description of steps 101 to 105 and steps 201 to 207 in the aforementioned embodiments, which will not be repeated here.

[0125] In step 306 of this embodiment, the method for extracting feature points in the target frame image can refer to the description of the method for extracting feature points in the key frame image and the frame image to be processed in the above-mentioned embodiment, which will not be repeated here.

[0126] In this embodiment, the feature points in the key frame image used to determine the first loss value are different from the feature points in the key frame image used to determine the transformation relationship. Exemplarily, if there are 100 feature points extracted from the key frame image, and 50 of them are selected for determining the transformation relationship, then at least some of the other 50 feature points can be selected for determining the first loss value. Accordingly, among the feature points extracted from the target frame image, feature points that match the feature points in the key frame image used to determine the first loss value are selected.

[0127] In step 306 of this embodiment, the determining of the first loss value based on the feature points in the key frame image and the feature points in the target frame image includes: determining multiple groups of feature point pairs that satisfy a matching relationship based on the coordinates of the feature points in the key frame image and the coordinates of the feature points in the target frame image; each group of feature point pairs in the multiple groups of feature point pairs includes a feature point in the key frame image and a feature point in the target frame image; and determining the first loss value based on the coordinates of the two feature points included in each group of feature point pairs. The first loss value can represent the degree of difference between the target frame image and the key frame image.

[0128] In some embodiments, determining the first loss based on the coordinates of the two feature points included in each set of feature point pairs includes: determining a first distance based on the coordinates of the two feature points included in each set of feature point pairs; obtaining an average of multiple first distances, and determining the first loss based on the average. The first distance may be an Euler distance, a Euclidean distance, etc., which is not limited in this embodiment.

[0129] In this embodiment, since the position of the target object in the target frame image is the same as or similar to the position of the target object in the key frame image, the coordinates of each feature point in the target frame image can be compared with the coordinates of each feature point in the key frame image to obtain a feature point pair with a matching relationship. In other embodiments, the records of steps 203 to 206 in the aforementioned embodiment can also be used to determine the feature point pair that satisfies the matching relationship, which will not be repeated here.

[0130] In this embodiment, if the first loss value exceeds the second threshold, it may indicate that the transformation relationship (or transformation matrix) is incorrect, or the change processing of the frame image to be processed is incorrect. In this case, the target frame image can be deleted, that is, the target frame image will not be processed in the subsequent image processing process; an alarm message can also be output to achieve the purpose of automatically determining whether the transformation is successful and processing the corresponding target frame image.

[0131] In this embodiment, if the first loss value does not exceed the second threshold value, it indicates that the transformation relationship (or transformation matrix) is relatively accurate, and the transformation processing of the frame image to be processed is successful. The target frame image can be retained, and the target area corresponding to other target objects in the frame image to be processed can be further transformed through the transformation relationship (or transformation matrix) to obtain the position of the target area corresponding to the other target object in the key frame image.

[0132] When the image processing method of the present embodiment is applied to a scenario in which the image of a gaming table is adaptively transformed, since the positions and angles of the cameras deployed on different gaming tables cannot be guaranteed to be completely consistent, nor can the positions of the target areas corresponding to the target objects (e.g., gaming tables) in the key frame images corresponding to each gaming table be guaranteed to be completely consistent, after the cameras at each gaming table are deployed, the image processing method of the present embodiment can be used to determine a transformation relationship (transformation matrix) to transform the image captured by each camera to the same viewing angle as that of the same key frame image, so that the images captured by each camera are transformed into images in which the target objects are in the same position, so as to facilitate subsequent detection and recognition of the target objects and other processing.

[0133] The embodiment of the present invention further provides an image processing device. Figure 5 The structure of the image processing device according to the embodiment of the present invention is shown in FIG. Figure 1 ;like Figure 5 As shown, the device includes: an image acquisition unit 41, a feature extraction unit 42, a matching unit 43 and a transformation unit 44; wherein,

[0134] The image acquisition unit 41 is used to obtain a key frame image containing a target object, and obtain a frame image to be processed containing the target object;

[0135] The feature extraction unit 42 is used to extract feature points in the key frame image and feature points in the frame image to be processed respectively;

[0136] The matching unit 43 is used to determine the matching relationship between each feature point in the key frame image and each feature point in the frame image to be processed;

[0137] The transformation unit 44 is used to determine a transformation relationship based on the feature points in the key frame image having a matching relationship and the feature points in the frame image to be processed; and process the frame image to be processed based on the transformation relationship to obtain a target frame image.

[0138] In an optional embodiment of the present invention, the matching unit 43 is used to obtain a first group of pixel pairs associated with a first feature point in the key frame image, and to obtain a second group of pixel pairs associated with a second feature point in the frame image to be processed; wherein the first feature point is a feature point in the key frame image; the second feature point is a feature point in the frame image to be processed; the relative position relationship between the first pixel pair in the first group of pixel pairs and the first feature point is consistent with the relative position relationship between the second pixel pair in the second group of pixel pairs and the second feature point; the first group of pixel pairs includes at least one group of first pixel pairs; the second group of pixel pairs includes at least one group of second pixel pairs; compare the pixel values ​​of two pixels included in the first pixel pair to obtain a first comparison result, and compare the pixel values ​​of two pixels included in the second pixel pair to obtain a second comparison result; determine the similarity between the first group of pixel pairs and the second group of pixel pairs based on at least one first comparison result corresponding to the at least one group of first pixel pairs and at least one second comparison result corresponding to the at least one group of second pixel pairs; when the similarity reaches a first preset threshold, determine that the first feature point and the second feature point satisfy a matching relationship.

[0139] In an optional embodiment of the present invention, the matching unit 43 is used to determine the similarity between the first group of pixel pairs and the second group of pixel pairs based on at least one first comparison result corresponding to the at least one group of first pixel pairs and at least one second comparison result corresponding to the at least one group of second pixel pairs, and is used to: generate a first string based on the at least one first comparison result; generate a second string based on the at least one second comparison result; determine a first Hamming distance based on the first string and the second string, and determine the similarity between the first group of pixel pairs and the second group of pixel pairs based on the first Hamming distance.

[0140] In an optional embodiment of the present invention, Figure 6 As shown, the device further includes a screening unit 45, which is used to screen the feature point pair set to obtain a target feature point pair set; wherein each feature point pair is composed of a feature point in the key frame image and a feature point in the frame image to be processed that have a matching relationship, and the feature point pair set includes multiple groups of feature point pairs;

[0141] The transformation unit 44 is used to determine the transformation relationship based on the feature points in the key frame image with a matching relationship and the feature points in the frame image to be processed, and is used to: determine the transformation relationship based on the target feature point pairs in the target feature point pair set obtained by the screening unit 45.

[0142] In an optional embodiment of the present invention, the screening unit 45 is used to obtain the similarity between the feature points in the key frame image contained in each feature point pair in the feature point pair set and the feature points in the frame image to be processed; determine the maximum value of the similarities corresponding to all feature point pairs in the feature point pair set, and determine a first threshold based on the maximum value; remove the feature point pairs in the feature point pair set whose corresponding similarities are less than the first threshold to obtain a target feature point pair set.

[0143] In an optional embodiment of the present invention, the feature extraction unit 42 is used to determine multiple second pixel points within a preset area range of the first pixel point in the key frame image or the frame image to be processed; determine whether the grayscale values ​​of the multiple second pixel points are different from the grayscale values ​​of the first pixel point; and when the grayscale values ​​of the multiple second pixel points are different from the grayscale values ​​of the first pixel point, determine that the first pixel point is a feature point in the key frame image or the frame image to be processed.

[0144] In an optional embodiment of the present invention, the transformation unit 44 is used to determine the transformation relationship based on the feature points in the key frame image with a matching relationship and the feature points in the frame image to be processed, and is used to: determine the transformation matrix based on the coordinates of the feature points in the key frame image contained in the target feature point pairs in the target feature point pair set and the coordinates of the feature points in the frame image to be processed.

[0145] In an optional embodiment of the present invention, the transformation unit 44 is used to process the frame image to be processed based on the transformation relationship to obtain the target frame image, and is used to: transform the target area of ​​the frame image to be processed containing the target object based on the transformation matrix to obtain the target frame image.

[0146] In an optional embodiment of the present invention, Figure 7As shown, the device further includes a determination unit 46 and a processing unit 47; wherein,

[0147] The feature extraction unit 42 is further used to extract feature points in the target frame image;

[0148] The determining unit 46 is used to determine a first loss value based on feature points in the key frame image and feature points in the target frame image; the feature points in the key frame image used to determine the first loss value are different from the feature points in the key frame image used to determine the transformation relationship;

[0149] The processing unit 47 is configured to retain the obtained target frame image when the first loss value does not exceed a second threshold.

[0150] In an optional embodiment of the present invention, the processing unit 47 is further configured to delete the obtained target frame image when the first loss exceeds a second threshold.

[0151] In the embodiment of the present invention, the image acquisition unit 41, the feature extraction unit 42, the matching unit 43, the transformation unit 44, the screening unit 45, the determination unit 46 and the processing unit 47 in the device can all be implemented by a central processing unit (CPU), a digital signal processor (DSP), a microcontroller unit (MCU) or a programmable gate array (FPGA) in practical applications.

[0152] It should be noted that: when the image processing device provided in the above embodiment performs image processing, only the division of the above program modules is used as an example. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the above-described processing. In addition, the image processing device provided in the above embodiment and the image processing method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0153] An embodiment of the present invention further provides an electronic device, Figure 8 FIG. 1 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Figure 8 As shown, the electronic device 50 includes a memory 52, a processor 51, and a computer program stored in the memory 52 and executable on the processor 51. When the processor 51 executes the program, the steps of the image processing method described in the embodiment of the present invention are implemented.

[0154] It is understood that the various components in the electronic device 50 of this embodiment can be coupled together via the bus system 53. It is understood that the bus system 53 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 53 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 8 Various buses are labeled as bus system 53.

[0155] It can be understood that the memory 52 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAM bus random access memory (DRRAM, Direct Rambus Random Access Memory).The memory 52 described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0156] The method disclosed in the above embodiment of the present invention can be applied to the processor 51, or implemented by the processor 51. The processor 51 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 51 or the instruction in the form of software. The above processor 51 can be a general-purpose processor, a DSP, or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 51 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiment of the present invention. The general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor can be combined to execute. The software module can be located in a storage medium, which is located in the memory 52. ​​The processor 51 reads the information in the memory 52 and completes the steps of the above method in combination with its hardware.

[0157] In an exemplary embodiment, the electronic device 50 may be implemented by one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), FPGAs, general purpose processors, controllers, MCUs, microprocessors, or other electronic components to execute the aforementioned method.

[0158] In an exemplary embodiment, the present invention also provides a computer-readable storage medium, such as a memory 52 including a computer program, and the computer program can be executed by a processor 51 of an electronic device 50 to complete the steps of the aforementioned method. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disk, or CD-ROM.

[0159] The embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the image processing method described in the embodiment of the present invention are implemented.

[0160] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0161] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0162] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0163] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium, which, when executed, executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, disks or optical disks.

[0164] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0165] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. An image processing method, characterized in that: The method comprises: Obtaining a key frame image containing a target object, and obtaining a frame image to be processed containing the target object; Respectively extracting one or more feature points in the key frame image and one or more feature points in the frame image to be processed; Determine the matching relationship between each feature point in the key frame image and each feature point in the frame image to be processed; Screening the feature point pair set to obtain a target feature point pair set to exclude the situation where the matching relationship is incorrect; wherein each feature point pair is composed of a feature point in the key frame image and a feature point in the frame image to be processed that have a matching relationship, and the feature point pair set includes multiple groups of feature point pairs; Determining a transformation relationship based on the target feature point pairs in the target feature point pair set; the transformation relationship is used to transform the acquisition viewing angle of the frame image to be processed to be the same as or substantially the same as the acquisition viewing angle of the key frame image; Processing the frame image to be processed based on the transformation relationship to obtain a target frame image; After obtaining the target frame image, the method further includes: Extracting feature points in the target frame image; Determine a first loss value based on feature points in the key frame image and feature points in the target frame image; the feature points in the key frame image used to determine the first loss value are different from the feature points in the key frame image used to determine the transformation relationship; the first loss value is used to characterize the degree of difference between the target frame image and the key frame image; When the first loss value does not exceed a second threshold, retaining the obtained target frame image; When the first loss value exceeds a second threshold, the obtained target frame image is deleted.

2. The method according to claim 1, characterized in that: Determining the matching relationship between each feature point in the key frame image and each feature point in the frame image to be processed includes: Obtain a first group of pixel pairs associated with a first feature point in the key frame image, and obtain a second group of pixel pairs associated with a second feature point in the frame image to be processed; wherein the first feature point is a feature point in the key frame image; the second feature point is a feature point in the frame image to be processed; the relative position relationship between the first pixel pair in the first group of pixel pairs and the first feature point is consistent with the relative position relationship between the second pixel pair in the second group of pixel pairs and the second feature point; the first group of pixel pairs includes at least one group of first pixel pairs; the second group of pixel pairs includes at least one group of second pixel pairs; Comparing pixel values ​​of two pixels included in the first pixel pair to obtain a first comparison result, and comparing pixel values ​​of two pixels included in the second pixel pair to obtain a second comparison result; Determining the similarity between the first group of pixel pairs and the second group of pixel pairs based on at least one first comparison result corresponding to the at least one group of first pixel pairs and at least one second comparison result corresponding to the at least one group of second pixel pairs; When the similarity reaches a first preset threshold, it is determined that the first feature point and the second feature point satisfy a matching relationship.

3. The method according to claim 2, characterized in that The determining the similarity between the first group of pixel pairs and the second group of pixel pairs based on at least one first comparison result corresponding to the at least one group of first pixel pairs and at least one second comparison result corresponding to the at least one group of second pixel pairs comprises: generating a first character string based on the at least one first comparison result; generating a second character string based on the at least one second comparison result; A first Hamming distance is determined based on the first character string and the second character string, and a similarity between the first group of pixel pairs and the second group of pixel pairs is determined based on the first Hamming distance.

4. The method according to claim 1, characterized in that The step of screening the feature point pair set to obtain the target feature point pair set includes: Obtaining the similarity between the feature points in the key frame image and the feature points in the frame image to be processed contained in each feature point pair in the feature point pair set; Determine a maximum value of similarities corresponding to all feature point pairs in the feature point pair set, and determine a first threshold based on the maximum value; The feature point pairs whose corresponding similarities are less than the first threshold value are removed from the feature point pair set to obtain a target feature point pair set.

5. The method according to any one of claims 1 to 4, characterized in that: The extracting the feature points in the key frame image and the feature points in the frame image to be processed respectively includes: Determine a plurality of second pixel points within a preset area range of a first pixel point in the key frame image or the frame image to be processed; Determining whether the grayscale values ​​of the plurality of second pixels are different from the grayscale value of the first pixel; When the grayscale values ​​of the plurality of second pixels are different from the grayscale value of the first pixel, the first pixel is determined to be a feature point in the key frame image or the frame image to be processed.

6. The method according to claim 1, characterized in that The determining of the transformation relationship based on the target feature point pairs in the target feature point pair set includes: A transformation matrix is ​​determined based on the coordinates of the feature points in the key frame image contained in the target feature point pairs in the target feature point pair set and the coordinates of the feature points in the frame image to be processed.

7. The method according to claim 6, characterized in that The step of processing the frame image to be processed based on the transformation relationship to obtain a target frame image includes: The target region including the target object in the frame image to be processed is transformed based on the transformation matrix to obtain a target frame image.

8. An image processing device, characterized in that: The device comprises: an image acquisition unit, a feature extraction unit, a matching unit, a transformation unit, a screening unit, a determination unit and a processing unit; wherein, The image acquisition unit is used to obtain a key frame image containing a target object, and obtain a frame image to be processed containing the target object; The feature extraction unit is used to extract one or more feature points in the key frame image and one or more feature points in the frame image to be processed respectively; The matching unit is used to determine the matching relationship between each feature point in the key frame image and each feature point in the frame image to be processed; The screening unit is used to screen the feature point pair set to obtain a target feature point pair set to exclude the situation where the matching relationship is incorrect; wherein each feature point pair is composed of a feature point in the key frame image with a matching relationship and a feature point in the frame image to be processed, and the feature point pair set includes multiple groups of feature point pairs; The transformation unit determines a transformation relationship based on the target feature point pairs in the target feature point pair set obtained by the screening unit; processes the frame image to be processed based on the transformation relationship to obtain a target frame image; the transformation relationship is used to transform the acquisition perspective of the frame image to be processed to be the same as or substantially the same as the acquisition perspective of the key frame image; The feature extraction unit is further used to extract feature points in the target frame image; The determining unit is used to determine a first loss value based on feature points in the key frame image and feature points in the target frame image; the feature points in the key frame image used to determine the first loss value are different from the feature points in the key frame image used to determine the transformation relationship; the first loss value is used to characterize the degree of difference between the target frame image and the key frame image; The processing unit is used to retain the obtained target frame image when the first loss value does not exceed a second threshold value; The processing unit is further configured to delete the obtained target frame image when the first loss value exceeds a second threshold.

9. The device according to claim 8, characterized in that The matching unit is used for: Obtain a first group of pixel pairs associated with a first feature point in the key frame image, and obtain a second group of pixel pairs associated with a second feature point in the frame image to be processed; wherein the first feature point is a feature point in the key frame image; the second feature point is a feature point in the frame image to be processed; the relative position relationship between the first pixel pair in the first group of pixel pairs and the first feature point is consistent with the relative position relationship between the second pixel pair in the second group of pixel pairs and the second feature point; the first group of pixel pairs includes at least one group of first pixel pairs; the second group of pixel pairs includes at least one group of second pixel pairs; Comparing pixel values ​​of two pixels included in the first pixel pair to obtain a first comparison result, and comparing pixel values ​​of two pixels included in the second pixel pair to obtain a second comparison result; Determining the similarity between the first group of pixel pairs and the second group of pixel pairs based on at least one first comparison result corresponding to the at least one group of first pixel pairs and at least one second comparison result corresponding to the at least one group of second pixel pairs; When the similarity reaches a first preset threshold, it is determined that the first feature point and the second feature point satisfy a matching relationship.

10. The device according to claim 9, characterized in that The matching unit is configured to determine the similarity between the first group of pixel pairs and the second group of pixel pairs based on at least one first comparison result corresponding to the at least one group of first pixel pairs and at least one second comparison result corresponding to the at least one group of second pixel pairs: generating a first character string based on the at least one first comparison result; generating a second character string based on the at least one second comparison result; A first Hamming distance is determined based on the first character string and the second character string, and a similarity between the first group of pixel pairs and the second group of pixel pairs is determined based on the first Hamming distance.

11. The device according to claim 8, characterized in that The screening unit is used for: Obtaining the similarity between the feature points in the key frame image and the feature points in the frame image to be processed contained in each feature point pair in the feature point pair set; Determine a maximum value of similarities corresponding to all feature point pairs in the feature point pair set, and determine a first threshold based on the maximum value; The feature point pairs whose corresponding similarities are less than the first threshold value are removed from the feature point pair set to obtain a target feature point pair set.

12. The device according to any one of claims 8 to 11, characterized in that The feature extraction unit is used to: Determine a plurality of second pixel points within a preset area range of a first pixel point in the key frame image or the frame image to be processed; Determining whether the grayscale values ​​of the plurality of second pixels are different from the grayscale value of the first pixel; When the grayscale values ​​of the plurality of second pixels are different from the grayscale value of the first pixel, the first pixel is determined to be a feature point in the key frame image or the frame image to be processed.

13. The device according to claim 8, characterized in that The transformation unit is used to determine the transformation relationship based on the target feature point pairs in the target feature point pair set, and is used to: A transformation matrix is ​​determined based on the coordinates of the feature points in the key frame image contained in the target feature point pairs in the target feature point pair set and the coordinates of the feature points in the frame image to be processed.

14. The device according to claim 13, characterized in that The transformation unit is used to process the frame image to be processed based on the transformation relationship to obtain the target frame image, and is used to: Based on the transformation matrix, a target region containing the target object in the frame image to be processed is transformed to obtain a target frame image.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

16. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 7 are implemented.

17. A computer program product comprising computer readable code, wherein: When the computer readable code is run on a device, a processor in the device executes instructions to implement the steps of the method according to any one of claims 1 to 7.

Images