An image matching method for children's electronic puzzles

Abstract

The present invention relates to the field of data processing technology, and specifically to an image matching method for children's electronic puzzles, which extracts feature points from a grayscale image of a puzzle image after a current puzzle operation to obtain each feature point; determines each local area of ​​the feature area of ​​each feature point and the frequency of each local area in each set gradient direction, and based on the frequency in each set gradient direction, performs fusion dimensionality reduction on each set gradient direction corresponding to each local area, and finally determines the feature descriptor of each feature point, and performs image matching based on the feature descriptor and the feature descriptor of the corresponding standard feature point after the current puzzle operation. The present invention uses a graphic recognition method and performs relevant data processing to ensure the real-time construction of feature descriptors, and effectively solves the problem of low matching efficiency between actual puzzle results and standard results.

Description

Technical Field

[0001] The invention relates to the technical field of data processing, and in particular to an image matching method for children's electronic puzzles. Background Art

[0002] Children's electronic puzzles are educational toys that can effectively improve children's puzzle ability and interest. The intelligent guidance of children's electronic puzzles can provide children with appropriate prompts and help according to their puzzle progress and errors, solve operational problems, and improve puzzle speed. In order to identify errors in children's electronic puzzles, scale-invariant feature transform (SIFT) can be used to extract feature descriptors of actual puzzle results and standard results, and feature matching is performed based on the feature descriptors. Based on the feature matching results, the possible puzzle errors of children can be intelligently identified.

[0003] The construction of feature descriptors using the SIFT algorithm consists of three parts, two of which use the Gaussian difference function, or DOG function, to determine the position and scale of feature points. For the third part, in order to make the feature descriptor rotationally invariant, the SIFT algorithm constructs a gradient histogram of a 4×4 region around the feature point, and performs Gaussian fitting to construct a 4×4×8-dimensional direction vector. Since the constructed direction vector has a large dimension, the SIFT algorithm has a large amount of calculation, which results in the real-time performance of using the SIFT algorithm to construct feature descriptors not meeting the requirements, which in turn leads to low matching efficiency between the actual puzzle results and the standard results. Summary of the invention

[0004] The object of the present invention is to provide an image matching method for children's electronic puzzles, so as to solve the problem of low matching efficiency between existing actual puzzle results and standard results.

[0005] In order to solve the above technical problems, the present invention provides an image matching method for children's electronic puzzles, comprising the following steps:

[0006] Obtaining a grayscale image of the puzzle image after the current puzzle operation, extracting feature points from the grayscale image, and obtaining each feature point;

[0007] Determine a feature area of ​​each feature point, divide the feature area into at least two local areas, and determine a gradient direction of each pixel point in each local area;

[0008] Determining the frequency of each of the local regions in each of the set gradient directions according to the distribution characteristics of the gradient directions of the pixels in each of the local regions;

[0009] According to the frequency of each local area in each set gradient direction, the main contribution gradient direction corresponding to each local area is determined, and according to the main contribution gradient direction corresponding to each local area and the frequency of each local area in each set gradient direction, each set gradient direction corresponding to each local area is fused and reduced in dimension to obtain each gradient direction after dimension reduction corresponding to each local area and the frequency in each gradient direction after dimension reduction;

[0010] According to the frequency of each local area of ​​each feature point in each gradient direction after dimensionality reduction, a feature descriptor of each feature point is determined, and image matching is performed based on the feature descriptor of the feature point and the feature descriptor of the standard feature point corresponding to the standard puzzle image after the current puzzle operation.

[0011] Further, each set gradient direction corresponding to each local area is fused and dimensionally reduced to obtain each gradient direction after dimension reduction corresponding to each local area and the frequency in each gradient direction after dimension reduction, including:

[0012] According to the frequency of each of the local regions in each set gradient direction, determining the maximum frequency among all frequencies corresponding to each of the local regions;

[0013] According to the main contribution gradient direction corresponding to each local area and the frequency of each local area in each set gradient direction, each set gradient direction other than the main contribution gradient direction corresponding to each local area is screened to determine each set gradient direction combination and the set gradient direction that does not need to be fused;

[0014] Respectively calculating the ratio of the frequency of each local region in each set gradient direction in each set gradient direction combination to the corresponding maximum frequency, so as to obtain the weight of each local region in each set gradient direction in each set gradient direction combination;

[0015] According to the weight of each local region in each set gradient direction in each set gradient direction combination, weighted sum and average are performed on the frequencies of each local region in each set gradient direction in each set gradient direction combination, and the weighted sum and average result is determined as the frequency in the corresponding set gradient direction combination;

[0016] Each of the set gradient direction combinations, main contribution gradient directions, and set gradient directions that do not need to be fused corresponding to each of the local regions are taken as a gradient direction after dimensionality reduction, and the frequencies in each of the set gradient direction combinations, main contribution gradient directions, and set gradient directions that do not need to be fused are taken as the frequencies in the corresponding gradient directions after dimensionality reduction.

[0017] Further, determining each set gradient direction combination and setting gradient directions without fusion includes:

[0018] Arranging the set gradient directions corresponding to each of the local regions in order of magnitude, thereby obtaining a set gradient direction sequence corresponding to each of the local regions;

[0019] According to the order of the set gradient direction sequence from front to back, the frequencies corresponding to each two adjacent set gradient directions except the main contribution gradient direction are compared, so as to determine each continuous sequence segment in the set gradient direction sequence, each two adjacent set gradient directions in the continuous sequence segment and each set gradient direction and the first set gradient direction both meet a fusion condition, and the first set gradient direction of each continuous sequence segment and the last set gradient direction of the previous continuous sequence segment adjacent to it do not meet a fusion condition;

[0020] All set gradient directions corresponding to each of the continuous sequence segments are combined as a set gradient direction, and each set gradient direction in the set gradient direction sequence except the main contribution gradient direction and the set gradient directions corresponding to all the continuous sequence segments is taken as a set gradient direction that does not need to be fused.

[0021] Furthermore, the fusion condition is that the ratio of the absolute value of the difference between the frequencies corresponding to the two set gradient directions to the maximum value of the frequencies corresponding to the two set gradient directions is less than a set value.

[0022] Furthermore, the main contribution gradient direction includes a main gradient direction and a secondary gradient direction, and determining the main gradient direction and the secondary gradient direction corresponding to each of the local regions includes:

[0023] Determine the set gradient direction corresponding to the maximum frequency among all frequencies corresponding to each of the local regions as the main gradient direction corresponding to each of the local regions;

[0024] The set gradient direction corresponding to the frequency whose difference ratio with the maximum frequency among all the frequencies except the maximum frequency corresponding to each local area is within a set range is determined as the secondary gradient direction corresponding to each local area.

[0025] Further, determining the feature area of ​​each feature point includes:

[0026] The rectangular area of ​​each feature point is determined according to the SIFT algorithm, and the area corresponding to the inscribed circle of the rectangular area is determined as the feature area of ​​each feature point.

[0027] The characteristic region is further divided into at least two local regions, including:

[0028] The characteristic region is evenly divided into a set number of sector-shaped intervals, and the sector-shaped intervals are determined as local regions.

[0029] Furthermore, before dividing the characteristic region into at least two local regions, the method further includes:

[0030] Perform two-dimensional Gaussian convolution processing on the feature area, and divide the feature area after the two-dimensional Gaussian convolution processing into at least two local areas.

[0031] Furthermore, the value range of the gradient direction of the pixel point is evenly divided, so as to obtain the set gradient directions corresponding to each of the local areas.

[0032] Further, determining the frequency of each of the local regions in each set gradient direction includes:

[0033] The gradient directions of the pixels in each of the local regions are counted to determine a gradient direction histogram, and the frequency of each of the local regions in each set gradient direction is determined according to the gradient direction histogram.

[0034] The present invention has the following beneficial effects: by extracting feature points from the grayscale image of the puzzle image after the current puzzle operation, each feature point is obtained. In order to describe the feature points, the feature area of ​​each feature point is determined, and the feature area is subdivided into multiple local areas. Based on the frequency of each local area in each set gradient direction, the main contribution gradient direction corresponding to each local area is determined. The main contribution gradient direction here refers to the gradient direction that contributes more to the feature description of the feature point. Therefore, based on the main contribution gradient direction, the distribution characteristics of the frequency of each local area in each set gradient direction are analyzed, and then the set gradient directions corresponding to each local area are fused and reduced in dimension, so as to obtain the gradient directions after dimension reduction corresponding to each local area and the frequency in each gradient direction after dimension reduction, and then determine the feature descriptor of each feature point. Since the main contributing gradient direction that contributes more to the feature description of the feature point is referred to when the set gradient directions corresponding to the local areas are fused and reduced in dimension, the gradient directions after dimension reduction and their corresponding frequencies can accurately describe the features of the feature points, effectively reducing the dimension of the feature descriptor while ensuring the quality of the feature descriptor, thereby reducing the amount of data calculation, and thus effectively improving the matching efficiency between the actual puzzle result and the standard result. The present invention improves the real-time performance of constructing the feature descriptor while ensuring the quality of the feature descriptor through graphic recognition and related data processing, and effectively solves the problem of low matching efficiency between the existing actual puzzle result and the standard result. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 The present invention is a flowchart of an image matching method for children's electronic puzzles according to an embodiment of the present invention. DETAILED DESCRIPTION

[0037] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects of the technical solutions proposed by the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0038] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. In addition, all parameters or indicators in the formulas involved in this article are normalized values ​​that eliminate the influence of dimensions.

[0039] This embodiment provides an image matching method for children's electronic puzzles. The corresponding flowchart of the method is as follows: Figure 1 As shown, the following steps are included:

[0040] Step S1: obtaining a grayscale image of the puzzle image after the current puzzle operation, extracting feature points from the grayscale image, and obtaining various feature points.

[0041] When performing intelligent guidance of electronic puzzles for children, the operation is generally performed on a computer, and the children play the electronic puzzle game. Each time the children perform a puzzle operation, the computer records it and obtains the puzzle result of the children, that is, the puzzle image after each puzzle operation. Since the puzzle image is not affected by noise, the puzzle image after each puzzle operation of the children can be directly obtained through the computer.

[0042] After obtaining the puzzle image after each puzzle operation of the child, the puzzle image is grayed to obtain the corresponding gray image. Since the puzzle image after each puzzle operation of the child obtained by the computer is less affected by noise and forms fewer pseudo feature points, the present embodiment uses the existing SIFT algorithm to extract feature points from the gray image, thereby obtaining each feature point on the gray image and the position and scale corresponding to each feature point. When the SIFT algorithm is used to extract feature points from the gray image, the upper and lower adjacent layers 26 ( ) points are compared to get the extreme point. The specific process of using the existing SIFT algorithm to extract feature points from grayscale images is as follows:

[0043] (1) Use a scale-variable Gaussian function to convolve with the acquired grayscale image to obtain images with different blur levels, and use downsampling to construct a Gaussian pyramid of the image. This step aims to construct Gaussian convolution images at different scales.

[0044] (2) Use the Gaussian difference function (DOG) to convolve with the acquired grayscale image to construct a Gaussian difference pyramid. That is, in the Gaussian pyramid, the images of adjacent layers at the same scale are subtracted to construct a Gaussian difference pyramid. The purpose of constructing a Gaussian difference pyramid is to obtain stable feature points in the scale space.

[0045] (3) In the image convolved with the DOG function, the extreme points are extracted, that is, the pixel point is compared with a total of 26 pixels in the adjacent layer of the pixel point at the same scale to ensure that the extreme points can be detected in both the original grayscale image and the scale space.

[0046] (4) Perform curve fitting on the DOG function, calculate the Hessian matrix of the pixels around the extreme point, and remove the points with strong edge response by judging the two eigenvalues ​​of the matrix. This step aims to remove the pseudo feature points formed by the edge response and eliminate the edge response.

[0047] (5) Perform Taylor expansion on the DOG function at the initial key point, retain it to its quadratic term, and take the derivative of the Taylor expansion to find the extreme point at the initial key point, thereby obtaining the exact position of the initial extreme point, which is also the feature point.

[0048] Through the above steps, each feature point in the grayscale image of the puzzle image after each puzzle operation by the child and the position and scale corresponding to each feature point can be obtained.

[0049] Step S2: determining a feature region of each feature point, dividing the feature region into at least two local regions, and determining a gradient direction of each pixel point in each local region.

[0050] The feature area of ​​each feature point selected by the existing SIFT algorithm is the area around the feature point. When the grayscale gradient direction of the surrounding pixels of each feature point is used to construct the feature descriptor of the feature point, the contribution of the grayscale gradient direction of the surrounding pixels to the gradient direction of the feature point is inversely proportional to the distance between the surrounding pixels and the feature point. In order to reduce the amount of calculation, this embodiment determines the feature area of ​​each feature point according to the SIFT algorithm, and determines the area corresponding to the inscribed circle of the rectangular area as the feature area of ​​each feature point. That is, the area around each feature point selected by the existing SIFT algorithm is In the rectangular area, determine the The inscribed circle of a rectangular area is used as the feature area of ​​the corresponding feature point. At this time, each feature point corresponds to a feature area, that is, an inscribed circle. There are two reasons for selecting the inscribed circle as the feature area of ​​each feature point: ① Since the contribution of the grayscale gradient direction of the surrounding pixels to the gradient direction of the feature point is inversely proportional to the distance between the surrounding pixels and the feature point, when selecting the feature area of ​​the feature point, the edge corners of the rectangular area do not contribute much to it. Removing the pixels with unstable edges can reduce the amount of calculation; ② The circle itself has rotation invariance.

[0051] By using the inscribed circle as the feature area of ​​each feature point, the number of pixels that need to be processed later is reduced, which effectively reduces the amount of calculation. In order to ensure that the feature area of ​​each selected feature point can accurately reflect the gradient direction distribution around the feature point, the gradient of the pixel is Gaussian-weighted in the selected feature area, and the feature area after Gaussian-weighted processing is subsequently divided into at least two local areas. Gaussian weighted processing is performed to enhance the accuracy of the pixel gradients around each feature point and reduce the impact of removing the rectangular corner pixels in the existing SIFT algorithm, thereby ensuring the quality of the feature descriptor calculated later. When Gaussian weighted processing is performed on the gradient of the pixel in the selected feature area, this embodiment uses a two-dimensional Gaussian distribution function to perform two-dimensional Gaussian convolution processing on the feature area, and the corresponding calculation formula is:

[0052]

[0053] in, The coordinates are The coordinates of the feature points in the feature area are The Gaussian weight of the pixel point, are the coordinates of the feature points, The coordinates are The scale corresponding to the feature point is It is an exponential function with the natural constant e as its base.

[0054] Based on the Gaussian weight of each pixel in the feature area of ​​each feature point determined by the above calculation formula, a two-dimensional Gaussian convolution process is performed on the feature area of ​​each feature point. Since the specific implementation process of the two-dimensional Gaussian convolution process belongs to the prior art, it will not be repeated here. For the pixels around the feature point, the contribution of its gradient direction to the main direction of the feature area describing the feature point is inversely proportional to its distance from the feature point, and the Gaussian distribution obeys this relationship. Therefore, by using a two-dimensional Gaussian distribution function for weight allocation, the problem of reduced quality of the feature descriptor caused by removing the edge pixels of the original rectangular area when the inscribed circle is used as the feature area can be effectively avoided.

[0055] After performing two-dimensional Gaussian convolution processing on the feature area of ​​each feature point, the feature area after the two-dimensional Gaussian convolution processing is divided into multiple local areas, and the implementation steps are: evenly dividing the feature area into a set number of sector intervals, and determining these sector intervals as local areas. In this embodiment, the feature area of ​​each feature point after the two-dimensional Gaussian convolution processing is evenly divided into 4 sector areas, and these 4 sector areas are used as local areas of the corresponding feature area.

[0056] Step S3: determining the frequency of each local area in each set gradient direction according to the distribution characteristics of the gradient direction of the pixel points in each local area.

[0057] After determining each local area of ​​the feature area of ​​each feature point, determine each set gradient direction of each local area. The determination method of each set gradient direction is: evenly divide the value range of the gradient direction of the pixel point, so as to obtain the set gradient direction corresponding to each local area. In this embodiment, for each local area, the value range of the gradient direction of the pixel point 0-360° is evenly divided into 18 parts, each 20° is used as one part, and each part is used as a set gradient direction. The purpose of subdividing the value range of the gradient direction of the pixel point 0-360° is to determine the accurate gradient distribution direction of the local area, so as to ensure the accuracy of the feature descriptor constructed subsequently.

[0058] For each local area of ​​the feature area of ​​each feature point, the local area refers to the local area after two-dimensional Gaussian convolution processing, and the gradient of each pixel in the local area is determined. The gradient includes the gradient direction and the gradient amplitude. Since the process of determining the gradient of each pixel belongs to the prior art, it will not be repeated here. The gradient direction of the pixel points in each local area is counted, that is, according to the gradient direction of the pixel points in each local area, each set gradient direction is voted to determine the gradient direction histogram. Since the horizontal coordinate of the determined gradient direction histogram is the set gradient direction, and the vertical coordinate is the number of pixels in the set gradient direction, that is, the number of votes, the frequency of each local area in each set gradient direction can be determined according to the vertical coordinate of the gradient direction histogram.

[0059] Step S4: According to the frequency of each local area in each set gradient direction, determine the main contribution gradient direction corresponding to each local area, and according to the main contribution gradient direction corresponding to each local area and the frequency of each local area in each set gradient direction, perform fusion and dimension reduction on each set gradient direction corresponding to each local area to obtain each gradient direction after dimension reduction corresponding to each local area and the frequency in each gradient direction after dimension reduction.

[0060] The existing SIFT algorithm uses 8 set gradient directions, and the angle interval of each set gradient direction is 45°. Although the existing SIFT algorithm can obtain better performance and robustness by dividing 0-360° into 8 set gradient directions, this embodiment can effectively improve the description ability of features by subdividing 0-360° into 18 set gradient directions, but it also increases the complexity of subsequent calculations. Therefore, in order to reduce the complexity of subsequent calculations, it is necessary to fuse and reduce the dimensionality of the 18 set gradient directions, thereby reducing the directional dimension of the feature descriptor.

[0061] In order to realize the fusion dimension reduction of each set gradient direction corresponding to each local area, after determining the frequency of each local area of ​​each feature point in each set gradient direction through the above steps, first determine the main contribution gradient direction corresponding to each local area, the main contribution gradient direction includes the main gradient direction and the secondary gradient direction, and the implementation steps include:

[0062] Determine the set gradient direction corresponding to the maximum frequency among all frequencies corresponding to each of the local regions as the main gradient direction corresponding to each of the local regions;

[0063] The set gradient direction corresponding to the frequency whose difference ratio with the maximum frequency among all the frequencies except the maximum frequency corresponding to each local area is within a set range is determined as the secondary gradient direction corresponding to each local area.

[0064] For ease of understanding, the frequency of each set gradient direction in the gradient direction histogram corresponding to each local area is determined, that is, the pixel ratio in each set gradient direction. , , Take an integer. Then determine the maximum pixel ratio That is, the set gradient direction corresponding to the maximum frequency, and the set gradient direction is determined as the main gradient direction corresponding to the local area. Then determine the difference ratio of the pixel ratio corresponding to each set gradient direction other than the main gradient direction and the pixel ratio corresponding to the main gradient direction, and the difference ratio refers to the ratio of the difference between the pixel ratio corresponding to the main gradient direction and the pixel ratio corresponding to each set gradient direction other than the main gradient direction and the pixel ratio corresponding to the main gradient direction. It should be understood that the difference ratio of the frequency corresponding to each set gradient direction other than the main gradient direction and the frequency corresponding to the main gradient direction can also be determined. At this time, the difference ratio refers to the difference between the frequency corresponding to the main gradient direction and the frequency corresponding to each set gradient direction other than the main gradient direction and the frequency corresponding to the main gradient direction. The set gradient direction corresponding to the difference ratio within the set range is determined as the secondary gradient direction corresponding to the local area. In this embodiment, the set range refers to 0-20%. In this way, the main gradient direction and the secondary gradient direction corresponding to each local area of ​​each feature point can be determined, that is, the main contribution gradient direction.

[0065] By subdividing the angles, accurate local gradient distribution can be obtained, thereby improving the descriptive ability of features, and the main gradient direction and the secondary gradient direction contribute the most to describing the gradient direction distribution of the local area. Therefore, based on the main gradient direction and the secondary gradient direction corresponding to each local area of ​​each feature point determined, combined with the frequency of each local area in each set gradient direction, each set gradient direction corresponding to each local area is fused and reduced in dimension to obtain each gradient direction after dimension reduction corresponding to each local area and the frequency of each gradient direction after dimension reduction. The implementation steps include:

[0066] According to the frequency of each of the local regions in each set gradient direction, determining the maximum frequency among all frequencies corresponding to each of the local regions;

[0067] According to the main contribution gradient direction corresponding to each local area and the frequency of each local area in each set gradient direction, each set gradient direction other than the main contribution gradient direction corresponding to each local area is screened to determine each set gradient direction combination and the set gradient direction that does not need to be fused;

[0068] Respectively calculating the ratio of the frequency of each local region in each set gradient direction in each set gradient direction combination to the corresponding maximum frequency, so as to obtain the weight of each local region in each set gradient direction in each set gradient direction combination;

[0069] According to the weight of each local region in each set gradient direction in each set gradient direction combination, weighted sum and average are performed on the frequencies of each local region in each set gradient direction in each set gradient direction combination, and the weighted sum and average result is determined as the frequency in the corresponding set gradient direction combination;

[0070] Each of the set gradient direction combinations, main contribution gradient directions, and set gradient directions that do not need to be fused corresponding to each of the local regions are taken as a gradient direction after dimensionality reduction, and the frequencies in each of the set gradient direction combinations, main contribution gradient directions, and set gradient directions that do not need to be fused are taken as the frequencies in the corresponding gradient directions after dimensionality reduction.

[0071] For ease of understanding, for each local area of ​​the feature area of ​​each feature point, the frequency in each set gradient direction determined by the gradient direction histogram of the local area is the pixel ratio. , calculate the proportion of each pixel With the maximum pixel ratio The ratio of the frequency of each local region in each set gradient direction can be calculated based on the frequency of each set gradient direction in the gradient direction histogram of the local region, and the ratio can be determined as the weight of each local region in each set gradient direction. It should be understood that the ratio of each frequency to the maximum frequency of all frequencies can also be calculated based on the frequency of each set gradient direction in the gradient direction histogram of the local region, and the ratio can be determined as the weight of each local region in each set gradient direction.

[0072] In order to achieve fusion dimensionality reduction of each set gradient direction corresponding to each local area, according to the main contribution gradient direction corresponding to each local area and the frequency of each local area in each set gradient direction, each set gradient direction other than the main contribution gradient direction corresponding to each local area is screened to determine each set gradient direction combination and the set gradient direction that does not need to be fused. The implementation steps include:

[0073] Arranging the set gradient directions corresponding to each of the local regions in order of magnitude, thereby obtaining a set gradient direction sequence corresponding to each of the local regions;

[0074] According to the order of the set gradient direction sequence from front to back, the frequencies corresponding to each two adjacent set gradient directions except the main contribution gradient direction are compared, so as to determine each continuous sequence segment in the set gradient direction sequence, each two adjacent set gradient directions in the continuous sequence segment and each set gradient direction and the first set gradient direction both meet a fusion condition, and the first set gradient direction of each continuous sequence segment and the last set gradient direction of the previous continuous sequence segment adjacent to it do not meet a fusion condition;

[0075] All set gradient directions corresponding to each of the continuous sequence segments are combined as a set gradient direction, and each set gradient direction in the set gradient direction sequence except the main contribution gradient direction and the set gradient directions corresponding to all the continuous sequence segments is taken as a set gradient direction that does not need to be fused.

[0076] For ease of understanding, for each local area of ​​each feature point, the 18 set gradient directions on the horizontal axis of the gradient direction histogram corresponding to the local area constitute a set gradient direction sequence. In the set gradient direction sequence, in the order from front to back, that is, the set gradient directions are in order from small to large, it is determined whether the frequencies corresponding to the adjacent first set gradient direction and the second set gradient direction other than the main contribution gradient direction meet the fusion condition. If the fusion condition is met, it means that the first set gradient direction and the second set gradient direction can be fused, and the two set gradient directions are divided into the same continuous sequence segment. The fusion condition here is: the ratio of the absolute value of the difference between the frequencies corresponding to the two set gradient directions and the maximum value of the frequencies corresponding to the two set gradient directions is less than the set value. The set value can be set as needed. In this embodiment, the set value is set to 0.1. Then, it is determined whether the frequencies corresponding to the third set gradient direction adjacent to the second set gradient direction and not the main contributing gradient direction and the second set gradient direction meet the fusion condition. If the fusion condition is met, it is determined whether the frequencies corresponding to the third set gradient direction and the first set gradient direction meet the fusion condition. If the fusion condition is met, it means that the first set gradient direction, the second set gradient direction and the third set gradient direction can be fused, and the third set gradient direction and the second set gradient direction are divided into the same continuous sequence segment. At this time, the first set gradient direction, the second set gradient direction and the third set gradient direction are divided into the same continuous sequence segment. By analogy, if the i-th set gradient direction that is not the main contributing gradient direction can be fused with its adjacent previous set gradient direction, then it is determined whether the frequency corresponding to the i+1-th set gradient direction that is not the main contributing gradient direction and the i-th set gradient direction meets the fusion condition, and whether the frequency corresponding to the i+1-th set gradient direction and the first set gradient direction that can be fused corresponding to the i-th set gradient direction meets the fusion condition. If both meet the fusion condition, the i+1-th set gradient direction and the i-th set gradient direction are divided into the same continuous sequence segment; if one of the two does not meet the fusion condition, it means that the i+1-th set gradient direction cannot be fused with the i-th set gradient direction, and the fusion is stopped at this time. For example, if the frequency corresponding to the third set gradient direction and the second set gradient direction does not meet the fusion condition, or the frequency corresponding to the third set gradient direction and the first set gradient direction does not meet the fusion condition, this means that the frequency corresponding to the third set gradient direction and the first set gradient direction or the second set gradient direction is greatly different, and the third set gradient direction cannot be fused with the first set gradient direction and the second set gradient direction. Therefore, the third set gradient direction cannot be divided into the continuous sequence segments corresponding to the first set gradient direction and the second set gradient direction, and the fusion is stopped at this time.

[0077] After the last fusion is completed, the next set gradient direction except the main contribution gradient direction is used as the first set gradient direction of the next fusion, and the fusion is continued in the same way as the last fusion until all set gradient directions except the main contribution gradient direction are traversed, thereby finally obtaining each continuous sequence segment.

[0078] It should be understood that, in the process of determining each continuous sequence segment, the main contribution gradient direction, that is, the main gradient direction and the secondary gradient direction, do not participate in the fusion, and because the two set gradient directions adjacent to the main gradient direction on the left and right are separated by the main gradient direction, and the two set gradient directions adjacent to the secondary gradient direction on the left and right are separated by the secondary gradient direction, the two set gradient directions adjacent to the main gradient direction on the left and right cannot be divided into the same continuous sequence segment, and the two set gradient directions adjacent to the secondary gradient direction on the left and right cannot be divided into the same continuous sequence segment.

[0079] According to the above method, the adjacent set gradient directions with similar corresponding frequencies in the set gradient direction sequence except the main contribution gradient direction are fused, that is, they are divided into the same continuous sequence segment, and finally each continuous sequence segment can be obtained. All the set gradient directions corresponding to each continuous sequence segment are combined as a set gradient direction, so that each set gradient direction combination can be obtained. Then, the single set gradient direction in the set gradient direction sequence except the main contribution gradient direction and the set gradient directions corresponding to all continuous sequence segments are taken as a set gradient direction that does not need to be fused, so that each set gradient direction that does not need to be fused can be obtained.

[0080] After determining each set gradient direction combination corresponding to each local area, the frequency in each set gradient direction combination is determined based on the weight and frequency in each set gradient direction in each set gradient direction combination, and then the frequency in each set gradient direction combination is determined. The corresponding calculation formula is:

[0081]

[0082] in, is the frequency of each local area of ​​the feature area of ​​each feature point corresponding to the j-th set gradient direction combination, is the frequency in the i-th set gradient direction in the j-th set gradient direction combination, that is, the pixel proportion, which can be obtained according to the frequency in the i-th set gradient direction in the j-th set gradient direction combination, is the weight in the i-th set gradient direction in the j-th set gradient direction combination, and n is the total number of set gradient directions in the j-th set gradient direction combination.

[0083] In the above calculation formula, by assigning a weight to each set gradient direction in each set gradient direction combination, that is, a weight, and based on the frequency of each set gradient direction in each set gradient direction combination, a weighted sum is performed to average the frequency in each set gradient direction combination, and the frequency is actually a corrected frequency. The corrected frequency can reflect the change details of the set gradient direction in each set gradient direction combination and the corresponding pixel proportion, and can achieve reliable fusion and dimensionality reduction of each set gradient direction corresponding to each local area. It should be understood that the frequency in the above calculation formula can also be replaced by the corresponding frequency, that is, the frequency in the above calculation formula is multiplied by the total number of pixels corresponding to each local area, so that the frequency in each set gradient direction combination can be determined.

[0084] Each set gradient direction combination, main gradient direction, sub-gradient direction and set gradient direction without fusion corresponding to each local area is taken as a gradient direction after dimensionality reduction, and the frequency in each set gradient direction combination, main gradient direction, sub-gradient direction and set gradient direction without fusion is taken as the frequency in the corresponding gradient direction after dimensionality reduction.

[0085] Step S5: Determine the feature descriptor of each feature point according to the frequency of each local area of ​​each feature point in each gradient direction after dimensionality reduction, and perform image matching based on the feature descriptor of the feature point and the feature descriptor of the standard feature point corresponding to the standard puzzle image after the current puzzle operation.

[0086] After determining the frequency or frequency of each local area of ​​each feature point in each gradient direction after dimensionality reduction through the above steps, the frequency here can be obtained by multiplying the frequency by the total number of pixels in the corresponding local area, then according to each gradient direction after dimensionality reduction and its corresponding frequency or frequency, determine the corresponding fused gradient histogram, in which the horizontal coordinate is each gradient direction after dimensionality reduction, and the vertical coordinate is the frequency corresponding to each gradient direction after dimensionality reduction. For the gradient direction after dimensionality reduction corresponding to the set gradient direction combination, the horizontal coordinate of such gradient direction after dimensionality reduction in the fused gradient histogram is the gradient direction interval formed by each set gradient direction in the corresponding set gradient direction combination. For example, if the gradient direction combination is set to 20°-40° and 40°-60°, then the horizontal coordinate of the gradient direction after dimensionality reduction corresponding to the set gradient direction combination in the fused gradient histogram is 20°-60°. According to the fused gradient histogram corresponding to each local area of ​​each feature point, the feature descriptor of each feature point can be determined.

[0087] The dimension of the feature descriptor of each feature point determined by the existing SIFT algorithm is Through the above steps, each set gradient direction corresponding to each local area of ​​each feature point is fused and reduced in dimension. At this time, the dimension of the feature descriptor of each feature point is 4×k. , the dimension of the feature descriptor is greatly reduced. While not changing the details of the local gradient direction distribution, that is, while ensuring the quality of the feature descriptor, by reducing the dimension of the feature descriptor, the matching efficiency can be effectively improved and the matching time can be reduced in the subsequent feature matching, thereby improving the timeliness of the puzzle operation guidance.

[0088] After determining the feature descriptor of each feature point, the feature descriptor of the feature point is matched with the feature descriptor of the standard feature point corresponding to the standard puzzle image after the current puzzle operation, and the matching result of the puzzle image after the current puzzle operation and the standard puzzle image after the current puzzle operation is determined. Since the specific implementation process of the matching belongs to the prior art, it is not repeated here. When determining the feature descriptor of the standard feature point corresponding to the standard puzzle image after the current puzzle operation, the same steps as the above-mentioned determination of the feature descriptor of each feature point in the grayscale image of the puzzle image after the current puzzle operation are followed to determine the feature descriptor of the standard feature point corresponding to the standard puzzle image after the current puzzle operation. According to the matching result, the current puzzle operation is guided. For example, when the matching result is that the feature points of the grayscale image of the puzzle image after the current puzzle operation are completely matched one-to-one with the standard feature points corresponding to the standard puzzle image after the current puzzle operation, it means that the current puzzle operation is correct. In this case, a corresponding sound sensor can be set to indicate that the puzzle is correct, or a voice prompt can be given to remind the child that the puzzle is correct and the next puzzle can be carried out for puzzle guidance. When the matching result is that the feature points of the grayscale image of the puzzle image after the current puzzle operation are not completely matched one-to-one with the standard feature points corresponding to the standard puzzle image after the current puzzle operation, it means that the current puzzle operation is wrong. In this case, a corresponding sound sensor can be set to indicate that the puzzle is correct, or a voice prompt can be given to remind the child that the puzzle is wrong and needs to be re-operated for puzzle guidance.

[0089] The present invention determines the main contribution gradient direction, that is, the main gradient direction and the secondary gradient direction, corresponding to each local area of ​​the feature area of ​​each feature point, and then fuses and reduces the dimension of each set gradient direction corresponding to each local area, and finally determines the feature descriptor of each feature point. Compared with the dimension of the feature descriptor of each feature point determined by the existing SIFT algorithm, while ensuring the quality of the feature descriptor, the data calculation amount is effectively reduced, and the matching efficiency between the actual puzzle result and the standard result can be effectively improved, so as to timely feedback the current puzzle operation of the child and guide the child to perform the puzzle operation in time.

[0090] It should be noted that the above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. An image matching method for children's electronic puzzles, characterized in that: The following steps are involved: Obtaining a grayscale image of the puzzle image after the current puzzle operation, extracting feature points from the grayscale image, and obtaining each feature point; Determine a feature area of ​​each feature point, divide the feature area into at least two local areas, and determine a gradient direction of each pixel point in each local area; Determining the frequency of each of the local regions in each of the set gradient directions according to the distribution characteristics of the gradient directions of the pixels in each of the local regions; According to the frequency of each local area in each set gradient direction, the main contribution gradient direction corresponding to each local area is determined, and according to the main contribution gradient direction corresponding to each local area and the frequency of each local area in each set gradient direction, each set gradient direction corresponding to each local area is fused and reduced in dimension to obtain each gradient direction after dimension reduction corresponding to each local area and the frequency in each gradient direction after dimension reduction; Determine a feature descriptor for each feature point according to the frequency of each local area of ​​each feature point in each gradient direction after dimensionality reduction, and perform image matching based on the feature descriptor of the feature point and the feature descriptor of the standard feature point corresponding to the standard puzzle image after the current puzzle operation; The set gradient directions corresponding to each local area are fused and reduced in dimension to obtain the gradient directions after dimension reduction corresponding to each local area and the frequency in each gradient direction after dimension reduction, including: According to the frequency of each of the local regions in each set gradient direction, determining the maximum frequency among all frequencies corresponding to each of the local regions; According to the main contribution gradient direction corresponding to each local area and the frequency of each local area in each set gradient direction, each set gradient direction other than the main contribution gradient direction corresponding to each local area is screened to determine each set gradient direction combination and the set gradient direction that does not need to be fused; Respectively calculating the ratio of the frequency of each local region in each set gradient direction in each set gradient direction combination to the corresponding maximum frequency, so as to obtain the weight of each local region in each set gradient direction in each set gradient direction combination; According to the weight of each local region in each set gradient direction in each set gradient direction combination, weighted sum and average are performed on the frequencies of each local region in each set gradient direction in each set gradient direction combination, and the weighted sum and average result is determined as the frequency in the corresponding set gradient direction combination; Each of the set gradient direction combinations, main contribution gradient directions, and set gradient directions that do not need to be fused corresponding to each of the local regions are taken as a gradient direction after dimensionality reduction, and the frequencies in each of the set gradient direction combinations, main contribution gradient directions, and set gradient directions that do not need to be fused are taken as the frequencies in the corresponding gradient directions after dimensionality reduction.

2. The image matching method for children's electronic puzzles according to claim 1, characterized in that: Determine the combination of each set gradient direction and the set gradient direction without fusion, including: Arranging the set gradient directions corresponding to each of the local regions in order of magnitude, thereby obtaining a set gradient direction sequence corresponding to each of the local regions; According to the order of the set gradient direction sequence from front to back, the frequencies corresponding to each two adjacent set gradient directions except the main contribution gradient direction are compared, so as to determine each continuous sequence segment in the set gradient direction sequence, each two adjacent set gradient directions in the continuous sequence segment and each set gradient direction and the first set gradient direction both meet a fusion condition, and the first set gradient direction of each continuous sequence segment and the last set gradient direction of the previous continuous sequence segment adjacent to it do not meet a fusion condition; All set gradient directions corresponding to each of the continuous sequence segments are combined as a set gradient direction, and each set gradient direction in the set gradient direction sequence except the main contribution gradient direction and the set gradient directions corresponding to all the continuous sequence segments is taken as a set gradient direction that does not need to be fused.

3. The image matching method for children's electronic puzzles according to claim 2, characterized in that: The fusion condition is that the ratio of the absolute value of the difference between the frequencies corresponding to the two set gradient directions to the maximum value of the frequencies corresponding to the two set gradient directions is less than a set value.

4. The image matching method for children's electronic puzzles according to claim 1, characterized in that: The main contribution gradient direction includes a main gradient direction and a secondary gradient direction, and determining the main gradient direction and the secondary gradient direction corresponding to each of the local regions includes: Determine the set gradient direction corresponding to the maximum frequency among all frequencies corresponding to each of the local regions as the main gradient direction corresponding to each of the local regions; The set gradient direction corresponding to the frequency whose difference ratio with the maximum frequency among all the frequencies except the maximum frequency corresponding to each local area is within a set range is determined as the secondary gradient direction corresponding to each local area.

5. The image matching method for children's electronic puzzles according to claim 1, characterized in that: Determining a feature area of ​​each feature point includes: The rectangular area of ​​each feature point is determined according to the SIFT algorithm, and the area corresponding to the inscribed circle of the rectangular area is determined as the feature area of ​​each feature point.

6. The image matching method for children's electronic puzzles according to claim 5, characterized in that: Dividing the characteristic region into at least two local regions includes: The characteristic region is evenly divided into a set number of sector-shaped intervals, and the sector-shaped intervals are determined as local regions.

7. The image matching method for children's electronic puzzles according to claim 1, characterized in that: Before dividing the characteristic region into at least two local regions, the method further includes: Perform two-dimensional Gaussian convolution processing on the feature area, and divide the feature area after the two-dimensional Gaussian convolution processing into at least two local areas.

8. The image matching method for children's electronic puzzles according to claim 1, characterized in that: The value range of the gradient direction of the pixel point is evenly divided to obtain the set gradient directions corresponding to each local area.

9. The image matching method for children's electronic puzzles according to claim 1, characterized in that: Determining the frequency of each of the local regions in each set gradient direction includes: The gradient directions of the pixels in each of the local regions are counted to determine a gradient direction histogram, and the frequency of each of the local regions in each set gradient direction is determined according to the gradient direction histogram.

Images