Early childhood education toy image recognition method based on surveillance video
By constructing the Gaussian differential pyramid and calculating the confidence of the dominant direction of feature points, the problem of feature descriptor inaccuracy caused by rotation transformation in early education toy image recognition is solved, and the recognition accuracy and multi-angle motion recognition ability in rotating scenes are improved.
Patent Information
- Application Number
- CN202411148568.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-21
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-08-21
AI Technical Summary
During the use of early education toys, due to the invariance of image rotation caused by rotation transformation, it is difficult for the SIFT algorithm to obtain accurate feature descriptors, which affects the accuracy of feature point matching.
By constructing a Gaussian differential pyramid, feature points at different scale levels are obtained, and the confidence of the dominant direction of the feature points is calculated based on the multi-dimensional position vector and local area direction sequence of the feature points, the dominant direction is corrected, and a more accurate feature descriptor is constructed.
The accuracy of feature descriptors in rotation transformation scenarios is improved, and the recognition accuracy of early education toys is enhanced, especially when moving in multiple angles.
Smart Images

Figure CN118658017B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to an early childhood education toy image recognition method based on monitoring videos. Background Art
[0002] With the emergence of Storydoing, an interactive storytelling combination robot early childhood education toy, it uses the theory of "Learning by doing" to support children in learning basic skills by guiding them to complete story tasks, and attracts children in a highly interactive way to stimulate their creativity. It consists of a small hardware robot and related modular toolkits, which can move along a specific route and speed on a map according to the child's interactive choices. Through video monitoring and recognition technology, the interactive process between children and early childhood education toys can be monitored in real time, and the child's behavior, movements and expressions can be evaluated, and personalized educational content and feedback can be provided based on the child's behavior and reaction.
[0003] When using the SIFT algorithm to detect and match feature values to complete the image recognition of early childhood education toys, since the early childhood education toys will constantly move on the map according to the children's settings during use, there will often be a large rotation transformation on the obtained multi-frame monitoring images. However, due to the rotation invariance of key points and feature descriptors in the SIFT algorithm, it will be difficult to obtain feature descriptors with good description accuracy in local areas due to large rotation transformations, thereby affecting the accuracy of feature point matching. Summary of the invention
[0004] In order to solve the above problems, the present invention provides an early childhood education toy image recognition method based on surveillance video, the method comprising:
[0005] Get the grayscale image of early childhood education toy interaction monitoring;
[0006] Construct the Gaussian difference scale space of the grayscale image of early childhood educational toy interaction monitoring to obtain feature points at different scale levels; obtain the multidimensional position vectors of all feature points at different scale levels; obtain the similarity of the distribution structure of feature points between each adjacent layer at each scale based on the multidimensional position vectors of all feature points at different scale levels;
[0007] Obtain the direction sequence of the local area corresponding to each feature point; obtain the dominant direction confidence of each feature point in each adjacent layer according to the direction sequence of the local area corresponding to each feature point and the similarity of the distribution structure of feature points between each adjacent layer at each scale;
[0008] According to the dominant direction confidence of each feature point in each adjacent layer, all feature points that need to be corrected in dominant direction are obtained; the dominant direction correction factors of all feature points that need to be corrected in dominant direction are obtained; according to the dominant direction correction factors of all feature points that need to be corrected in dominant direction, the corrected dominant directions of all feature points are obtained; a feature descriptor is constructed according to the corrected dominant directions of all feature points, and based on the feature descriptor, the early childhood toy target recognition in the early childhood toy interactive monitoring grayscale image is completed.
[0009] Preferably, the Gaussian difference scale space of the early childhood educational toy interactive monitoring grayscale image is constructed to obtain feature points at different scale levels, including the following specific steps:
[0010] The SIFT algorithm is used to construct the Gaussian difference scale space of the grayscale image of early childhood educational toy interaction monitoring, and then generate a Gaussian difference pyramid. The Gaussian difference pyramid contains several scale levels, and feature points at different scale levels are obtained on the Gaussian difference pyramid.
[0011] Preferably, the step of obtaining the multi-dimensional position vectors of all feature points at different scale levels includes the following specific steps:
[0012] Preset distance threshold , with vertical upward as the initial direction; for the feature points, in At the scale level where the feature points are located, feature points as the center point, and the distance threshold As the radius, the circular area is obtained as the The neighborhood of feature points; get the The feature point and The connection lines of all feature points in the neighborhood of the feature point are sorted from small to large by the angles between all the connection lines and the initial direction, and the first The neighborhood angle sequence of feature points is The neighborhood angle sequence of feature points is taken as the The multi-dimensional position vector of feature points.
[0013] Preferably, the step of obtaining the similarity of the distribution structure of feature points between each adjacent layer at each scale according to the multi-dimensional position vectors of all feature points at different scales includes the following specific steps:
[0014] For The scale adjacent layers, through the The scale The similarity of the multidimensional position vectors of the feature points at the same position between adjacent layers is used to obtain the first The scale The distribution structure similarity of feature points between adjacent layers is calculated as follows:
[0015]
[0016] In the formula, Indicates The scale The similarity of the distribution structure of feature points between adjacent layers; Indicates The scale The first layer in the adjacent layers The multi-dimensional position vector of feature points; Indicates The scale The first layer of the adjacent feature points correspond to the multi-dimensional position vectors of the feature points in the second layer; Indicates the cosine similarity of vectors; Indicates The scale The number of feature points in the first layer among the adjacent layers.
[0017] Preferably, the step of obtaining the direction sequence of the local area corresponding to each feature point includes the following specific steps:
[0018] According to the SIFT algorithm, the dominant direction of each feature point is obtained. feature points, feature points as the center point, with the distance to the preset threshold As the radius, the circular area is obtained as the The feature point corresponding to a local area; The direction sequence composed of the dominant directions of all feature points in the local area is taken as the first A sequence of directions in a local area.
[0019] Preferably, the method of obtaining the dominant direction confidence of each feature point in each adjacent layer according to the direction sequence of the local area corresponding to each feature point and the similarity of the distribution structure of feature points between each adjacent layer at each scale includes the following specific steps:
[0020] Through the standard one-dimensional Gaussian function in OPENCV, the Gaussian weighted inter-layer distance of each adjacent layer at each scale is obtained. The scale The first layer in the adjacent layers The calculation expression of the dominant direction confidence of a feature point is:
[0021]
[0022] In the formula, Indicates The scale The first layer in the adjacent layers The dominant direction confidence of feature points; Indicates The scale Gaussian weighted inter-layer distances of adjacent layers; Indicates The scale The similarity of the distribution structure of feature points between adjacent layers; Indicates The scale The first layer in the adjacent layers A sequence of directions of local regions; Indicates The scale The total number of all local regions in the first layer among the adjacent layers; Indicates The scale The first layer in the adjacent layers The local regions correspond to the direction sequences of the local regions in the second layer; Indicates the similarity between two sequences; represents linear normalization.
[0023] Preferably, the method of obtaining all feature points that need to be corrected in dominant direction according to the dominant direction confidence of each feature point in each adjacent layer includes the following specific steps:
[0024] Preset Threshold , for the The next adjacent layer The dominant direction of the feature point, if The next adjacent layer The dominant direction confidence of feature points is less than the threshold , then the feature point is recorded as the feature point that needs to be corrected in the dominant direction; if The next adjacent layer The dominant direction confidence of feature points is greater than or equal to the threshold , then the feature point is recorded as a feature point that does not require dominant direction correction; similarly, all feature points that require dominant direction correction are obtained.
[0025] Preferably, the obtaining of the dominant direction correction factors of all feature points that need to be corrected in the dominant direction comprises the following specific steps:
[0026] For the first feature points, get the The dominant direction of the feature points in each adjacent layer is sorted from top to bottom as the The dominant direction sequence of feature points is The average of all dominant directions that need to be corrected in the dominant direction sequence of feature points is recorded as the first direction; The average of all the dominant directions that do not need to be corrected in the dominant direction sequence of the feature points is recorded as the second direction; the average of the first direction and the second direction is recorded as the first The dominant direction correction factor of each feature point.
[0027] Preferably, the method of obtaining the corrected dominant directions of all feature points according to the dominant direction correction factors of all feature points that need to be corrected in dominant directions comprises the following specific steps:
[0028] For feature point, if feature points in each adjacent layer are feature points that do not require dominant direction correction, then The dominant direction average of the feature points in all adjacent layers is taken as the The dominant direction after the feature point is corrected; if the If a feature point is a feature point that needs to be corrected in the dominant direction in any adjacent layer, then The dominant direction correction factor of the feature point is The product of the mean of the dominant directions of the feature points in all adjacent layers is taken as the The corrected dominant direction of the feature points is obtained; similarly, the corrected dominant directions of all feature points are obtained.
[0029] Preferably, the step of constructing a feature descriptor according to the corrected dominant directions of all feature points and completing the early childhood education toy target recognition in the early childhood education toy interactive monitoring grayscale image according to the feature descriptor includes the following specific steps:
[0030] Through the SIFT algorithm, the feature descriptor is constructed using the corrected dominant direction of each feature point. The feature descriptors are matched on adjacent frame images of early childhood educational toy interaction monitoring using the nearest neighbor distance and next nearest neighbor distance methods to obtain the feature descriptor matching results. If the feature descriptor matches successfully, the location of the feature point corresponding to the feature descriptor is the early childhood educational toy target.
[0031] The beneficial effects of the technical solution of the present invention are: to address the problem that the deformation caused by rotation significantly changes the local structure and texture features in the original image, making it difficult to obtain a more accurate feature descriptor; the present invention quantifies the overall directional change trend of the local area on a larger scale, obtains a direction correction factor, and corrects the main direction of the obtained feature area, so that the feature descriptor of the local area has better description accuracy, and realizes the early childhood toy target recognition scenario that copes with larger rotational transformations; the present invention obtains the dominant direction of the feature point when the target has a larger rotational transformation by combining the regional directional trend to obtain a more accurate feature descriptor, thereby improving the recognition accuracy of early childhood toys during multi-angle movement. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0033] Figure 1 The present invention is a flowchart of the steps of the early childhood education toy image recognition method based on surveillance video. DETAILED DESCRIPTION
[0034] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the early childhood education toy image recognition method based on surveillance video proposed by the present invention, its specific implementation method, structure, features and effects, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.
[0035] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
[0036] The specific scheme of the early childhood education toy image recognition method based on monitoring video provided by the present invention is described in detail below with reference to the accompanying drawings.
[0037] See also Figure 1 , which shows a flowchart of a method for early childhood education toy image recognition based on surveillance video provided by an embodiment of the present invention, the method comprising the following steps:
[0038] Step S001: Obtaining a grayscale image of early childhood education toy interaction monitoring.
[0039] It should be noted that, since early childhood educational toys will constantly move on the map according to the children's settings during use, the obtained early childhood educational toy interactive monitoring images often have a large rotation transformation. However, due to the rotation invariance of key points and feature descriptors, the SIFT algorithm will find it difficult to obtain feature descriptors with good description accuracy in local areas due to large rotation transformations, thereby affecting the accuracy of feature point matching.
[0040] Specifically, surveillance videos of children using early childhood education toys and interacting with the toys are obtained through surveillance probes. The surveillance probes are set at a higher place, located on the upper part of the wall, and shoot the entire process of children using early childhood education toys and interacting with them at a fixed angle. Early childhood education toy interaction monitoring images are extracted from the surveillance videos, and the images are grayscaled to obtain early childhood education toy interaction monitoring grayscale images.
[0041] At this point, the early childhood education toy interaction monitoring grayscale image is obtained.
[0042] Step S002: extract feature points from the early childhood educational toy interactive monitoring grayscale image to obtain the similarity of the feature point distribution structure between each adjacent layer at each scale.
[0043] It should be noted that the deformation caused by rotation will significantly change the local structure and texture features in the grayscale image of early childhood educational toys interactive monitoring, and the dominant direction of the feature point is obtained by the gradient direction distribution characteristics of all feature points in the neighborhood feature area of the feature point, so the feature point is highly sensitive to the gradient distribution; when the local structure and texture features of the grayscale image of early childhood educational toys interactive monitoring change significantly, its gradient distribution will also change significantly, and the direction allocation using the gradient histogram may not be able to correctly select the dominant direction, and the feature points on the adjacent images of each layer of each group of the Gaussian difference pyramid have a certain structural stability, and the feature point structures between layers have a certain similarity.
[0044] It should be further explained that, due to the scale invariance and rotation invariance of the feature descriptor in the SIFT feature point extraction process, it is mainly achieved by the accuracy of the dominant direction of the feature descriptor. However, when the local area of the feature point undergoes a large rotation transformation, that is, due to the constant changes in the different planes of the early childhood education toys and the viewing angle of the surveillance camera, the imaging area on the early childhood education toy interactive monitoring grayscale image can be regarded as a rotation change of varying degrees. When the rotation change amplitude is large, the texture distribution of the local area undergoes perspective distortion, causing it to lose its original regional texture distribution characteristics, which is manifested as a large change in the gradient of all feature points in the neighborhood feature area of the feature point, which is very different from the gradient direction and size of each part of the original area. Therefore, there will be a large error in obtaining the dominant direction of the feature point only by relying on the gradient of the feature point on a single layer of the image.
[0045] Therefore, thanks to the fact that each layer of each group of the Gaussian difference pyramid is obtained by Gaussian blur difference of the lower layer image, its differential image contains different high-frequency information points, but its scale-invariant feature points often have the same structural invariance, which makes the distribution structure of pixel points between layers similar, thereby obtaining the similarity of the distribution structure of feature points between layers. According to the similarity of the distribution structure of feature points between layers, the confidence level of the dominant directions of multiple feature points can be obtained through subsequent analysis to extract the best dominant direction.
[0046] Preset a parameter , wherein this embodiment is based on The present embodiment is not specifically limited. It depends on the specific implementation situation.
[0047] Specifically, the SIFT algorithm is used to obtain all feature points on the grayscale image of early childhood educational toy interaction monitoring;
[0048] It should be noted that the principle of the SIFT algorithm is to first obtain a Gaussian difference pyramid of the grayscale image of the early childhood educational toy interactive monitoring. The Gaussian difference pyramid contains several scales, each scale contains several levels, and feature points are obtained at different scale levels, that is, the feature points are distributed at different scale levels. At the same time, each feature point corresponds to a feature descriptor, and the feature descriptor has a direction, which is recorded as a feature point. Among them, the SIFT algorithm is an existing algorithm and will not be described in detail here.
[0049] Get the location coordinate information of feature points at different scales; set the distance threshold , with vertical upward as the initial direction and clockwise as the positive direction; for the feature points, in At the scale level where the feature points are located, feature points as the center point, and the distance threshold As the radius, the circular area is obtained as the The neighborhood of feature points; get the The feature point and The connection lines of all feature points in the neighborhood of the feature point are sorted from small to large by the angles between all the connection lines and the initial direction, and the first The neighborhood angle sequence of feature points is The neighborhood angle sequence of feature points is taken as the The multi-dimensional position vectors of feature points are obtained; until the multi-dimensional position vectors of all feature points in the feature point position matrix are obtained; similarly, the multi-dimensional position vectors of all feature points at different scales, among which, for repeated angles in the neighborhood angle sequence, they need to be deleted and only one is retained.
[0050] For The scale adjacent layers, through the The scale The similarity of the multidimensional position vectors of the feature points at the same position between adjacent layers is used to obtain the first The scale The distribution structure similarity of feature points between adjacent layers is calculated as follows:
[0051]
[0052] In the formula, Indicates The scale The similarity of the distribution structure of feature points between adjacent layers; Indicates The scale The first layer in the adjacent layers The multi-dimensional position vector of feature points; Indicates The scale The first layer of the adjacent feature points correspond to the multi-dimensional position vectors of the feature points in the second layer; Indicates the cosine similarity of vectors; Indicates The scale The number of feature points in the first layer among the adjacent layers.
[0053] It should be noted that if the multidimensional position vector With multidimensional position vector If the dimensions of the multidimensional position vectors are different, the multidimensional position vector with a smaller dimension is padded with 0 to make the dimensions of the two multidimensional position vectors consistent; if there are no feature points at the same position between adjacent layers, this implementation does not analyze this situation.
[0054] At this point, the similarity of the distribution structure of feature points between each adjacent layer at each scale is obtained.
[0055] Step S003: Obtain the confidence of the dominant direction of the feature points according to the similarity of the distribution structure of the feature points between the layers.
[0056] It should be noted that, because the conventional SIFT algorithm uses the gradient histogram to obtain the dominant direction of each feature point, the local gradient of the image undergoes a large rotation transformation, resulting in a large difference between the part of the dominant direction with a greater distortion effect and the direction in which the gradient should actually extend. The dominant direction obtained by the gradient on a single layer of the image will have a large error, making it impossible for the feature descriptor constructed thereby to accurately describe the local feature information.
[0057] Therefore, in order to solve the problem that there will be large errors in the dominant direction of the gradient on a single layer image, the confidence of the dominant direction of the feature point is obtained according to the similarity of the distribution structure of feature points between each adjacent layer at each scale and the difference of the dominant direction. It can use the similarity of the distribution structure of regional feature points between multiple layers to characterize the consistency of the change of the dominant direction of the feature point on multiple layers, and then obtain the confidence of the dominant direction of each feature point.
[0058] Preset a parameter , wherein this embodiment is based on The present embodiment is not specifically limited. It depends on the specific implementation situation.
[0059] Specifically, according to the SIFT algorithm, the feature vector of each feature point is obtained, and the feature vector is used as the dominant direction of the feature point. feature points, The feature point is the center point. At the scale level where the feature points are located, the distance threshold As the radius, the circular area is obtained as the The feature point corresponding to a local area; The direction sequence composed of the dominant directions of all feature points in the local area is taken as the first A sequence of directions in a local area.
[0060] It should be noted that the direction sequence of the local area is used to characterize the distribution characteristics of the feature points of the local area. The similarity comparison between the direction sequences of different local areas shows the similarity of the distribution structure of the feature points in the two local areas. Since the direction sequences of different local areas are different in length, The directional sequences of the two local areas are dynamically regularized, and their sequence similarity is used to characterize the distribution similarity of the feature points in the local areas. The Gaussian-weighted inter-layer feature point distribution structure similarity is used to characterize the confidence of the distribution similarity of the feature points in the local areas. The Gaussian-weighted inter-layer feature point distribution structure similarity is used here because the similarity of the inter-layer feature point distribution structure between adjacent layers has a greater impact. The greater the distance, the less correlation there is in the distribution structure of the inter-layer feature points.
[0061] Specifically, the Gaussian weighted inter-layer distance of each adjacent layer at each scale is obtained through the standard one-dimensional Gaussian function cv2.getGaussianKernel() in OPENCV, where the output of the one-dimensional Gaussian function is the distance between each adjacent layer at each scale. The scale The first layer in the adjacent layers The calculation expression of the dominant direction confidence of a feature point is:
[0062]
[0063] In the formula, Indicates The scale The first layer in the adjacent layers The dominant direction confidence of feature points; Indicates The scale Gaussian weighted inter-layer distances of adjacent layers; Indicates The scale The similarity of the distribution structure of feature points between adjacent layers; Indicates The scale The first layer in the adjacent layers A sequence of directions of local regions; Indicates The scale The total number of all local regions in the first layer among the adjacent layers; Indicates The scale The first layer in the adjacent layers The local regions correspond to the direction sequences of the local regions in the second layer; Indicates the similarity between two sequences; represents linear normalization.
[0064] At this point, the dominant direction confidence of each feature point in each adjacent layer is obtained.
[0065] Step S004: obtaining a direction correction factor of the feature point according to the dominant direction confidence of the feature point, constructing a feature descriptor, and completing the early childhood education toy target recognition in the early childhood education toy interactive monitoring grayscale image according to the feature descriptor.
[0066] It should be noted that in the SIFT algorithm, a feature descriptor can be constructed through each feature point. The feature descriptor has a direction, which is recorded as the dominant direction. In order to identify early childhood educational toy targets in early childhood educational toy images, this embodiment needs to correct the dominant direction of the feature point to obtain a feature descriptor after correcting the dominant direction. The specific correction of the dominant direction is as follows.
[0067] Preset a threshold , wherein this embodiment is based on The present embodiment is not specifically limited. It depends on the specific implementation situation.
[0068] Specifically, for The next adjacent layer The dominant direction of the feature point, if The next adjacent layer The dominant direction confidence of feature points is less than the threshold , then the feature point is recorded as the feature point that needs to be corrected in the dominant direction; if The next adjacent layer The dominant direction confidence of feature points is greater than or equal to the threshold , then the feature point is recorded as a feature point that does not require dominant direction correction; similarly, all feature points that require dominant direction correction are obtained.
[0069] For the first feature points, get the The dominant direction of the feature points in each adjacent layer is sorted from top to bottom as the The dominant direction sequence of feature points is The average of all dominant directions that need to be corrected in the dominant direction sequence of feature points is recorded as the first direction; The average of all the dominant directions that do not need to be corrected in the dominant direction sequence of the feature points is recorded as the second direction; the average of the first direction and the second direction is recorded as the first The dominant direction correction factor of each feature point is obtained; similarly, the dominant direction correction factors of all feature points that need dominant direction correction are obtained.
[0070] For feature point, if The dominant direction of feature points in each adjacent layer does not need to be corrected, so the first The dominant direction average of the feature points in all adjacent layers is taken as the The dominant direction after the feature point is corrected; if the If the dominant direction of a feature point in a certain adjacent layer needs to be corrected, then The dominant direction correction factor of the feature point is The product of the mean of the dominant directions of the feature points in all adjacent layers is taken as the The corrected dominant direction of the feature points is obtained; similarly, the corrected dominant directions of all feature points are obtained.
[0071] Through the SIFT algorithm, the feature descriptor is constructed using the corrected dominant direction of each feature point. The feature descriptors are matched on adjacent frame images of early childhood educational toy interaction monitoring using the nearest neighbor distance and next nearest neighbor distance methods to obtain the feature descriptor matching results. If the feature descriptor matches successfully, the location of the feature point corresponding to the feature descriptor is the early childhood educational toy target.
[0072] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. An early childhood education toy image recognition method based on surveillance video, characterized in that: The method comprises the following steps: Get the grayscale image of early childhood education toy interaction monitoring; Construct the Gaussian difference scale space of the grayscale image of early childhood educational toy interaction monitoring to obtain feature points at different scale levels; obtain the multidimensional position vectors of all feature points at different scale levels; obtain the similarity of the distribution structure of feature points between each adjacent layer at each scale based on the multidimensional position vectors of all feature points at different scale levels; Obtain the direction sequence of the local area corresponding to each feature point; obtain the dominant direction confidence of each feature point in each adjacent layer according to the direction sequence of the local area corresponding to each feature point and the similarity of the distribution structure of feature points between each adjacent layer at each scale; According to the dominant direction confidence of each feature point in each adjacent layer, all feature points that need to be corrected in dominant direction are obtained; dominant direction correction factors of all feature points that need to be corrected in dominant direction are obtained; according to the dominant direction correction factors of all feature points that need to be corrected in dominant direction, the corrected dominant directions of all feature points are obtained; feature descriptors are constructed according to the corrected dominant directions of all feature points, and early childhood education toy target recognition in the early childhood education toy interactive monitoring grayscale image is completed according to the feature descriptors; The specific steps of obtaining the multi-dimensional position vectors of all feature points at different scale levels are as follows: Preset distance threshold , with vertical upward as the initial direction; for the feature points, in At the scale level where the feature points are located, feature points as the center point, and the distance threshold As the radius, the circular area is obtained as the The neighborhood of feature points; get the The feature point and The connection lines of all feature points in the neighborhood of the feature point are sorted from small to large by the angles between all the connection lines and the initial direction, and the first The neighborhood angle sequence of feature points is The neighborhood angle sequence of feature points is taken as the The multi-dimensional position vector of feature points.
2. The early childhood education toy image recognition method based on surveillance video according to claim 1 is characterized in that: The Gaussian difference scale space of the early childhood educational toy interactive monitoring grayscale image is constructed to obtain feature points at different scale levels, and the specific steps include the following: The SIFT algorithm is used to construct the Gaussian difference scale space of the grayscale image of early childhood educational toy interaction monitoring, and then generate a Gaussian difference pyramid. The Gaussian difference pyramid contains several scale levels, and feature points at different scale levels are obtained on the Gaussian difference pyramid.
3. The early childhood education toy image recognition method based on surveillance video according to claim 1 is characterized in that: The specific steps of obtaining the similarity of the distribution structure of feature points between each adjacent layer at each scale according to the multi-dimensional position vectors of all feature points at different scales are as follows: For The scale adjacent layers, through the The scale The similarity of the multidimensional position vectors of the feature points at the same position between adjacent layers is used to obtain the first The scale The distribution structure similarity of feature points between adjacent layers is calculated as follows: In the formula, Indicates The scale The similarity of the distribution structure of feature points between adjacent layers; Indicates The scale The first layer in the adjacent layers The multi-dimensional position vector of feature points; Indicates The scale The first layer of the adjacent feature points correspond to the multi-dimensional position vectors of the feature points in the second layer; Indicates the cosine similarity of vectors; Indicates The scale The number of feature points in the first layer among the adjacent layers.
4. The early childhood education toy image recognition method based on surveillance video according to claim 1, characterized in that: The specific steps of obtaining the direction sequence of the local area corresponding to each feature point include the following: According to the SIFT algorithm, the dominant direction of each feature point is obtained. feature points, feature points as the center point, with the distance to the preset threshold As the radius, the circular area is obtained as the The feature point corresponding to a local area; The direction sequence composed of the dominant directions of all feature points in the local area is taken as the first A sequence of directions in a local area.
5. The early childhood education toy image recognition method based on surveillance video according to claim 1, characterized in that: The method of obtaining the dominant direction confidence of each feature point in each adjacent layer according to the direction sequence of the local area corresponding to each feature point and the similarity of the distribution structure of feature points between each adjacent layer at each scale includes the following specific steps: Through the standard one-dimensional Gaussian function in OPENCV, the Gaussian weighted inter-layer distance of each adjacent layer at each scale is obtained. The scale The first layer in the adjacent layers The calculation expression of the dominant direction confidence of a feature point is: In the formula, Indicates The scale The first layer in the adjacent layers The dominant direction confidence of feature points; Indicates The scale Gaussian weighted inter-layer distances of adjacent layers; Indicates The scale The similarity of the distribution structure of feature points between adjacent layers; Indicates The scale The first layer in the adjacent layers A sequence of directions of local regions; Indicates The scale The first layer in the adjacent layers The local regions correspond to the direction sequences of the local regions in the second layer; Indicates The scale The total number of all local regions in the first layer among the adjacent layers; Indicates the similarity between two sequences; represents linear normalization.
6. The early childhood education toy image recognition method based on surveillance video according to claim 1, characterized in that: The specific steps of obtaining all feature points that need to be corrected in the dominant direction according to the dominant direction confidence of each feature point in each adjacent layer are as follows: Preset Threshold , for the The next adjacent layer The dominant direction of the feature point, if The next adjacent layer The dominant direction confidence of feature points is less than the threshold , then the feature point is recorded as the feature point that needs to be corrected in the dominant direction; Jordi The next adjacent layer The dominant direction confidence of feature points is greater than or equal to the threshold , then the feature point is recorded as a feature point that does not require dominant direction correction; Similarly, all feature points that need to be corrected in the dominant direction are obtained.
7. The early childhood education toy image recognition method based on surveillance video according to claim 1, characterized in that: The specific steps of obtaining the dominant direction correction factors of all feature points that need to be corrected in the dominant direction are as follows: For the first feature points, get the The dominant direction of the feature points in each adjacent layer is sorted from top to bottom as the The dominant direction sequence of feature points is The average of all dominant directions that need to be corrected in the dominant direction sequence of feature points is recorded as the first direction; The average of all the dominant directions that do not need to be corrected in the dominant direction sequence of the feature points is recorded as the second direction; the average of the first direction and the second direction is recorded as the first The dominant direction correction factor of each feature point.
8. The early childhood education toy image recognition method based on surveillance video according to claim 6, characterized in that: The method of obtaining the corrected dominant directions of all feature points according to the dominant direction correction factors of all feature points that need to be corrected in dominant directions includes the following specific steps: For feature point, if feature points in each adjacent layer are feature points that do not require dominant direction correction, then The dominant direction average of the feature points in all adjacent layers is taken as the The dominant direction after the feature point is corrected; if the If a feature point is a feature point that needs to be corrected in the dominant direction in any adjacent layer, then The dominant direction correction factor of the feature point is The product of the mean of the dominant directions of the feature points in all adjacent layers is taken as the The corrected dominant direction of the feature points is obtained; similarly, the corrected dominant directions of all feature points are obtained.
9. The early childhood education toy image recognition method based on surveillance video according to claim 1, characterized in that: The feature descriptor is constructed according to the corrected dominant directions of all feature points, and the early childhood education toy target recognition in the early childhood education toy interactive monitoring grayscale image is completed according to the feature descriptor, including the following specific steps: Through the SIFT algorithm, the feature descriptor is constructed using the corrected dominant direction of each feature point. The feature descriptors are matched on adjacent frame images of early childhood educational toy interaction monitoring using the nearest neighbor distance and next nearest neighbor distance methods to obtain the feature descriptor matching results. If the feature descriptor matches successfully, the location of the feature point corresponding to the feature descriptor is the early childhood educational toy target.
