Interaction control method and system based on robot control

Abstract

The invention provides an interaction control method and system based on robot control, and relates to the technical field of robot control, and the method comprises the steps: carrying out the differential processing of a visible light image and an infrared image in an early education environment, and extracting a suspicious target region in a differential image; designing a feature descriptor for the suspicious target area, and performing feature analysis on the suspicious target area according to the feature descriptor to obtain a feature analysis result; and according to a feature analysis result, performing preliminary discrimination on the suspicious target area, and processing a discriminated target set through a clustering algorithm to obtain a target list. According to the method, the accuracy of target recognition in the early education environment is improved, the early education robot can execute personalized interaction response according to the recognition result, and education interactivity and interestingness are enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of robot control, and particularly to an interactive control method and system based on robot control. Background Art

[0002] Traditional interaction methods of early education robots are often relatively single and lack sufficient intelligence. For example, some early education robots only support simple touch interaction methods, and users need to interact with the robot by touching the screen. Although this interaction method is intuitive and easy to use, it limits the operation freedom and flexibility of users and cannot fully utilize the potential of artificial intelligence technology.

[0003] In addition, although some early education robots are equipped with voice recognition functions, there are still many problems in actual applications. These robots often cannot accurately recognize users' voice commands, especially in noisy environments or when users' pronunciation is unclear. At the same time, they also lack the ability to make intelligent adjustments according to the specific situation of users. For example, when the distance between the user and the robot changes, the robot cannot automatically adjust the volume or recognition sensitivity; in environments with different light intensities, the screen brightness and display content of the robot cannot make corresponding adjustments, thus affecting the user experience. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide an interactive control method and system based on robot control, which can accurately extract suspicious target areas and improve the accuracy and efficiency of target detection.

[0005] To solve the above technical problem, the technical solution of the present invention is as follows:

[0006] In the first aspect, an interactive control method based on robot control, the method includes:

[0007] Perform differential processing on visible light images and infrared images in the early education environment, and extract suspicious target areas in the differential images;

[0008] For the suspicious target areas, design feature descriptors, and perform feature analysis on the suspicious target areas according to the feature descriptors to obtain feature analysis results;

[0009] According to the feature analysis results, perform preliminary discrimination on the suspicious target areas, and process the discriminated target set through a clustering algorithm to obtain a target list;

[0010] According to the target list, construct a convolutional neural network model and a random forest model, and respectively use visible light images and infrared images to extract features and identify the targets in the target list to obtain preliminary identification results;

[0011] Compare the preliminary recognition results with the target information in the preset database of the early education robot, and use the flower pollination algorithm to optimize and adjust the parameters of the convolutional neural network model and the random forest model to obtain the final recognition results and target position information;

[0012] According to the final recognition results and target position information, the early education robot executes corresponding interaction responses, including playing educational content related to the recognized target, having a conversation with the child, and displaying expression animations related to the recognized target.

[0013] Furthermore, perform differential processing on the visible light image and the infrared image in the early education environment, and extract the suspicious target areas in the differential image, including:

[0014] Use the camera device built into the early education robot to collect the visible light image and the infrared image in the early education environment;

[0015] Perform differential processing on the visible light image and the infrared image by calculating the difference in pixel values between the visible light image and the infrared image to obtain the differentially processed image;

[0016] Analyze the differentially processed image, including threshold segmentation, edge detection, and morphological processing, to identify the suspicious target areas in the early education environment, including dropped toys and misplaced teaching aids.

[0017] Furthermore, for the suspicious target area, design a feature descriptor, and perform feature analysis on the suspicious target area according to the feature descriptor to obtain the feature analysis results, including:

[0018] For the circularity feature, according to the preset threshold, judge whether the shape of the suspicious target area conforms to the typical shape of objects in the early education environment, including circular toys and spherical teaching aids; for the highlight area area feature, analyze the area size of the highlight area and its proportion in the suspicious target area to identify objects with reflective characteristics, including metal toys and mirror teaching aids; for the gray level change feature before and after infrared illumination, analyze the amplitude and trend of the gray level value change to judge whether the suspicious target area includes objects with specific thermal characteristics, including heating electronic devices and heat-sensitive materials; for the local contrast feature, judge the distinguishability of the suspicious target area in the image from the surrounding environment according to the strength of the contrast.

[0019] Obtain the feature analysis results according to the analysis of circularity, highlight area area, gray level change before and after infrared illumination, and local contrast.

[0020] Furthermore, according to the feature analysis results, conduct a preliminary discrimination on the suspicious target area, and process the discriminated target set through a clustering algorithm to obtain a target list, including:

[0021] Set a discrimination threshold. When the result of the feature analysis of the suspicious target area is ≥ the discrimination threshold, the corresponding feature is determined as a camera target, and the corresponding feature is added to the target set after discrimination;

[0022] According to the feature descriptors of the targets in the target set after discrimination, including position, shape, and size, set clustering parameters, including the number of clusters and the distance threshold;

[0023] Divide the target set after discrimination into different clustering clusters, and for each clustering cluster, extract the cluster center;

[0024] Arrange the cluster centers of all clustering clusters in order to form a target list.

[0025] Furthermore, according to the target list, construct a convolutional neural network model and a random forest model, and respectively use visible light images and infrared images to extract features and identify the targets in the target list to obtain preliminary recognition results, including:

[0026] According to the target list, construct a convolutional neural network model, including a convolutional layer, a pooling layer, and a fully connected layer; input the visible light image into the convolutional neural network model, and through the operations of the convolutional layer and the pooling layer, extract the high-level semantic features in the image, including shape, color pattern, and texture, and input the high-level semantic features into the fully connected layer of the convolutional neural network model for classification, and output the preliminary recognition results of the visible light image, including target type, position, and confidence information;

[0027] According to the target list, construct a random forest model; determine the infrared image corresponding to the target from the target list as training data, and extract features from the image, including calculating temperature distribution and infrared reflectivity underlying texture features; input the infrared image features into the random forest model, and each decision tree in the random forest model makes an independent judgment according to the features, and outputs the preliminary recognition results of the infrared image, including target type, position, and confidence information.

[0028] Integrate the preliminary recognition results of the visible light image and the preliminary recognition results of the infrared image to obtain the integrated preliminary recognition results.

[0029] Furthermore, compare the preliminary recognition results with the target information in the preset database of the early education robot, and use the flower pollination algorithm to optimize and adjust the parameters of the convolutional neural network model and the random forest model to obtain the final recognition results and target position information, including:

[0030] Compare the preliminary recognition results with the target information in the preset database of the early education robot to determine the differences between the preliminary recognition results and the targets;

[0031] Initialize the relevant parameters of the flower pollination algorithm according to the differences between the preliminary recognition results and the targets, including the number of flowers, the pollination probability, and the local search radius, and optimize the parameters of the convolutional neural network model and the random forest model. Among them, each flower performs local search and global pollination according to the evaluation function to obtain the optimized parameters output by the flower pollination algorithm.

[0032] Re-recognize the targets in the target list according to the optimized parameters output by the flower pollination algorithm to obtain new recognition results.

[0033] Perform a comparison operation again between the new recognition results and the target information in the preset database of the early education robot to calculate the new error.

[0034] Repeat the process of evaluation function design, parameter optimization, re-recognition, and error calculation until the preset number of iterations is reached. Take the recognition results obtained in the last iteration as the final recognition results and target position information.

[0035] Furthermore, the calculation process of the evaluation function is as follows:

[0036] Traverse all target types. For each target type, compare it with the predicted type. If the target type is the same as the predicted type, set the indication value of the corresponding situation to 1; otherwise, set it to 0.

[0037] Accumulate all the situations with an indication value of 1 to obtain the number of correctly recognized target types, and combine the number of target types with the preset weight coefficient to obtain the weighted value of the type recognition accuracy.

[0038] Traverse all target positions. For each target position, compare it with the predicted position. If the target position is the same as the predicted position, set the indication value of the corresponding situation to 1; otherwise, set it to 0.

[0039] Accumulate all the situations with an indication value of 1 to obtain the number of correctly recognized target positions, and fuse the number of target positions with the preset weight coefficient to obtain the weighted value of the position recognition accuracy.

[0040] Traverse all recognition results. For each recognition result, calculate the absolute value of the difference between the confidence level and the matching probability.

[0041] According to the absolute value and the weight of the corresponding recognition result, obtain the total sum of the weighted confidence level errors.

[0042] According to the total sum of the weighted confidence level errors and the normalization factor, obtain the normalized confidence level error, and fuse the normalized confidence level error with the preset weight coefficient to obtain the weighted value of the confidence level error.

[0043] Obtain a preliminary evaluation value based on the weighted value of the type recognition accuracy and the weighted value of the position recognition accuracy;

[0044] On the basis of the preliminary evaluation value, adjust the weighted value of the confidence error to obtain an unnormalized evaluation result;

[0045] Adjust the ratio of the unnormalized evaluation result to the total number of recognition targets for normalization processing, and finally obtain the value of the evaluation function.

[0046] In a second aspect, an interactive control system based on robot control includes:

[0047] A differential processing module for performing differential processing on visible light images and infrared images in the early education environment to extract suspicious target areas in the differential images;

[0048] A feature analysis module for designing feature descriptors for the suspicious target areas and performing feature analysis on the suspicious target areas according to the feature descriptors to obtain feature analysis results;

[0049] A discrimination and clustering module for performing preliminary discrimination on the suspicious target areas according to the feature analysis results to obtain a set of discriminated targets, and processing the set of discriminated targets through a clustering algorithm to obtain a target list;

[0050] An identification module for constructing a convolutional neural network model and a random forest model according to the target list, and respectively using visible light images and infrared images to extract features and identify the targets in the target list to obtain preliminary identification results;

[0051] A comparison and optimization module for comparing the preliminary identification results with the target information in the preset database of the early education robot, and using an optimization algorithm to optimize and adjust the parameters of the convolutional neural network model and the random forest model to obtain the final identification results and target position information;

[0052] An interactive response module for the early education robot to perform corresponding interactive responses according to the final identification results and target position information, including playing educational content related to the identified target, having a conversation with the child, and displaying expression animations related to the identified target.

[0053] In a third aspect, a computing device includes:

[0054] One or more processors;

[0055] A storage device for storing one or more programs, which when executed by the one or more processors cause the one or more processors to implement the described method.

[0056] Fourth aspect, a computer-readable storage medium stores a program, which when executed by a processor implements the method described above.

[0057] The above solution of the present invention has at least the following beneficial effects:

[0058] By performing differential processing on visible light images and infrared images in the early education environment, the differences between the two can be effectively highlighted, so as to accurately extract the suspicious target area, improve the accuracy and efficiency of target detection, and reduce the possibility of false detection and missed detection. Design specific feature descriptors for the suspicious target area and perform feature analysis to obtain detailed feature analysis results, providing rich feature information for preliminary discrimination and clustering processing, and improving the accuracy of recognition. According to the feature analysis results, conduct preliminary discrimination on the suspicious target area, and process the discriminated target set through a clustering algorithm to obtain a clear target list, which helps to classify and organize the targets. Construct a convolutional neural network model and a random forest model, respectively use visible light images and infrared images to extract features and identify the targets in the target list, combine the advantages of the two different models, and improve the robustness and accuracy of recognition.

[0059] Compare the preliminary recognition results with the target information in the preset database of the early education robot, and use the flower pollination algorithm to optimize and adjust the parameters of the convolutional neural network model and the random forest model. As an intelligent optimization algorithm, the flower pollination algorithm can effectively find the optimal solution of the model parameters, further improving the accuracy of recognition. According to the final recognition results and target position information, the early education robot executes corresponding interaction responses, including playing educational content related to the recognized target, having a conversation with the child, and displaying expression animations related to the recognized target, enhancing the interactivity and interest of the early education robot, improving the learning interest and participation of children, and at the same time, demonstrating the intelligent and personalized service characteristics of the early education robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 FIG. is a schematic flowchart of an interaction control method based on robot control provided by an embodiment of the present invention.

[0061] Figure 2 FIG. is a schematic diagram of an interaction control system based on robot control provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0062] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be fully conveyed to those skilled in the art.

[0063] As Figure 1 shown, an embodiment of the present invention provides an interactive control method based on robot control, and the method includes the following steps:

[0064] Step 1: Perform differential processing on visible light images and infrared images in the early education environment, and extract suspicious target areas in the differential images;

[0065] Step 2: Design feature descriptors for the suspicious target areas, and perform feature analysis on the suspicious target areas according to the feature descriptors to obtain a feature analysis result;

[0066] Step 3: According to the feature analysis result, perform a preliminary discrimination on the suspicious target areas, and process the discriminated target set through a clustering algorithm to obtain a target list;

[0067] Step 4: According to the target list, construct a convolutional neural network model and a random forest model, and respectively use visible light images and infrared images to extract features and identify the targets in the target list to obtain a preliminary identification result;

[0068] Step 5: Compare the preliminary identification result with the target information in the preset database of the early education robot, and use the flower pollination algorithm to optimize and adjust the parameters of the convolutional neural network model and the random forest model to obtain the final identification result and the target position information;

[0069] Step 6: According to the final identification result and the target position information, the early education robot executes corresponding interactive responses, including playing educational content related to the identified target, having a conversation with the child, and displaying an expression animation related to the identified target.

[0070] In the embodiments of the present invention, through differential processing and feature analysis techniques, it is possible to more accurately extract suspicious target areas in the early education environment, and perform feature analysis based on rich feature descriptors, thereby improving the accuracy of target recognition. Combining with the interaction response ability of the early education robot, corresponding interaction operations can be executed according to the recognition results, such as playing educational content related to the recognized target, having a simple conversation with children, etc., greatly enhancing the interactivity and educational effect of the early education robot. Using machine learning models and optimization algorithms to continuously optimize and adjust the recognition model can continuously improve the intelligent level of the early education robot, enabling it to better adapt to different scenarios and user needs. This method is not only applicable to the early education environment, but also can be extended to other scenarios that require intelligent recognition and interaction, providing a broader application prospect for the development of intelligent robot technology.

[0071] In a preferred embodiment of the present invention, step 1, performing differential processing on the visible light image and the infrared image in the early education environment and extracting the suspicious target area in the differential image may include:

[0072] Step 112, using the camera device built in the early education robot to collect the visible light image and the infrared image in the early education environment;

[0073] Step 113, performing differential processing on the visible light image and the infrared image by calculating the difference in pixel values between the visible light image and the infrared image to obtain the differentially processed image;

[0074] Step 114, analyzing the differentially processed image, including threshold segmentation, edge detection, and morphological processing, to identify the suspicious target areas in the early education environment, including dropped toys and unreturned teaching aids.

[0075] In the embodiments of the present invention, the early education robot is built with a high-definition visible light camera and an infrared camera to ensure that both images can be collected simultaneously. The visible light camera collects the color image of the early education environment at a standard resolution (such as 1080p), recording visual information such as the illumination and color of the environment. The infrared camera uses an infrared light source and an infrared sensor to collect the infrared radiation image in the environment, mainly for identifying the thermal characteristics of objects, such as body temperature, heat sources, etc. The central processing unit (CPU) or the graphics processing unit (GPU) of the early education robot controls the synchronous collection of the two cameras to ensure the temporal consistency of the two images, facilitating subsequent differential processing.

[0076] Step 113: Perform grayscale processing on the collected visible light image and infrared image to convert the color image into a grayscale image. Normalize the grayscale image to unify the pixel value range to between [0, 1] to reduce the influence of illumination changes on the differential result. Perform pixel-by-pixel difference calculation on the preprocessed visible light image and infrared image, that is, for each pixel point (x, y), calculate the difference in its grayscale value. Map the difference result to a new grayscale image to form the image after differential processing. The highlighted areas in the differential image represent the areas with significant differences between the two images, which are the suspicious target areas.

[0077] Step 114: Set a suitable threshold T to divide the pixel values in the differential image into two categories: pixel points greater than T are considered to belong to the suspicious target area, and pixel points ≤ T are considered to belong to the background area. Through binary processing, convert the differential image into a binary image. Apply an edge detection algorithm (such as Canny edge detection) to process the binary image to identify the edge contour of the suspicious target area, and the edge contour provides important geometric feature information for subsequent morphological processing. Apply morphological opening operation (erosion first and then dilation) and closing operation (dilation first and then erosion) to process the image after edge detection to remove noise and small holes and fill cracks and narrow gaps. Through the image processed by morphology, the suspicious target area in the early education environment, such as dropped toys and misplaced teaching aids, can be more clearly identified.

[0078] Suppose an early education robot is working in a children's room, and there is a red toy ball dropped on the ground and a misplaced building block teaching aid in the room. The visible light camera and infrared camera built into the early education robot simultaneously collect images in the room. The visible light camera captures the colorful scene in the room, including the red toy ball and the building block teaching aid; the infrared camera captures the thermal radiation distribution in the room. Since the temperatures of the toy ball and the building block teaching aid are different from the surrounding environment, they will also be reflected in the infrared image. Perform differential processing on the collected visible light image and infrared image. Since the positions and grayscale values of the toy ball and the building block teaching aid are different in the two images, they will be highlighted in the differential image. Perform threshold segmentation, edge detection, and morphological processing on the image after differential processing. By setting a suitable threshold, convert the differential image into a binary image; then apply an edge detection algorithm to identify the edge contour of the suspicious target area; finally, remove noise and fill cracks through morphological processing to obtain a clear image of the suspicious target area. Based on the recognition result, the early education robot can judge that there are a dropped toy ball and a misplaced building block teaching aid in the room and remind children or parents to tidy up.

[0079] Through differential processing and image analysis techniques, it is possible to more accurately identify suspicious target areas in the early education environment, such as dropped toys, unreturned teaching aids, etc., reducing the occurrence of misjudgments and missed judgments. The early education robot can timely discover and remind children or parents to handle potential safety hazards, such as tripping or accidental ingestion caused by small objects dropped on the ground, improving the safety of the early education environment. The early education robot can provide personalized interaction services according to the recognition results, such as playing educational content related to the recognized target, guiding children to put items in place, etc., enhancing the user's interaction experience and educational effect. This method provides a new technical path and solution for the intelligent development of early education robots, contributing to the continuous progress and application expansion of early education robot technology.

[0080] In a preferred embodiment of the present invention, in step 2, for the suspicious target area, a feature descriptor is designed, and feature analysis is performed on the suspicious target area according to the feature descriptor to obtain a feature analysis result, including:

[0081] Step 221, for the circularity feature, according to a preset threshold, determine whether the shape of the suspicious target area conforms to the typical shape of objects in the early education environment, including circular toys and spherical teaching aids; for the highlight area area feature, analyze the area size of the highlight area and its proportion in the suspicious target area to identify objects with reflective characteristics, including metal toys and mirror teaching aids; for the gray-scale change feature before and after infrared illumination, analyze the amplitude and trend of the gray-scale value change to determine whether the suspicious target area includes objects with specific thermal characteristics, including heating electronic devices and heat-sensitive materials; for the local contrast feature, according to the strength of the contrast, determine the distinguishability of the suspicious target area in the image from the surrounding environment.

[0082] Step 222, obtain a feature analysis result according to the analysis of circularity, highlight area area, gray-scale change before and after infrared illumination, and local contrast.

[0083] In an embodiment of the present invention, edge detection and contour extraction are performed on the image of the suspicious target area after differential processing. For each contour, its circularity is calculated (such as using a shape factor, circularity index, etc.). Circularity is an index that measures the degree to which a contour approaches a circle. The calculated circularity is compared with a preset threshold. If the circularity is higher than the threshold, it is considered that the shape of the suspicious target area conforms to the typical shape of objects in the early education environment (such as circular toys and spherical teaching aids), and the judgment result is recorded as part of the feature analysis result. Brightness analysis is performed on the image of the suspicious target area to identify the highlight area. Calculate the area size of the highlight area and its proportion in the suspicious target area. According to the area and proportion of the highlight area, determine whether the suspicious target area has reflective characteristics (such as metal toys and mirror teaching aids), and record the judgment result as part of the feature analysis result.

[0084] Compare the grayscale values of the visible light image and the infrared image before differential processing, and calculate the amplitude and trend of the grayscale change in the suspicious target area before and after infrared illumination. According to the amplitude and trend of the grayscale change, determine whether the suspicious target area includes objects with specific thermal characteristics (such as a heating electronic device, heat-sensitive material). Record the judgment result as part of the feature analysis result. Conduct a contrast analysis on the suspicious target area and its surrounding environment, and calculate the strength of the local contrast. According to the strength of the contrast, judge the distinguishability of the suspicious target area from the surrounding environment in the image, and record the judgment result as part of the feature analysis result.

[0085] Step 222: Integrate the analysis results of circularity, highlight area, grayscale change before and after infrared illumination, and local contrast to form a complete feature analysis result.

[0086] Suppose the early education robot is working in a children's room and detects a suspicious target area. The robot identifies the contour of the suspicious target area through edge detection and contour extraction, calculates the circularity of the contour, and finds that its value is higher than the preset threshold, determining that the shape of the suspicious target area conforms to the typical shape of a circular toy in the early education environment. The robot conducts a brightness analysis on the suspicious target area, identifies the highlight area, calculates the area and proportion of the highlight area, and finds that the proportion is relatively large, determining that the suspicious target area may have reflective characteristics, such as a metal toy. The robot compares the images before and after differential processing, calculates the amplitude and trend of the grayscale change in the suspicious target area, and finds that the amplitude of the grayscale change is small but the trend is steadily increasing, determining that the suspicious target area may include objects with specific thermal characteristics, such as a slowly heating electronic device. The robot conducts a contrast analysis on the suspicious target area and its surrounding environment, and finds that the contrast between the suspicious target area and the surrounding environment is strong, determining that the suspicious target area is prominent and easy to identify in the image. The robot integrates the above analysis results to form a complete feature analysis result.

[0087] By designing multiple feature descriptors and conducting comprehensive analysis, it is possible to more accurately identify suspicious target areas in the early education environment and reduce the occurrence of false positives and false negatives. This method can identify suspicious target areas with different characteristics, such as shape, reflective characteristics, thermal characteristics, and contrast, enhancing the recognition ability of the early education robot. This method provides new technical means and solutions for the intelligent application of early education robots, contributing to the continuous progress and application expansion of early education robot technology. By accurately identifying and alerting users to suspicious target areas, it is possible to improve the user experience and educational effect of early education robots, enhancing users' trust and satisfaction with early education robots.

[0088] In a preferred embodiment of the present invention, in step 3, according to the feature analysis result, the suspicious target area is preliminarily discriminated, and the discriminated target set is processed by a clustering algorithm to obtain a target list, which may include:

[0089] Step 331, set a discrimination threshold. When the feature analysis result of the suspicious target area ≥ the discrimination threshold, the corresponding feature is determined as a camera target, and the corresponding feature is added to the discriminated target set;

[0090] Step 332, according to the feature descriptors of the targets in the discriminated target set, including position, shape, and size, set clustering parameters, including the number of clusters and the distance threshold;

[0091] Step 333, divide the discriminated target set into different clusters, and for each cluster, extract the cluster center;

[0092] Step 334, arrange the cluster centers of all clusters in order to form a target list.

[0093] In the embodiment of the present invention, the early education robot sets a discrimination threshold according to its preset safety standards and monitoring requirements. This threshold is used to determine whether the suspicious target area has the characteristics of a camera target. The early education robot captures environmental images through a camera and preprocesses and analyzes the features of the images. For each suspicious target area, the robot calculates its feature values (such as edge density, reflection intensity, infrared thermal imaging features, etc.). The calculated feature value is compared with the discrimination threshold. If the feature value is greater than or equal to the threshold, it is determined that the suspicious target area is a camera target. The features corresponding to the area determined to be a camera target (such as position coordinates, shape description, size dimensions, etc.) are added to the discriminated target set.

[0094] Step 332, the early education robot performs feature descriptor analysis on the discriminated target set to understand the distribution of these camera targets in terms of position, shape, size, etc. According to the analysis result, the robot can determine whether the camera targets show aggregation and the degree of aggregation. According to the feature distribution of the target set and the application requirements of the early education robot, set the number of clusters (i.e., the expected number of clusters to divide the camera targets into) and the distance threshold (i.e., the distance standard for determining whether two camera targets belong to the same cluster). The number of clusters can be determined based on the number, distribution density, and recognition accuracy of the camera targets. The distance threshold can be determined based on the position distance, shape similarity, or size difference between the camera targets.

[0095] Step 333: The early education robot uses clustering algorithms (such as K-means, DBSCAN, etc.) to perform clustering processing on the discriminated target set. According to the set number of clusters and distance threshold, the camera targets are divided into different clusters, and each cluster represents a group of camera targets that are similar in terms of position, shape, size, etc. For each cluster, the early education robot calculates its cluster center. The cluster center can be the average of the position coordinates of all camera targets in the cluster, the center point of the shape description, or the median of the size dimensions, etc. The cluster center serves as the representative of the cluster.

[0096] Step 334: The early education robot arranges the cluster centers of all clusters in order. The arrangement order can be based on the position coordinates of the cluster centers, the priority of the shape description, or the level of the size dimensions, etc. The arranged cluster centers form an ordered target list, which contains the cluster center information of all the target sets determined as camera targets in the early education environment. The robot can display the target list on the user interface for the user to view and process.

[0097] Suppose the early education robot detects multiple suspicious target areas in a children's room and conducts feature analysis.

[0098] The robot sets the discrimination threshold to 0.7 (determined based on historical data analysis). The robot captures the room image, preprocesses and analyzes the features of the image, and finds that the feature values of three areas are ≥0.7, which are determined as camera targets. The features of these three camera targets are added to the discriminated target set. The robot conducts feature descriptor analysis on the discriminated target set and finds that the camera targets show aggregation at different positions in the room. The number of clusters is set to 2 (expecting to divide the camera targets into two clusters), and the distance threshold is set to 0.2 meters (determined based on the position distance between camera targets). The K-means clustering algorithm is used to perform clustering processing on the discriminated target set. The camera targets are divided into two clusters, located in the upper left and lower right corners of the room respectively. The cluster centers of each cluster are calculated to obtain the position coordinates of the two cluster centers. The cluster centers of the two clusters are arranged in order (such as sorted from left to right according to the position of the cluster centers). A target list is formed, which contains the position coordinate information of the cluster centers of the two clusters, and the target list is displayed on the user interface to remind the user to pay attention to the camera targets in the room.

[0099] Through preliminary discrimination and clustering processing, the early education robot can quickly and accurately identify the camera targets in the room, improving the monitoring efficiency. The early education robot can promptly detect and remind users to pay attention to the camera targets in the room, enhancing the safety guarantee for children. By displaying the target list, the early education robot can enable users to more intuitively understand the camera distribution in the room, optimizing the user experience. This method provides new technical means and solutions for the intelligent application of early education robots, contributing to the continuous progress and application expansion of early education robot technology. For example, the early education robot can perform more refined monitoring and management based on the camera target information in the target list.

[0100] In a preferred embodiment of the present invention, step 4, according to the target list, constructing a convolutional neural network model and a random forest model, and respectively using visible light images and infrared images to extract features and identify the targets in the target list to obtain a preliminary recognition result, may include:

[0101] Step 441, according to the target list, constructing a convolutional neural network model, including a convolutional layer, a pooling layer, and a fully connected layer; inputting the visible light image into the convolutional neural network model, through the operations of the convolutional layer and the pooling layer, extracting the high-level semantic features in the image, including shape, color pattern, and texture, and inputting the high-level semantic features into the fully connected layer of the convolutional neural network model for classification, and outputting the preliminary recognition result of the visible light image, including target type, position, and confidence information;

[0102] Step 442, according to the target list, constructing a random forest model; determining the infrared image corresponding to the target in the target list as training data, and performing feature extraction on the image, including calculating the temperature distribution and the underlying texture features of the infrared reflectivity; inputting the infrared image features into the random forest model, and each decision tree in the random forest model makes an independent judgment based on the features, and outputs the preliminary recognition result of the infrared image, including target type, position, and confidence information.

[0103] Step 443, integrating the preliminary recognition result of the visible light image and the preliminary recognition result of the infrared image to obtain the integrated preliminary recognition result.

[0104] In the embodiment of the present invention, according to the characteristics of the target list in the monitoring environment of the early education robot, a convolutional neural network (CNN) model suitable for processing visible light images is designed. It includes an input layer (for receiving visible light images), multiple convolutional layers (for extracting spatial features of the images), a pooling layer (for reducing the dimension of the feature map and retaining important features), a fully connected layer (for mapping features to target categories), and an output layer (for giving the recognition result). The early education robot captures visible light images through a camera and transfers them as input data to the CNN model. In the CNN model, the convolutional layer performs a convolution operation on the input image through a convolution kernel to extract low-level features such as the shape and color pattern of the image. The pooling layer reduces the dimension of the feature map output by the convolutional layer, retains important high-level semantic features, and inputs the high-level semantic features output by the pooling layer into the fully connected layer for target category classification. The output layer gives the target type (such as camera, toy, person, etc.), the position of the target in the image, and confidence information (indicating the degree of certainty of the model about the recognition result).

[0105] Step 442: According to the characteristics of the target list in the monitoring environment of the early education robot, construct a random forest model suitable for processing infrared images. The random forest model consists of multiple decision trees. Each tree makes an independent judgment, and finally gives the recognition result through a voting mechanism. Determine the infrared image corresponding to the target from the target list as the training data, and extract features from the infrared image, including calculating the temperature distribution, infrared reflectivity, and underlying texture features of the target in the image. These features can reflect the unique attributes of the target in the infrared band and help distinguish different types of targets. Use the extracted infrared image features to train the random forest model. In the recognition stage, input the new infrared image features into the random forest model. Each decision tree makes an independent judgment based on the features. Finally, the random forest model synthesizes the outputs of all decision trees and gives the target type, position, and confidence information.

[0106] Step 443: The early education robot adopts a suitable strategy (such as weighted average, voting mechanism, etc.) to integrate the preliminary recognition results of visible light images and infrared images. This strategy should consider the complementarity of the two image informations to improve the accuracy and reliability of recognition. Through the integration strategy, the early education robot obtains the integrated preliminary recognition result, which combines the advantages of visible light and infrared images and can more accurately reflect the real situation of the target in the monitoring environment. The integrated result includes the target type, position, and confidence information, providing a basis for the subsequent decision-making and processing of the early education robot.

[0107] Suppose an early education robot is monitoring a children's activity area. The target list includes a camera target, and a visible light image containing the camera is captured. The image is input into a convolutional neural network model, which extracts features such as the shape and color pattern of the camera through convolutional layers and pooling layers. The fully connected layer maps these features to the category of "camera" and outputs the recognition result, including the target type (camera), location (coordinates in the image), and confidence level (e.g., 95%). The early education robot also captures an infrared image containing the camera, extracts features from the infrared image, calculates features such as the temperature distribution and infrared reflectivity of the camera, and inputs these features into a random forest model. Each decision tree makes an independent judgment based on the features and finally outputs the recognition result, which also includes the target type, location, and confidence level. The recognition results of the visible light image and the infrared image are integrated, and through weighted averaging or voting mechanism, the final recognition result is obtained: the target is a camera, the location is accurate, and the confidence level is as high as 98%.

[0108] By combining the information of visible light and infrared images, the early education robot can more accurately identify the targets in the monitoring area, which helps to detect and handle potential safety hazards in a timely manner, such as unauthorized cameras. Visible light images and infrared images have different environmental adaptabilities. In case of insufficient light or occlusion, the infrared image can still provide effective target information. By integrating the information of the two images, the early education robot can maintain stable monitoring performance under various environmental conditions. Accurate recognition and monitoring can provide a safer and more comfortable learning environment for children, and parents and educators can feel more at ease letting their children move freely under the supervision of the early education robot.

[0109] In a preferred embodiment of the present invention, in step 5 above, comparing the preliminary recognition result with the target information in the preset database of the early education robot, and optimizing and adjusting the parameters of the convolutional neural network model and the random forest model using the flower pollination algorithm to obtain the final recognition result and the target location information may include:

[0110] Step 551, comparing the preliminary recognition result with the target information in the preset database of the early education robot to determine the difference between the preliminary recognition result and the target;

[0111] Step 552, according to the difference between the preliminary recognition result and the target, initializing the relevant parameters of the flower pollination algorithm, including the number of flowers, pollination probability, local search radius, and optimizing the parameters of the convolutional neural network model and the random forest model. Among them, each flower performs local search and global pollination according to the evaluation function to obtain the optimized parameters output by the flower pollination algorithm;

[0112] Step 553: Re-identify the targets in the target list according to the optimized parameters output by the flower pollination algorithm to obtain new identification results;

[0113] Step 554: Compare the new identification results with the target information in the preset database of the early education robot again, and calculate the new error;

[0114] Step 555: Repeat the processes of evaluation function design, parameter optimization, re-identification, and error calculation until the preset number of iterations is reached. Take the identification result obtained in the last iteration as the final identification result and target position information.

[0115] In the embodiment of the present invention, the early education robot loads the preset target information database from its internal storage. This database contains information such as the feature descriptions, types, and positions of known targets. Compare the integrated preliminary identification results (including target type, position, and confidence information) obtained in step 443 with the target information in the preset database one by one, and calculate the differences between the preliminary identification results and the targets, such as type matching degree, position deviation, etc.

[0116] Step 552: Set the relevant parameters of the flower pollination algorithm according to the differences between the preliminary identification results and the targets, including the number of flowers (representing different solutions or parameter combinations), pollination probability (controlling the ratio of global search and local search), and local search radius (determining the scope of local search). For each flower, calculate its identification performance under the current parameter combination using the evaluation function. According to the pollination probability, perform global pollination (randomly select the parameters of another flower for crossover) or local search (fine-tune near the current parameter combination), update the parameter combination of the flower, and repeat the above process until the preset number of iterations or convergence condition is reached. Finally, the flower pollination algorithm outputs the optimized parameters of the convolutional neural network model and the random forest model.

[0117] Step 553: Apply the optimized parameters output by the flower pollination algorithm to the convolutional neural network model and the random forest model. Use the optimized models to re-identify the targets in the target list to obtain new identification results (including target type, position, and confidence information).

[0118] Step 554: Compare the new identification results with the target information in the preset database again, and calculate the new identification error according to the comparison results, such as type mis-identification rate, position error, etc.

[0119] Step 555: Repeat the process of Steps 552 to 554. For each iteration, use the optimized parameters obtained in the previous iteration for model training and recognition. After each iteration, update the design of the evaluation function to more accurately reflect the recognition performance of the model. When the preset number of iterations is reached, stop the optimization process, and use the recognition result obtained in the last iteration as the final recognition result and target location information, which is output to the early education robot for subsequent processing or decision-making.

[0120] Suppose the early education robot initially identifies an unknown object (such as a new toy) when monitoring a children's activity area.

[0121] Compare the initial recognition result with the target information in the preset database. Since the object is not in the database, initialize the flower pollination algorithm and set parameters such as the number of flowers, pollination probability, and local search radius. Through the flower pollination algorithm, the early education robot continuously optimizes the parameters of the convolutional neural network model and the random forest model to improve the recognition performance. Using the optimized model, the early education robot re-identifies the unknown object and obtains a more accurate recognition result (such as determining that the object is a new toy). The early education robot compares the new recognition result with the preset database and calculates the recognition error. After multiple iterations of optimization, the early education robot obtains the final recognition result and target location information, which are output for subsequent processing or decision-making.

[0122] By optimizing the model parameters through the flower pollination algorithm, the early education robot can more accurately identify the target, reducing the cases of misrecognition and missed recognition. The global search and local search capabilities of the flower pollination algorithm enable the early education robot to adapt to different environments and target changes, improving the robustness of recognition. The accurate recognition result and target location information provide more reliable data support for the early education robot, helping to enhance the user experience and satisfaction. By continuously optimizing the model parameters and recognition algorithms, the early education robot can better adapt to complex and changing environments, promoting its development towards a more intelligent and autonomous direction.

[0123] In a preferred embodiment of the present invention, the calculation process of the evaluation function is as follows:

[0124] Traverse all target types. For each target type, compare it with the predicted type. If the target type is the same as the predicted type, set the indication value for the corresponding case to 1; otherwise, set it to 0.

[0125] Accumulate all cases where the indication value is 1 to obtain the number of correctly recognized target types, and combine the number of target types with the preset weight coefficient to obtain the weighted value of the type recognition accuracy.

[0126] Traverse all target positions. For each target position, compare it with the predicted position. If the target position is the same as the predicted position, set the indication value for the corresponding case to 1; otherwise, set it to 0.

[0127] Accumulate all cases with an indication value of 1 to obtain the number of correctly recognized target positions, and fuse the number of target positions with a preset weight coefficient to obtain the weighted value of the position recognition accuracy.

[0128] Traverse all recognition results. For each recognition result, calculate the absolute value of the difference between the confidence level and the matching probability.

[0129] Based on the absolute value and the weight of the corresponding recognition result, obtain the total sum of the weighted confidence level errors.

[0130] Based on the total sum of the weighted confidence level errors and the normalization factor, obtain the normalized confidence level error, and fuse the normalized confidence level error with a preset weight coefficient to obtain the weighted value of the confidence level error.

[0131] Based on the weighted value of the type recognition accuracy and the weighted value of the position recognition accuracy, obtain the preliminary evaluation value.

[0132] On the basis of the preliminary evaluation value, adjust the weighted value of the confidence level error to obtain the unnormalized evaluation result.

[0133] Adjust the ratio of the unnormalized evaluation result to the total number of recognized targets for normalization processing, and finally obtain the value of the evaluation function.

[0134] In the embodiment of the present invention, traverse all target types j (j = 1, 2,..., M). For each target type T j , compare it with the predicted type given by the user . If then the indication value Otherwise, Accumulate all cases with an indication value of 1 to obtain the number of correctly recognized target types Combine this number with a preset weight coefficient a to obtain the weighted value of the type recognition accuracy Traverse all target positions k (k = 1, 2,..., L). For each target position P k , compare it with the predicted position given by the user . If then the indication value Otherwise, Accumulate all cases with an indication value of 1 to obtain the number of correctly recognized target positions Fuse this number with a preset weight coefficient b to obtain the weighted value of the position recognition accuracy

[0135] Traverse all recognition results \(i\) (\(i = 1, 2, \ldots, N\)). For each recognition result, calculate the confidence level \(C\) i and the matching probability \(P\) i The absolute value of the difference \(|C\) i - \(P\) i |. According to this absolute value and the weight \(w(i)\) of the corresponding recognition result, obtain the total weighted confidence error Adjust the ratio of this total to the normalization factor \(Z\) to obtain the normalized confidence error. Fuse the normalized confidence error with the preset weight coefficient \(c\) to obtain the weighted value of the confidence error According to the weighted value of the type recognition accuracy rate and the weighted value of the position recognition accuracy rate, obtain the preliminary evaluation value Based on the preliminary evaluation value, adjust (i.e., subtract) the weighted value of the confidence error to obtain the unnormalized evaluation result. Divide the unnormalized evaluation result by the total number of recognition targets \(N\) for normalization processing, and finally obtain the value of the evaluation function

[0136] Through the evaluation function, the early education robot can more comprehensively evaluate its own recognition performance, including the accuracy of type recognition and position recognition, as well as the consistency between the confidence level and the matching probability. The evaluation function takes into account the recognition weights of different types and positions, as well as the influence of the confidence error, making the evaluation result more objective and accurate. Through normalization processing, the evaluation results can be compared and analyzed between different recognition tasks, which helps the early education robot continuously optimize its recognition algorithm and performance. In addition, this evaluation function can also provide feedback and reference for the users of the early education robot, helping them understand the recognition ability and effect of the robot, so as to better utilize the robot for early education activities.

[0137] In a preferred embodiment of the present invention, in step 6 above, according to the final recognition result and the target position information, the early education robot performs corresponding interaction responses, including playing educational content related to the recognition target, having a conversation with the child, and displaying an expression animation related to the recognition target, which may include:

[0138] In the embodiments of the present invention, the recognition result includes the recognized target type (such as animals, plants, items, etc.) and the detailed features of the target (such as color, shape, size, etc.), and the target position information indicates the specific position of the target in the camera's field of view. The robot analyzes the received recognition result and target position information, and extracts key information such as the target type, features, and position coordinates. According to this information, it is possible to determine the educational opportunities existing in the current environment and prepare corresponding interactive content. According to the recognized target type, the early education robot selects educational content related to the recognized target from its built-in educational content library. The robot plays the educational content in the form of voice to children through its built-in audio playback system. For example, if a kitten is recognized, the robot may play educational content about the living habits and interesting stories of the kitten. The early education robot can also have real-time conversation exchanges with children based on the recognition result and target position information. The robot can understand children's questions and give corresponding answers through speech recognition and synthesis technologies. For example, if a child asks "What kind of animal is this?", the robot can answer "This is a kitten!"

[0139] The early education robot can also display expression animations related to the recognized target on its display screen to increase the fun of interaction. According to the recognized target type and features, the robot can select corresponding expressions and actions to display. For example, if a happy kitten is recognized, the robot can display an expression animation of a smiling kitten and be accompanied by lively sound effects. During the process of executing the interactive response, the early education robot continuously monitors the changes in the camera's field of view. If the recognition result or target position information changes, the robot will promptly update the interactive response content to maintain the coherence and fun of the interaction with children.

[0140] As Figure 2 shown, the embodiments of the present invention also provide an interactive control system based on robot control, including:

[0141] A differential processing module, configured to perform differential processing on the visible light image and the infrared image in the early education environment, and extract the suspicious target area in the differential image;

[0142] A feature analysis module, configured to design a feature descriptor for the suspicious target area, and perform feature analysis on the suspicious target area according to the feature descriptor to obtain a feature analysis result;

[0143] A discrimination and clustering module, configured to perform preliminary discrimination on the suspicious target area according to the feature analysis result to obtain a set of discriminated targets, and process the set of discriminated targets through a clustering algorithm to obtain a target list;

[0144] An identification module, configured to construct a convolutional neural network model and a random forest model according to a target list, and respectively extract and identify features of targets in the target list by using visible light images and infrared images to obtain preliminary identification results;

[0145] A comparison and optimization module, configured to compare the preliminary identification results with target information in a preset database of the early education robot, and optimize and adjust the parameters of the convolutional neural network model and the random forest model by using an optimization algorithm to obtain final identification results and target position information;

[0146] An interaction response module, configured to enable the early education robot to execute corresponding interaction responses according to the final identification results and target position information, including playing educational content related to the identified target, having a conversation with a child, and displaying an expression animation related to the identified target.

[0147] It should be noted that this system corresponds to the above method, and all implementation manners in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0148] An embodiment of the present invention further provides a computing device, including: a processor and a memory storing a computer program. When the computer program is run by the processor, the above-mentioned method is executed. All implementation manners in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0149] An embodiment of the present invention further provides a computer-readable storage medium storing instructions. When the instructions are run on a computer, the computer is enabled to execute the above-mentioned method. All implementation manners in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0150] The above are the preferred embodiments of the present invention. It should be pointed out that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. An interactive control method based on robot control, characterized in that The method includes: Performing differential processing on the visible light image and the infrared image in the early education environment, and extracting the suspicious target area in the differential image; Designing a feature descriptor for the suspicious target area, and performing feature analysis on the suspicious target area according to the feature descriptor to obtain a feature analysis result; According to the feature analysis result, performing a preliminary discrimination on the suspicious target area, and processing the discriminated target set through a clustering algorithm to obtain a target list; According to the target list, constructing a convolutional neural network model and a random forest model, and respectively using the visible light image and the infrared image to extract features and identify the targets in the target list to obtain a preliminary identification result; Comparing the preliminary identification result with the target information in the preset database of the early education robot, and using the flower pollination algorithm to optimize and adjust the parameters of the convolutional neural network model and the random forest model to obtain the final identification result and the target position information; According to the final identification result and the target position information, the early education robot performs corresponding interaction responses, including playing educational content related to the identified target, having a conversation with the child, and displaying an expression animation related to the identified target.

2. The interactive control method based on robot control according to claim 1, wherein, Performing differential processing on the visible light image and the infrared image in the early education environment, and extracting the suspicious target area in the differential image, including: Using the camera device built in the early education robot to collect the visible light image and the infrared image in the early education environment; Performing differential processing on the visible light image and the infrared image by calculating the difference in pixel values between the visible light image and the infrared image to obtain a differentially processed image; Analyzing the differentially processed image, including threshold segmentation, edge detection, and morphological processing, to identify the suspicious target area in the early education environment, including dropped toys and unreturned teaching aids.

3. The interactive control method based on robot control according to claim 2, wherein Designing a feature descriptor for the suspicious target area, and performing feature analysis on the suspicious target area according to the feature descriptor to obtain a feature analysis result, including: For the circularity feature, according to the preset threshold, judging whether the shape of the suspicious target area conforms to the typical shape of the objects in the early education environment, including circular toys and spherical teaching aids; for the highlight area area feature, analyzing the area size of the highlight area and its proportion in the suspicious target area to identify objects with reflective characteristics, including metal toys and mirror teaching aids; for the gray level change feature before and after infrared illumination, analyzing the amplitude and trend of the gray level value change to judge whether the suspicious target area includes objects with specific thermal characteristics, including heating electronic devices and heat-sensitive materials; for the local contrast feature, judging the distinguishability of the suspicious target area in the image from the surrounding environment according to the strength of the contrast. Obtaining the feature analysis result according to the analysis of circularity, highlight area area, gray level change before and after infrared illumination, and local contrast.

4. The interactive control method based on robot control according to claim 3, characterized in that, According to the feature analysis result, performing a preliminary discrimination on the suspicious target area, and processing the discriminated target set through a clustering algorithm to obtain a target list, including: Setting a discrimination threshold, when the feature analysis result of the suspicious target area ≥ the discrimination threshold, determining the corresponding feature as a camera target, and adding the corresponding feature to the discriminated target set; Set clustering parameters, including the number of clusters and the distance threshold, according to the feature descriptors of the targets in the target set after discrimination, including position, shape, and size. Divide the target set after discrimination into different clusters, and for each cluster, extract the cluster center. Arrange the cluster centers of all clusters in order to form a target list.

5. The interactive control method based on robot control according to claim 4, wherein According to the target list, construct a convolutional neural network model and a random forest model, and respectively use visible light images and infrared images to extract features and identify the targets in the target list to obtain preliminary recognition results, including: According to the target list, construct a convolutional neural network model, including a convolutional layer, a pooling layer, and a fully connected layer; input the visible light image into the convolutional neural network model, and through the operations of the convolutional layer and the pooling layer, extract the high-level semantic features in the image, including shape, color pattern, and texture, and input the high-level semantic features into the fully connected layer of the convolutional neural network model for classification, and output the preliminary recognition results of the visible light image, including target type, position, and confidence information. According to the target list, construct a random forest model; determine the infrared image corresponding to the target from the target list as the training data, and extract features from the image, including calculating the temperature distribution and the underlying texture features of the infrared reflectivity; input the infrared image features into the random forest model, and each decision tree in the random forest model makes an independent judgment according to the features and outputs the preliminary recognition results of the infrared image, including target type, position, and confidence information. Integrate the preliminary recognition results of the visible light image and the infrared image to obtain the integrated preliminary recognition results.

6. The interactive control method based on robot control according to claim 5, characterized in that, Compare the preliminary recognition results with the target information in the preset database of the early education robot, and use the flower pollination algorithm to optimize and adjust the parameters of the convolutional neural network model and the random forest model to obtain the final recognition results and target position information, including: Compare the preliminary recognition results with the target information in the preset database of the early education robot to determine the difference between the preliminary recognition results and the target. According to the difference between the preliminary recognition results and the target, initialize the relevant parameters of the flower pollination algorithm, including the number of flowers, the pollination probability, and the local search radius, and optimize the parameters of the convolutional neural network model and the random forest model. Among them, each flower performs local search and global pollination according to the evaluation function to obtain the optimized parameters output by the flower pollination algorithm. According to the optimized parameters output by the flower pollination algorithm, re-identify the targets in the target list to obtain new recognition results. Perform a comparison operation again between the new recognition results and the target information in the preset database of the early education robot to calculate the new error. Repeat the process of evaluation function design, parameter optimization, re-identification, and error calculation until the preset number of iterations is reached, and use the recognition results obtained in the last iteration as the final recognition results and target position information.

7. The interactive control method based on robot control according to claim 6, wherein, The calculation process of the evaluation function is as follows: Traverse all target types. For each target type, compare it with the predicted type. If the target type is the same as the predicted type, set the indication value of the corresponding situation to 1, otherwise set it to 0. Accumulate all cases where the indicated value is 1 to obtain the number of correctly recognized target types, and combine the number of target types with a preset weight coefficient to obtain a weighted value of the type recognition accuracy rate; Traverse all target positions. For each target position, compare it with the predicted position. If the target position is the same as the predicted position, set the indicated value of the corresponding case to 1, otherwise set it to 0; Accumulate all cases where the indicated value is 1 to obtain the number of correctly recognized target positions, and fuse the number of target positions with a preset weight coefficient to obtain a weighted value of the position recognition accuracy rate; Traverse all recognition results. For each recognition result, calculate the absolute value of the difference between the confidence level and the matching probability; According to the absolute value and the weight of the corresponding recognition result, obtain the total sum of the weighted confidence level errors; According to the total sum of the weighted confidence level errors and the normalization factor, obtain the normalized confidence level error, and fuse the normalized confidence level error with a preset weight coefficient to obtain a weighted value of the confidence level error; According to the weighted value of the type recognition accuracy rate and the weighted value of the position recognition accuracy rate, obtain a preliminary evaluation value; Based on the preliminary evaluation value, adjust the weighted value of the confidence level error to obtain an unnormalized evaluation result; Adjust the ratio of the unnormalized evaluation result to the total number of recognized targets for normalization processing, and finally obtain the value of the evaluation function.

8. An interactive control system based on robot control, which implements the method according to any one of claims 1 to 7, characterized in that including: A differential processing module for performing differential processing on visible light images and infrared images in the early education environment to extract suspicious target areas in the differential images; A feature analysis module for designing feature descriptors for the suspicious target areas and performing feature analysis on the suspicious target areas according to the feature descriptors to obtain feature analysis results; A discrimination and clustering module for preliminarily discriminating the suspicious target areas according to the feature analysis results to obtain a set of discriminated targets, and processing the set of discriminated targets through a clustering algorithm to obtain a target list; An identification module for constructing a convolutional neural network model and a random forest model according to the target list, and respectively using visible light images and infrared images to extract features and identify the targets in the target list to obtain preliminary identification results; A comparison and optimization module for comparing the preliminary identification results with the target information in the preset database of the early education robot, and using an optimization algorithm to optimize and adjust the parameters of the convolutional neural network model and the random forest model to obtain the final identification results and target position information; An interaction response module for the early education robot to perform corresponding interaction responses according to the final identification results and target position information, including playing educational content related to the identified target, having a conversation with the child, and displaying an expression animation related to the identified target.

9. A computing device, characterized in that, including: One or more processors; A storage device for storing one or more programs, which when executed by the one or more processors, cause the one or more processors to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A program is stored in the computer-readable storage medium, and when the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images