Motion control method and system of interactive teaching robot

Abstract

The present invention provides a motion control method and system for an interactive teaching robot, which collects sound information around the environment in which the teaching robot is located, controls the teaching robot to switch working states and wake it up according to voiceprint feature information and semantic information in the sound information, and then determines the location of the user according to the sound intensity distribution information in the sound information so as to capture an environmental image; then, identifies obstacles in the environmental image to determine the moving path of the teaching robot, and instructs the teaching robot to play corresponding voice reply messages during the movement, so that it can be ensured that the teaching robot can follow the user in a timely and accurate manner under the user's voice control, and quickly interact with the user by voice, thereby improving the movement flexibility of the teaching robot in the process of interaction with students and improving the intelligent and automated degree of interaction between the teaching robot and students.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent teaching robot control, and in particular to a motion control method and system of an interactive teaching robot. Background Art

[0002] The teaching robot can interact with students by voice, and can also teach different types of knowledge during the voice interaction, so that teaching and entertainment interaction can be organically combined. Existing teaching robots usually have a single voice interaction function, and they cannot effectively interact with students in terms of movement, that is, the teaching robot cannot move to the vicinity of the student's position according to the student's voice command, and the teaching robot cannot effectively and accurately avoid obstacles during movement, which does not improve the movement flexibility of the teaching robot during the interaction with students, and cannot improve the intelligence and automation of the interaction between the teaching robot and students. Summary of the invention

[0003] In view of the defects existing in the prior art, the present invention provides a motion control method and system for an interactive teaching robot, which collects sound information around the environment where the teaching robot is located; after analyzing the sound information, obtains voiceprint feature information, semantic information and sound intensity distribution information corresponding to the sound information; then controls the teaching robot to switch its working state according to the voiceprint feature information and the semantic information; determines the relative position of the user and the teaching robot in the environment where the teaching robot is located according to the sound intensity distribution information; then, based on the relative position, shoots the position area where the user is located, so as to obtain a corresponding environmental image; finally, analyzes the environmental image to determine the existence status of obstacles in the space area between the teaching robot and the user; determines the moving path of the teaching robot to the user's position according to the existence status of the obstacles; and instructs the teaching robot to respond to the language in the process of the teaching robot moving along the moving path. The motion control method and system of the interactive teaching robot collects sound information around the teaching robot, controls the teaching robot to switch working states and wake it up according to the voiceprint feature information and semantic information in the sound information, and then determines the user's location area according to the sound intensity distribution information in the sound information to facilitate the capture of environmental images; then, the obstacles in the environmental image are identified to determine the moving path of the teaching robot, and the teaching robot is instructed to play the corresponding voice reply message during the movement, so as to ensure that the teaching robot can follow the user in a timely and accurate manner under the user's voice control, and quickly interact with the user by voice, thereby improving the movement flexibility of the teaching robot in the process of interaction with students and improving the intelligence and automation of the interaction between the teaching robot and students.

[0004] The present invention provides a motion control method for an interactive teaching robot, characterized in that it comprises the following steps:

[0005] Step S1, collecting sound information around the environment where the teaching robot is located; analyzing the sound information to obtain voiceprint feature information, semantic information and sound intensity distribution information corresponding to the sound information; and then controlling the teaching robot to switch working states according to the voiceprint feature information and the semantic information;

[0006] Step S2, determining the relative position of the user and the teaching robot in the environment where the teaching robot is located according to the sound intensity distribution information; and then photographing the position area where the user is located according to the relative position, so as to obtain a corresponding environment image;

[0007] Step S3, analyzing the environmental image to determine the existence status of obstacles in the space area between the teaching robot and the user; determining a moving path for the teaching robot to move to the user's location according to the existence status of the obstacles; and instructing the teaching robot to feedback a corresponding voice reply message to the user in response to the semantic information during the movement of the teaching robot along the moving path;

[0008] Further, in the step S1, the teaching robot is instructed to collect sound information around the environment in which it is located; after analyzing the sound information, voiceprint feature information, semantic information and sound intensity distribution information corresponding to the sound information are obtained; and then according to the voiceprint feature information and the semantic information, the teaching robot is controlled to switch the working state; specifically including:

[0009] Step S101, collecting sound information around the environment where the teaching robot is located; analyzing the sound information to obtain voiceprint feature information, semantic information and sound intensity distribution information corresponding to the sound information; and then controlling the teaching robot to switch working states according to the voiceprint feature information and the semantic information;

[0010] Step S102, performing voiceprint recognition on the sound information, thereby extracting voiceprint feature information contained in the sound information; performing semantic recognition on the sound information, thereby extracting semantic information corresponding to the sound information; performing sound intensity waveform analysis on the sound information, thereby extracting sound intensity peak distribution contained in the sound information;

[0011] Step S103, comparing the voiceprint feature information with the actual voiceprint feature of the user stored in advance; if the two are different, controlling the teaching robot to remain in a dormant state; if the two are the same, comparing the semantic information with the pre-stored semantic keywords; if the semantic information does not contain the semantic keywords, controlling the teaching robot to remain in a dormant state; if the semantic information contains the semantic keywords, controlling the teaching robot to switch from the dormant state to a normal working state;

[0012] Further, in step S2, the relative position of the user and the teaching robot in the environment where the teaching robot is located is determined according to the sound intensity distribution information; and then the position area where the user is located is photographed according to the relative position, so as to obtain the corresponding environment image, which specifically includes:

[0013] Step S201, analyzing the sound intensity distribution information to obtain a corresponding sound intensity waveform distribution; extracting a sound intensity peak with a maximum amplitude from the sound intensity waveform distribution, and determining when the sound intensity peak with a maximum amplitude appears, scanning the sound information of the environment where the teaching robot is located to collect the corresponding spatial orientation, so as to use the spatial orientation as the relative orientation of the user and the teaching robot in the environment where the teaching robot is located;

[0014] Step S202, expanding the spatial orientation by a preset multiple of the spatial orientation angle as the orientation area where the user is located, scanning and photographing the orientation area where the user is located, thereby obtaining a corresponding environmental image;

[0015] Further, in step S3, analyzing the environmental image to determine the existence state of obstacles in the space area between the teaching robot and the user; determining the movement path of the teaching robot to the user's location according to the existence state of the obstacles; and instructing the teaching robot to feedback a corresponding voice reply message to the user according to the semantic information during the movement of the teaching robot along the movement path. Specifically, the method includes:

[0016] Step S301, converting the environment image into a grayscale environment image, and extracting corresponding image pixel contour information from the grayscale environment image; and then determining the position and size of obstacles existing in the space area between the teaching robot and the user according to the image pixel contour information;

[0017] Step S302, determining a continuous moving path that can accommodate the movement of the teaching robot in the space area between the teaching robot and the user according to the position and size of the obstacle, so that the teaching robot can move to the position of the user along the continuous moving path;

[0018] Step S303, selecting a voice reply message that matches the semantic information from a preset response voice database; and repeatedly playing the voice reply message with a variable volume while the teaching robot moves along the continuous moving path, wherein the variable volume means that the closer the teaching robot is to the user during movement, the smaller the corresponding volume.

[0019] Further, in the step S301, determining the position and size of obstacles existing in the space area between the teaching robot and the user according to the image pixel profile information specifically includes:

[0020] Step S3011, based on the image pixel contour information, analyze whether all pixel points in the space area between the teaching robot and the user are points inside the image pixel contour, and then determine whether they are pixel points belonging to obstacles, and mark all pixel points of obstacles in the space area between the teaching robot and the user, so as to determine the position of the obstacles, specifically,

[0021] Using the following formula (1), according to the image pixel profile information, determine the location of obstacles in the space between the teaching robot and the user:

[0022]

[0023] In the above formula (1), A a Indicates the judgment value of whether the ath pixel point in the space area between the teaching robot and the user belongs to the pixel point of the obstacle. a = 1, it means that the ath pixel point belongs to the pixel point of the obstacle and the ath pixel point is marked to determine the obstacle position. a = 0, it means that the a-th pixel does not belong to the obstacle pixel and the a-th pixel is not marked. a ,Y a ) represents the coordinate value of the ath pixel point in the space between the teaching robot and the user, (X i ,Y i ) represents the coordinate value of the i-th contour point in the image pixel contour, X imin Indicates the minimum horizontal coordinate value of all contour points in the image pixel contour, X imax Indicates the maximum horizontal coordinate value of all contour points in the image pixel contour, Y imin Indicates the minimum ordinate value of all contour points in the image pixel contour, Y imax It represents the maximum ordinate value of all contour points in the image pixel contour, and n represents the total number of contour points contained in the image pixel contour;

[0024] Step S3012, using the following formula (2), according to the image pixel profile information, determine the size of the obstacle in the space between the teaching robot and the user,

[0025]

[0026] In the above formula (2), S p represents the length of the pth row of pixels in the horizontal direction of the obstacle image corresponding to the spatial area between the teaching robot and the user in the real space, k represents the total number of pixels in each row in the horizontal direction corresponding to the spatial area between the teaching robot and the user, and L 0 Indicates the length of each row of pixels in the real space in the horizontal direction corresponding to the spatial area between the teaching robot and the user;

[0027] In step S302, according to the position and size of the obstacle, determining a continuous moving path that can accommodate the movement of the teaching robot in the space area between the teaching robot and the user specifically includes:

[0028] Using the following formula (3), according to the position and size of the obstacle, the space area between the teaching robot and the user is marked in the row and column directions, so that a number of pixel points are marked in the space area.

[0029]

[0030] In the above formula (3), A pq represents the pixel point annotation value obtained after annotating the space area between the teaching robot and the user in the direction of the pth row and the qth column, and L represents the length of the teaching robot in the horizontal direction;

[0031] All pixel points whose pixel mark value is 1 are determined, and a continuous moving path is found in the space corresponding to the pixel points, so as to serve as a continuous moving path to accommodate the movement of the teaching robot.

[0032] The present invention also provides a motion control system for an interactive teaching robot, which is characterized in that it includes a sound collection and analysis module, a working state switching module, an environmental image shooting module, a moving path determination module and a voice reply message playing module; wherein,

[0033] The sound collection and analysis module is used to collect sound information around the environment where the teaching robot is located; after analyzing the sound information, the voiceprint feature information, semantic information and sound intensity distribution information corresponding to the sound information are obtained;

[0034] The working state switching module is used to control the teaching robot to switch the working state according to the voiceprint feature information and the semantic information;

[0035] The environmental image shooting module is used to determine the relative position of the user and the teaching robot in the environment where the teaching robot is located according to the sound intensity distribution information; and then shoot the position area where the user is located according to the relative position, so as to obtain the corresponding environmental image;

[0036] The moving path determination module is used to analyze the environmental image to determine the existence status of obstacles in the space area between the teaching robot and the user; according to the existence status of the obstacles, determine the moving path of the teaching robot to the user's location;

[0037] The voice reply message playing module is used to instruct the teaching robot to feed back a corresponding voice reply message to the user according to the semantic information during the movement of the teaching robot along the moving path;

[0038] Furthermore, the sound collection and analysis module is used to collect sound information around the environment where the teaching robot is located; after analyzing the sound information, the voiceprint feature information, semantic information and sound intensity distribution information corresponding to the sound information are obtained, which specifically include:

[0039] Using a microphone array to scan and collect sound information from all directions of the environment in which the teaching robot is located; and performing Kalman filtering on the sound information to remove background noise contained in the sound information;

[0040] Performing voiceprint recognition on the sound information to extract voiceprint feature information contained in the sound information; performing semantic recognition on the sound information to extract semantic information corresponding to the sound information; performing sound intensity waveform analysis on the sound information to extract sound intensity peak distribution contained in the sound information;

[0041] as well as,

[0042] The working state switching module is used to control the teaching robot to switch the working state according to the voiceprint feature information and the semantic information, and specifically includes:

[0043] The voiceprint feature information is compared with the actual voiceprint feature of the user stored in advance; if the two are different, the teaching robot is controlled to remain in a dormant state; if the two are the same, the semantic information is compared with the pre-stored semantic keywords; if the semantic information does not contain the semantic keywords, the teaching robot is controlled to remain in a dormant state; if the semantic information contains the semantic keywords, the teaching robot is controlled to switch from the dormant state to a normal working state;

[0044] Further, the environmental image shooting module is used to determine the relative position of the user and the teaching robot in the environment where the teaching robot is located according to the sound intensity distribution information; and then shoot the position area where the user is located according to the relative position, so as to obtain the corresponding environmental image. Specifically, it includes:

[0045] Analyze the sound intensity distribution information to obtain a corresponding sound intensity waveform distribution; extract a sound intensity peak with a maximum amplitude from the sound intensity waveform distribution, and determine the corresponding spatial orientation when the sound intensity peak with the maximum amplitude appears, and scan the sound information of the environment where the teaching robot is located to collect the sound information, so as to use the spatial orientation as the relative orientation of the user and the teaching robot in the environment where the teaching robot is located;

[0046] The spatial orientation is expanded by a preset multiple of the spatial orientation angle as the orientation area where the user is located, and the orientation area where the user is located is scanned and photographed to obtain a corresponding environmental image;

[0047] Further, the movement path determination module is used to analyze the environmental image to determine the existence state of obstacles in the space area between the teaching robot and the user; according to the existence state of the obstacles, determining the movement path of the teaching robot to the user's position specifically includes:

[0048] Converting the environment image into a grayscale environment image, and extracting corresponding image pixel contour information from the grayscale environment image; and determining the position and size of obstacles existing in the space area between the teaching robot and the user according to the image pixel contour information;

[0049] According to the position and size of the obstacle, a continuous moving path capable of accommodating the movement of the teaching robot is determined in the space between the teaching robot and the user, so that the teaching robot can move along the continuous moving path to the position of the user;

[0050] as well as,

[0051] The voice reply message playing module is used to instruct the teaching robot to feed back a corresponding voice reply message to the user according to the semantic information during the movement of the teaching robot along the moving path, and specifically includes:

[0052] A voice reply message matching the semantic information is selected from a preset response voice database; and the voice reply message is repeatedly played with a variable volume while the teaching robot moves along the continuous moving path, wherein the variable volume means that the closer the teaching robot is to the user during the movement, the smaller the corresponding volume.

[0053] Compared with the prior art, the motion control method and system of the interactive teaching robot collects sound information around the teaching robot's environment; after analyzing the sound information, obtains voiceprint feature information, semantic information and sound intensity distribution information corresponding to the sound information; then controls the teaching robot to switch its working state according to the voiceprint feature information and semantic information; determines the relative position of the user and the teaching robot in the teaching robot's environment according to the sound intensity distribution information; then, based on the relative position, shoots the user's location area to obtain the corresponding environmental image; finally, analyzes the environmental image to determine the existence status of obstacles in the space area between the teaching robot and the user; determines the moving path of the teaching robot to the user's location according to the existence status of the obstacles; and instructs the teaching robot to move to the user's location according to the semantic information during the movement of the teaching robot along the moving path. The user feeds back a corresponding voice reply message; it can be seen that the motion control method and system of the interactive teaching robot collects sound information around the teaching robot, controls the teaching robot to switch working states and realize wake-up according to the voiceprint feature information and semantic information in the sound information, and then determines the user's location area according to the sound intensity distribution information in the sound information to facilitate the capture of the environmental image; then, identifies the existing obstacles in the environmental image to determine the moving path of the teaching robot, and instructs the teaching robot to play the corresponding voice reply message during the movement, which can ensure that the teaching robot can follow the user in a timely and accurate manner under the user's voice control, and quickly interact with the user by voice, thereby improving the movement flexibility of the teaching robot in the interaction process with students and improving the intelligence and automation of the interaction between the teaching robot and students.

[0054] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0055] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1 This is a structural schematic diagram of the motion control method and system of the interactive teaching robot provided by the present invention.

[0058] Figure 2 This is a structural schematic diagram of the motion control method and system of the interactive teaching robot provided by the present invention. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] See also Figure 1 , is a flow chart of a motion control method for an interactive teaching robot provided by an embodiment of the present invention. The motion control method for an interactive teaching robot comprises the following steps:

[0061] Step S1, collecting sound information around the environment where the teaching robot is located; analyzing the sound information to obtain voiceprint feature information, semantic information and sound intensity distribution information corresponding to the sound information; and then controlling the teaching robot to switch working states according to the voiceprint feature information and the semantic information;

[0062] Step S2, determining the relative position of the user and the teaching robot in the environment where the teaching robot is located according to the sound intensity distribution information; and then photographing the position area where the user is located according to the relative position, so as to obtain a corresponding environment image;

[0063] Step S3, analyzing the environmental image to determine the existence status of obstacles in the space area between the teaching robot and the user; determining the moving path of the teaching robot to the user's position based on the existence status of the obstacles; and instructing the teaching robot to feedback a corresponding voice reply message to the user based on the semantic information during the movement of the teaching robot along the moving path.

[0064] The beneficial effects of the above technical solution are as follows: the motion control method of the interactive teaching robot collects sound information around the environment in which the teaching robot is located, controls the teaching robot to switch working states and wake up according to the voiceprint feature information and semantic information in the sound information, and then determines the user's location area according to the sound intensity distribution information in the sound information to facilitate the capture of environmental images; then, identifies existing obstacles in the environmental image to determine the moving path of the teaching robot, and instructs the teaching robot to play corresponding voice reply messages during the movement, so as to ensure that the teaching robot can follow the user in a timely and accurate manner under the user's voice control, and quickly interact with the user by voice, thereby improving the movement flexibility of the teaching robot during the interaction with students and improving the intelligence and automation of the interaction between the teaching robot and students.

[0065] Preferably, in step S1, collecting sound information around the environment where the teaching robot is located; analyzing the sound information to obtain voiceprint feature information, semantic information and sound intensity distribution information corresponding to the sound information; and controlling the teaching robot to switch working states according to the voiceprint feature information and the semantic information specifically includes:

[0066] Step S101, using a microphone array to scan and collect sound information from all directions of the environment in which the teaching robot is located; and performing Kalman filtering on the sound information to remove background noise contained in the sound information;

[0067] Step S102, performing voiceprint recognition on the sound information to extract voiceprint feature information contained in the sound information; performing semantic recognition on the sound information to extract semantic information corresponding to the sound information; performing sound intensity waveform analysis on the sound information to extract sound intensity peak distribution contained in the sound information;

[0068] Step S103, compare the voiceprint feature information with the actual voiceprint feature of the user stored in advance; if the two are different, control the teaching robot to remain in a dormant state; if the two are the same, compare the semantic information with the pre-stored semantic keywords; if the semantic information does not contain the semantic keywords, control the teaching robot to remain in a dormant state; if the semantic information contains the semantic keywords, control the teaching robot to switch from the dormant state to the normal working state.

[0069] The beneficial effect of the above technical solution is that when the teaching robot is working, the student may be in any direction around the teaching robot (such as the south or northeast direction). When the student needs to call the teaching robot to move closer to him / her, he / she needs to send a corresponding voice sound to the teaching robot. At this time, the microphone array is used to scan and collect sound information from all directions of the environment, so that the voice sound from the student can be collected comprehensively and accurately.

[0070] Since the voiceprint feature information of the student is specific and inherent, the voiceprint feature information in the sound information is extracted and compared with the real-time voiceprint feature of the pre-stored authorized user. When the two are the same, it indicates that the corresponding student is a user authorized to use the teaching robot. At this time, the semantic information in the sound information is compared with the pre-stored semantic keywords to determine whether the voice sound emitted by the user is the voice sound for summoning the teaching robot. For example, the semantic keyword can be but is not limited to "come here". When the semantic information contains the keyword "come here", the teaching robot is switched from the dormant state to the normal working state, which can ensure that the teaching robot can enter different working modes at any time.

[0071] Preferably, in step S2, the relative position of the user and the teaching robot in the environment where the teaching robot is located is determined according to the sound intensity distribution information; and then the position area where the user is located is photographed according to the relative position, so as to obtain the corresponding environment image, which specifically includes:

[0072] Step S201, analyzing the sound intensity distribution information to obtain a corresponding sound intensity waveform distribution; extracting a sound intensity peak with a maximum amplitude from the sound intensity waveform distribution, and determining when the sound intensity peak with a maximum amplitude appears, scanning the sound information of the environment where the teaching robot is located to collect the corresponding spatial orientation, so as to use the spatial orientation as the relative orientation of the user and the teaching robot in the environment where the teaching robot is located;

[0073] Step S202, the spatial orientation is expanded by a preset multiple of the spatial orientation angle as the orientation area where the user is located, and the orientation area where the user is located is scanned and photographed to obtain a corresponding environmental image.

[0074] The beneficial effect of the above technical solution is: when the user sends a voice sound to the teaching robot from a certain position (such as the south), in the corresponding scanning and collecting sound information process, when the position (such as the south) where the user is located is scanned, the sound intensity peak corresponding to the collected sound information has the maximum amplitude. Based on the above process, the sound intensity peak with the maximum amplitude is extracted from the sound intensity waveform distribution, and the spatial orientation corresponding to the sound information scanning and collecting of the environment where the teaching robot itself is located when the sound intensity peak with the maximum amplitude appears is determined, so that the relative orientation of the user and the teaching robot can be accurately determined. In addition, in order to ensure wide-angle shooting of the user's location area, the spatial orientation angle corresponding to the spatial orientation can be doubled or tripled in the up, down, left and right directions as the user's location area, and then the orientation area is scanned and photographed, so that the obstacles around the user can be photographed and recorded to the maximum extent.

[0075] Preferably, in step S3, analyzing the environmental image to determine the presence of obstacles in the space between the teaching robot and the user; determining a moving path for the teaching robot to move to the user's location according to the obstacle presence; and instructing the teaching robot to feedback a corresponding voice reply message to the user in response to the semantic information during the movement of the teaching robot along the moving path. Specifically, the method includes:

[0076] Step S301, converting the environment image into a grayscale environment image, and extracting corresponding image pixel contour information from the grayscale environment image; and then determining the position and size of obstacles existing in the space area between the teaching robot and the user according to the image pixel contour information;

[0077] Step S302, determining a continuous moving path that can accommodate the movement of the teaching robot in the space between the teaching robot and the user according to the position and size of the obstacle, so that the teaching robot can move to the position of the user along the continuous moving path;

[0078] Step S303, select a voice reply message that matches the semantic information from a preset response voice database; and repeatedly play the voice reply message with a variable volume while the teaching robot moves along the continuous moving path, wherein the variable volume means that the closer the teaching robot is to the user during the movement, the smaller the corresponding volume.

[0079] The beneficial effect of the above technical solution is: since the image pixel contour corresponding to the obstacle in the environmental image will be different from the image pixel contour of other areas in the contour line direction and contour line density (the contour line of the image pixel contour corresponding to the obstacle is usually curved and the contour line density is also large, while the contour line of the image pixel contour of other areas is usually straight and the contour line density is also small), the above difference can accurately determine the obstacle position and obstacle size in the space area between the teaching robot and the user. When the obstacle position and obstacle size are determined, the gap area between different obstacles can be determined, and then these gap areas are spliced ​​to determine the continuous moving path that can accommodate the movement of the teaching robot, so that the teaching robot can move smoothly and quickly to the user's position without colliding with the obstacle. In addition, a voice reply message that matches the semantic information is selected from the preset answer voice database, which can improve the interactive flexibility between the teaching robot and the user (for example, the voice information can be "come here", and the corresponding voice reply message can be "OK"). And when the teaching robot moves along the continuous moving path, the voice reply message is repeatedly played with varying volume, so that when the teaching robot is far away from the user, the voice reply message can be clearly heard, and when the teaching robot is close to the user, the voice reply message can be prevented from being too loud and damaging the user's hearing.

[0080] Preferably, in step S301, determining the position and size of obstacles existing in the space area between the teaching robot and the user according to the image pixel profile information specifically includes:

[0081] Step S3011, based on the image pixel contour information, analyze whether all pixel points in the space area between the teaching robot and the user are points inside the image pixel contour, and then determine whether they are pixel points belonging to obstacles, and mark all pixel points of obstacles in the space area between the teaching robot and the user, so as to determine the position of the obstacles, specifically,

[0082] Using the following formula (1), according to the pixel contour information of the image, the location of obstacles in the space between the teaching robot and the user is determined.

[0083]

[0084] In the above formula (1), A a Indicates the judgment value of whether the ath pixel point in the space area between the teaching robot and the user belongs to the pixel point of the obstacle. a = 1, it means that the ath pixel point belongs to the pixel point of the obstacle and the ath pixel point is marked to determine the obstacle position. a= 0, it means that the a-th pixel does not belong to the obstacle pixel and the a-th pixel is not marked. a ,Y a ) represents the coordinate value of the ath pixel point in the space between the teaching robot and the user, (X i ,Y i ) represents the coordinate value of the i-th contour point in the image pixel contour, X imin Indicates the minimum horizontal coordinate value of all contour points in the image pixel contour, X imax Indicates the maximum horizontal coordinate value of all contour points in the image pixel contour, Y imin Indicates the minimum ordinate value of all contour points in the image pixel contour, Y imax It represents the maximum ordinate value of all contour points in the image pixel contour, and n represents the total number of contour points contained in the image pixel contour;

[0085] Step S3012, using the following formula (2), according to the image pixel profile information, determine the size of the obstacle in the space between the teaching robot and the user,

[0086]

[0087] In the above formula (2), S p represents the length of the pth row of pixels in the horizontal direction of the obstacle image corresponding to the spatial area between the teaching robot and the user in the real space, k represents the total number of pixels in each row in the horizontal direction corresponding to the spatial area between the teaching robot and the user, and L 0 Indicates the length of each row of pixels in the real space in the horizontal direction corresponding to the spatial area between the teaching robot and the user;

[0088] In step S302, according to the position and size of the obstacle, determining a continuous moving path that can accommodate the movement of the teaching robot in the space area between the teaching robot and the user specifically includes:

[0089] Using the following formula (3), according to the position and size of the obstacle, the space area between the teaching robot and the user is marked in the row and column directions, so that a number of pixel points are marked in the space area.

[0090]

[0091] In the above formula (3), A pq represents the pixel point annotation value obtained after annotating the space area between the teaching robot and the user in the direction of the pth row and the qth column, and L represents the length of the teaching robot in the horizontal direction;

[0092] Determine all the pixels whose pixel annotation value is 1, and find a continuous moving path in the space corresponding to the pixel, so as to serve as a continuous moving path to accommodate the movement of the teaching robot.

[0093] The beneficial effect of the above technical solution is as follows: since the image pixel contour information contains the pixel contour information of the obstacle, the position and size of the obstacle can be quickly and accurately determined by identifying the contour points of the pixel contour information of the obstacle. By using the above formula (1), all the pixel points in the space area between the teaching robot and the user are identified and checked one by one, so as to determine all the pixel points belonging to the obstacle, so that the spatial position occupied by the obstacle can be accurately determined. Secondly, by using the above formula (2), all the pixel points belonging to the obstacle can be used as a reference, and the size of the teaching robot in the horizontal direction can be determined by combining the image and the real space conversion method, because the size of the teaching robot in the horizontal direction during the movement directly determines whether it will collide with the obstacle. Finally, by using the above formula (3), the pixel point area that allows the teaching robot to pass can be marked in the space area, and the continuous pixel point area that allows the teaching robot to pass can be found as a continuous moving path, which can ensure that the teaching robot moves smoothly and steadily to the vicinity of the user.

[0094] See also Figure 2 , is a structural diagram of the motion control system of the interactive teaching robot provided by an embodiment of the present invention. The motion control system of the interactive teaching robot includes a sound collection and analysis module, a working state switching module, an environmental image shooting module, a moving path determination module and a voice reply message playing module; wherein,

[0095] The sound collection and analysis module is used to collect the sound information around the teaching robot; after analyzing the sound information, the voiceprint feature information, semantic information and sound intensity distribution information corresponding to the sound information are obtained;

[0096] The working state switching module is used to control the teaching robot to switch the working state according to the voiceprint feature information and the semantic information;

[0097] The environment image shooting module is used to determine the relative position of the user and the teaching robot in the environment where the teaching robot is located according to the sound intensity distribution information; and then shoot the position area where the user is located according to the relative position, so as to obtain the corresponding environment image;

[0098] The moving path determination module is used to analyze the environmental image to determine the existence state of obstacles in the space area between the teaching robot and the user; according to the existence state of the obstacles, determine the moving path of the teaching robot to the user's location;

[0099] The voice reply message playing module is used to instruct the teaching robot to feed back a corresponding voice reply message to the user according to the semantic information when the teaching robot moves along the moving path.

[0100] The beneficial effects of the above technical solution are as follows: the motion control system of the interactive teaching robot collects sound information around the teaching robot, controls the teaching robot to switch working states and wake up according to the voiceprint feature information and semantic information in the sound information, and then determines the user's location area according to the sound intensity distribution information in the sound information to facilitate the capture of environmental images; then, identifies existing obstacles in the environmental image to determine the moving path of the teaching robot, and instructs the teaching robot to play corresponding voice reply messages during the movement, so as to ensure that the teaching robot can follow the user in a timely and accurate manner under the user's voice control, and quickly interact with the user by voice, thereby improving the movement flexibility of the teaching robot during the interaction with students and improving the intelligence and automation of the interaction between the teaching robot and students.

[0101] Preferably, the sound collection and analysis module is used to collect sound information around the environment where the teaching robot is located; after analyzing the sound information, the voiceprint feature information, semantic information and sound intensity distribution information corresponding to the sound information are obtained, which specifically include:

[0102] The microphone array is used to scan and collect the sound information in the surrounding directions of the teaching robot; and the sound information is processed by Kalman filtering to eliminate the background noise contained in the sound information;

[0103] Performing voiceprint recognition on the sound information to extract voiceprint feature information contained in the sound information; performing semantic recognition on the sound information to extract semantic information corresponding to the sound information; performing sound intensity waveform analysis on the sound information to extract sound intensity peak distribution contained in the sound information;

[0104] as well as,

[0105] The working state switching module is used to control the teaching robot to switch the working state according to the voiceprint feature information and the semantic information, and specifically includes:

[0106] The voiceprint feature information is compared with the actual voiceprint feature of the user stored in advance; if the two are different, the teaching robot is controlled to remain in a dormant state; if the two are the same, the semantic information is compared with the pre-stored semantic keywords; if the semantic information does not contain the semantic keywords, the teaching robot is controlled to remain in a dormant state; if the semantic information contains the semantic keywords, the teaching robot is controlled to switch from the dormant state to a normal working state.

[0107] The beneficial effect of the above technical solution is that when the teaching robot is working, the student may be in any direction around the teaching robot (such as the south or northeast direction). When the student needs to call the teaching robot to move closer to him / her, he / she needs to send a corresponding voice sound to the teaching robot. At this time, the microphone array is used to scan and collect sound information from all directions of the environment, so that the voice sound from the student can be collected comprehensively and accurately.

[0108] Since the voiceprint feature information of the student is specific and inherent, the voiceprint feature information in the sound information is extracted and compared with the real-time voiceprint feature of the pre-stored authorized user. When the two are the same, it indicates that the corresponding student is a user authorized to use the teaching robot. At this time, the semantic information in the sound information is compared with the pre-stored semantic keywords to determine whether the voice sound emitted by the user is the voice sound for summoning the teaching robot. For example, the semantic keyword can be but is not limited to "come here". When the semantic information contains the keyword "come here", the teaching robot is switched from the dormant state to the normal working state, which can ensure that the teaching robot can enter different working modes at any time.

[0109] Preferably, the environment image shooting module is used to determine the relative position of the user and the teaching robot in the environment where the teaching robot is located according to the sound intensity distribution information; and then shoot the position area where the user is located according to the relative position, so as to obtain the corresponding environment image. Specifically, it includes:

[0110] Analyze the sound intensity distribution information to obtain a corresponding sound intensity waveform distribution; extract a sound intensity peak with a maximum amplitude from the sound intensity waveform distribution, and determine the corresponding spatial orientation when the sound intensity peak with a maximum amplitude appears and scan the sound information of the environment where the teaching robot is located to collect the sound information, so as to use the spatial orientation as the relative orientation of the user and the teaching robot in the environment where the teaching robot is located;

[0111] The spatial orientation is expanded by a preset multiple of the spatial orientation angle as the orientation area where the user is located, and the orientation area where the user is located is scanned and photographed to obtain a corresponding environmental image.

[0112] The beneficial effect of the above technical solution is: when the user sends a voice sound to the teaching robot from a certain position (such as the south), in the corresponding scanning and collecting sound information process, when the position (such as the south) where the user is located is scanned, the sound intensity peak corresponding to the collected sound information has the maximum amplitude. Based on the above process, the sound intensity peak with the maximum amplitude is extracted from the sound intensity waveform distribution, and the spatial orientation corresponding to the sound information scanning and collecting of the environment where the teaching robot itself is located when the sound intensity peak with the maximum amplitude appears is determined, so that the relative orientation of the user and the teaching robot can be accurately determined. In addition, in order to ensure wide-angle shooting of the user's location area, the spatial orientation angle corresponding to the spatial orientation can be doubled or tripled in the up, down, left and right directions as the user's location area, and then the orientation area is scanned and photographed, so that the obstacles around the user can be photographed and recorded to the maximum extent.

[0113] Preferably, the movement path determination module is used to analyze the environmental image to determine the existence state of obstacles in the space area between the teaching robot and the user; according to the existence state of the obstacles, determining the movement path of the teaching robot to the user's location specifically includes:

[0114] The environment image is converted into a grayscale environment image, and corresponding image pixel contour information is extracted from the grayscale environment image; and the position and size of obstacles existing in the space area between the teaching robot and the user are determined according to the image pixel contour information;

[0115] According to the position and size of the obstacle, a continuous moving path capable of accommodating the movement of the teaching robot is determined in the space between the teaching robot and the user, so that the teaching robot can move along the continuous moving path to the position of the user;

[0116] as well as,

[0117] The voice reply message playing module is used to instruct the teaching robot to feed back a corresponding voice reply message to the user according to the semantic information during the movement of the teaching robot along the moving path, and specifically includes:

[0118] A voice reply message matching the semantic information is selected from a preset response voice database; and the voice reply message is repeatedly played with a variable volume while the teaching robot moves along the continuous moving path, wherein the variable volume means that the closer the teaching robot is to the user during the movement, the smaller the corresponding volume.

[0119] The beneficial effect of the above technical solution is: since the image pixel contour corresponding to the obstacle in the environmental image will be different from the image pixel contour of other areas in the contour line direction and contour line density (the contour line of the image pixel contour corresponding to the obstacle is usually curved and the contour line density is also large, while the contour line of the image pixel contour of other areas is usually straight and the contour line density is also small), the above difference can accurately determine the obstacle position and obstacle size in the space area between the teaching robot and the user. When the obstacle position and obstacle size are determined, the gap area between different obstacles can be determined, and then these gap areas are spliced ​​to determine the continuous moving path that can accommodate the movement of the teaching robot, so that the teaching robot can move smoothly and quickly to the user's position without colliding with the obstacle. In addition, a voice reply message that matches the semantic information is selected from the preset answer voice database, which can improve the interactive flexibility between the teaching robot and the user (for example, the voice information can be "come here", and the corresponding voice reply message can be "OK"). And when the teaching robot moves along the continuous moving path, the voice reply message is repeatedly played with varying volume, so that when the teaching robot is far away from the user, the voice reply message can be clearly heard, and when the teaching robot is close to the user, the voice reply message can be prevented from being too loud and damaging the user's hearing.

[0120] It can be seen from the contents of the above embodiments that the motion control method and system of the interactive teaching robot collects sound information around the environment in which the teaching robot is located, controls the teaching robot to switch working states and realize wake-up according to the voiceprint feature information and semantic information in the sound information, and then determines the orientation area where the user is located according to the sound intensity distribution information in the sound information to facilitate the capture of the environmental image; then, identifies the existing obstacles in the environmental image to determine the moving path of the teaching robot, and instructs the teaching robot to play corresponding voice reply messages during the movement process, so as to ensure that the teaching robot can follow the user in a timely and accurate manner under the user's voice control, and quickly interact with the user by voice, thereby improving the movement flexibility of the teaching robot in the process of interaction with students and improving the intelligence and automation of the interaction between the teaching robot and students.

[0121] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. Motion control method of interactive teaching robot, It is characterized in that It includes the following steps: Step S1, collecting sound information around the environment where the teaching robot is located; analyzing the sound information to obtain voiceprint feature information, semantic information and sound intensity distribution information corresponding to the sound information; and then controlling the teaching robot to switch working states according to the voiceprint feature information and the semantic information; Step S2, determining the relative position of the user and the teaching robot in the environment where the teaching robot is located according to the sound intensity distribution information; and then photographing the position area where the user is located according to the relative position, so as to obtain a corresponding environment image; Step S3, analyzing the environmental image to determine the existence status of obstacles in the space area between the teaching robot and the user; determining a moving path for the teaching robot to move to the user's location according to the existence status of the obstacles; and instructing the teaching robot to feedback a corresponding voice reply message to the user in response to the semantic information during the movement of the teaching robot along the moving path; Wherein, the step S3 comprises: Step S301, converting the environment image into a grayscale environment image, and extracting corresponding image pixel contour information from the grayscale environment image; and then determining the position and size of obstacles existing in the space area between the teaching robot and the user according to the image pixel contour information; Step S302, determining a continuous moving path that can accommodate the movement of the teaching robot in the space area between the teaching robot and the user according to the position and size of the obstacle, so that the teaching robot can move to the position of the user along the continuous moving path; Step S303, selecting a voice reply message that matches the semantic information from a preset reply voice database; and repeatedly playing the voice reply message with a variable volume while the teaching robot moves along the continuous moving path, wherein the variable volume means that the closer the teaching robot is to the user during the movement, the smaller the corresponding volume; Wherein, the step S301 includes: Step S3011: Analyze all pixel points in the space area between the teaching robot and the user based on the image pixel contour information to see if they are points inside the image pixel contour, and then determine whether they are pixel points belonging to obstacles. Mark all pixel points of obstacles in the space area between the teaching robot and the user to determine the position of obstacles. Specifically, the following formula (1) is used to determine the position of obstacles in the space area between the teaching robot and the user based on the image pixel contour information: In the above formula (1), A a Indicates the judgment value of whether the ath pixel point in the space area between the teaching robot and the user belongs to the pixel point of the obstacle. a = 1, it means that the ath pixel point belongs to the pixel point of the obstacle and the ath pixel point is marked to determine the obstacle position. a = 0, it means that the a-th pixel does not belong to the obstacle pixel and the a-th pixel is not marked. a ,Y a ) represents the coordinate value of the ath pixel point in the space between the teaching robot and the user, (X i ,Y i ) represents the coordinate value of the i-th contour point in the image pixel contour, X imin Indicates the minimum horizontal coordinate value of all contour points in the image pixel contour, X imax Indicates the maximum horizontal coordinate value of all contour points in the image pixel contour, Y imin Indicates the minimum ordinate value of all contour points in the image pixel contour, Y imax It represents the maximum ordinate value of all contour points in the image pixel contour, and n represents the total number of contour points contained in the image pixel contour; Step S3012, using the following formula (2), according to the image pixel profile information, determine the size of the obstacle in the space between the teaching robot and the user, In the above formula (2), S p represents the length of the pth row of pixels in the horizontal direction of the obstacle image corresponding to the spatial area between the teaching robot and the user in the real space, k represents the total number of pixels in each row in the horizontal direction corresponding to the spatial area between the teaching robot and the user, and L 0 Indicates the length of each row of pixels in the real space in the horizontal direction corresponding to the spatial area between the teaching robot and the user; Wherein, the step S302 includes: Using the following formula (3), according to the position and size of the obstacle, the space area between the teaching robot and the user is marked in the row and column directions, so that a number of pixel points are marked in the space area. In the above formula (3), A pq represents the pixel point annotation value obtained after annotating the space area between the teaching robot and the user in the direction of the pth row and the qth column, and L represents the length of the teaching robot in the horizontal direction; All pixel points whose pixel mark value is 1 are determined, and a continuous moving path is found in the space corresponding to the pixel points, so as to serve as a continuous moving path to accommodate the movement of the teaching robot.

2. The motion control method of the interactive teaching robot according to claim 1, Features: In the step S1, the sound information of the environment around the teaching robot is collected; the voiceprint feature information, semantic information and sound intensity distribution information corresponding to the sound information are obtained after analyzing the sound information; and the teaching robot is controlled to switch the working state according to the voiceprint feature information and the semantic information; specifically, the method includes: Step S101, using a microphone array to scan and collect sound information from all directions of the environment in which the teaching robot is located; and performing Kalman filtering on the sound information to remove background noise contained in the sound information; Step S102, performing voiceprint recognition on the sound information, thereby extracting voiceprint feature information contained in the sound information; performing semantic recognition on the sound information, thereby extracting semantic information corresponding to the sound information; performing sound intensity waveform analysis on the sound information, thereby extracting sound intensity peak distribution contained in the sound information; Step S103, compare the voiceprint feature information with the actual voiceprint feature of the user stored in advance; if the two are different, control the teaching robot to maintain a dormant state; if the two are the same, compare the semantic information with the pre-stored semantic keywords; if the semantic information does not contain the semantic keywords, control the teaching robot to maintain a dormant state; if the semantic information contains the semantic keywords, control the teaching robot to switch from the dormant state to the normal working state.

3. The motion control method of the interactive teaching robot as claimed in claim 2, Features: In the step S2, the relative position of the user and the teaching robot in the environment where the teaching robot is located is determined according to the sound intensity distribution information; Then, according to the relative position, the position area where the user is located is photographed, so as to obtain the corresponding environmental image, which specifically includes: Step S201, analyzing the sound intensity distribution information to obtain a corresponding sound intensity waveform distribution; extracting a sound intensity peak with a maximum amplitude from the sound intensity waveform distribution, and determining when the sound intensity peak with a maximum amplitude appears, scanning the sound information of the environment where the teaching robot is located to collect the corresponding spatial orientation, so as to use the spatial orientation as the relative orientation of the user and the teaching robot in the environment where the teaching robot is located; Step S202, the spatial orientation is expanded by a preset multiple of the spatial orientation angle as the orientation area where the user is located, and the orientation area where the user is located is scanned and photographed to obtain a corresponding environmental image.

4. Motion control system of interactive teaching robot, It is characterized in that It includes a sound collection and analysis module, a working state switching module, an environmental image shooting module, a moving path determination module and a voice reply message playing module; the control system is used to execute the motion control method according to claim 1, wherein, The sound collection and analysis module is used to collect sound information around the environment where the teaching robot is located; after analyzing the sound information, the voiceprint feature information, semantic information and sound intensity distribution information corresponding to the sound information are obtained; The working state switching module is used to control the teaching robot to switch the working state according to the voiceprint feature information and the semantic information; The environmental image shooting module is used to determine the relative position of the user and the teaching robot in the environment where the teaching robot is located according to the sound intensity distribution information; and then shoot the position area where the user is located according to the relative position, so as to obtain the corresponding environmental image; The moving path determination module is used to analyze the environmental image to determine the existence status of obstacles in the space area between the teaching robot and the user; according to the existence status of the obstacles, determine the moving path of the teaching robot to the user's location; The voice reply message playing module is used to instruct the teaching robot to feed back a corresponding voice reply message to the user based on the semantic information when the teaching robot moves along the moving path.

5. The motion control system of the interactive teaching robot as claimed in claim 4, Features: The sound collection and analysis module is used to collect sound information around the teaching robot; after analyzing the sound information, the voiceprint feature information, semantic information and sound intensity distribution information corresponding to the sound information are obtained, which specifically include: Collecting sound information around the environment where the teaching robot is located; analyzing the sound information to obtain voiceprint feature information, semantic information and sound intensity distribution information corresponding to the sound information; and then controlling the teaching robot to switch working states according to the voiceprint feature information and the semantic information; Performing voiceprint recognition on the sound information to extract voiceprint feature information contained in the sound information; performing semantic recognition on the sound information to extract semantic information corresponding to the sound information; performing sound intensity waveform analysis on the sound information to extract sound intensity peak distribution contained in the sound information; as well as, The working state switching module is used to control the teaching robot to switch the working state according to the voiceprint feature information and the semantic information, and specifically includes: Comparing the voiceprint feature information with the pre-stored actual voiceprint feature of the user; If the two are different, the teaching robot is controlled to remain in a dormant state; if the two are the same, the semantic information is compared with pre-stored semantic keywords; if the semantic information does not contain the semantic keywords, the teaching robot is controlled to remain in a dormant state; if the semantic information contains the semantic keywords, the teaching robot is controlled to switch from the dormant state to the normal working state.

6. The motion control system of the interactive teaching robot as claimed in claim 5, Features: The environmental image shooting module is used to determine the relative position of the user and the teaching robot in the environment where the teaching robot is located according to the sound intensity distribution information; and then shoot the position area where the user is located according to the relative position, so as to obtain the corresponding environmental image. Specifically, it includes: analyzing the sound intensity distribution information to obtain the corresponding sound intensity waveform distribution; extracting the sound intensity peak with the largest amplitude from the sound intensity waveform distribution, and determining when the sound intensity peak with the largest amplitude appears, scanning the sound information of the environment where the teaching robot itself is located to collect the corresponding spatial orientation, so as to use the spatial orientation as the relative position of the user and the teaching robot in the environment where the teaching robot is located; The spatial orientation is expanded by a preset multiple of the spatial orientation angle as the orientation area where the user is located, and the orientation area where the user is located is scanned and photographed to obtain a corresponding environmental image.

Images