Toy interaction action recognition system based on machine vision
By establishing joint fluctuation association tree and interactive motion joint node cloud trajectory in the toy interactive action recognition system, the accuracy and real-time response speed of toy interactive action recognition in complex backgrounds are solved, and higher recognition accuracy and response speed are achieved .
Patent Information
- Application Number
- CN202411093339.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-11-19
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing toy interactive action recognition technologies are difficult to accurately identify user actions in complex contexts, and the diversity of actions leads to insufficient recognition accuracy and real-time response speed.
Adopting a toy interactive action recognition system based on machine vision, through the interactive data analysis module of the cloud computing platform and the toy interactive decision generation module, a matching relationship between the joint fluctuation correlation tree and the toy recognition point displacement trajectory is established, and interactive motion is generated based on real-time scene video data. The node cloud trajectory is set, the interaction time node is matched, and toy interaction instructions are generated and executed.
It reduces the interference of the scene environment on interactive action recognition, improves the accuracy and real-time response speed of interactive action recognition, and pays attention to the action judgment of target personnel.
Smart Images

Figure CN118982845A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of machine vision, and in particular to a toy interactive action recognition system based on machine vision. Background Art
[0002] With the rapid development of artificial intelligence and machine vision technology, smart toys have gradually become a hot spot in the market. Smart toys can not only interact with users in a basic way, but also achieve a higher level of interactive experience through complex algorithms.
[0003] The existing toy interactive action recognition intelligent recognition technology has the following defects: Background complexity problem: Complex backgrounds can interfere with action recognition algorithms, making it difficult to extract the user's specific actions from the image. For example, moving objects or texture changes in the background may be mistaken for the user's actions.
[0004] Action diversity problem: Users’ actions and gestures are diverse and not fixed, which places high demands on the system’s action recognition. The accuracy of recognition needs to cover all possible action patterns.
[0005] Therefore, how to reduce the interference of the scene environment on the interactive action recognition while improving the accuracy and real-time response speed of the interactive action recognition is a difficulty in the prior art. To this end, a toy interactive action recognition system based on machine vision is provided. Summary of the invention
[0006] In order to solve the above technical problems, an object of the present invention is to provide a toy interactive action recognition system based on machine vision.
[0007] In order to achieve the above object, the present invention provides the following technical solutions: A toy interactive action recognition system based on machine vision includes a cloud computing platform, wherein the cloud computing platform is communicatively connected with an interactive data analysis module and a toy interactive decision generation module; The interactive data analysis module is used to establish a toy identification point displacement trajectory and a person identification point displacement trajectory according to several pre-stored historical toy interaction event records, extract several trajectory fluctuation points from the person identification point displacement trajectory, connect each trajectory fluctuation point in sequence according to the human joint position corresponding to each trajectory fluctuation point to obtain a joint fluctuation association tree, and then match and associate the joint fluctuation association tree with the toy identification point displacement trajectory, establish a person-toy interaction trajectory according to the matching association result, and cross-connect each person-toy interaction trajectory to obtain an interactive action feature trajectory; The toy interaction decision generation module is provided with an interaction scene analysis unit and a scene interaction decision generation unit; The interactive scene analysis unit is used to collect real-time scene video data of the interactive scene, establish a corresponding interactive scene model according to the real-time scene video data, and then establish an interactive motion joint point cloud trajectory according to the interactive scene model; The scene interaction decision generation unit is used to set a number of interaction time nodes, match the interactive motion joint point cloud trajectory with each interactive action feature trajectory, generate and execute toy interaction instructions according to the matching, until the target person's interactive action ends.
[0008] Furthermore, the historical toy interaction event record is composed of the name of the intelligent toy type, the name of the toy interaction event and a plurality of historical multi-view interaction video data, wherein each of the historical multi-view interaction video data is collected at a different time, but all completely correspond to the same toy interaction event; According to the historical multi-view interactive video data, the corresponding historical interactive scene dynamic model is established, and the toy dynamic model and the target person dynamic model are extracted from each historical interactive scene dynamic model through image feature extraction technology.
[0009] Furthermore, the process of establishing the joint fluctuation association tree includes: Toy motion recognition points and personnel motion recognition points are set at the motion joint positions of the toy dynamic model and the target personnel dynamic model, and displacement trajectories of each toy motion recognition point and personnel motion recognition point on each toy dynamic model and the target personnel dynamic model are obtained respectively, and recorded as toy recognition point displacement trajectory and personnel recognition point displacement trajectory respectively; Establish a three-dimensional coordinate system, map the displacement trajectories of all personnel identification points to the three-dimensional coordinate system in order of their height on the human body, set the curve slope threshold, and then obtain the slope value of each spatial coordinate point on the displacement trajectory of each personnel identification point. If the slope value is greater than or equal to the curve slope threshold, mark the corresponding spatial coordinate point as a trajectory fluctuation point. If the slope value is less than the curve slope threshold, do nothing. The trajectory fluctuation points of the displacement trajectory of each personnel identification point are extracted respectively, and the trajectory fluctuation points are spliced according to the height order of the displacement trajectory of each personnel identification point corresponding to the human joints and the time order of each trajectory fluctuation point to obtain the joint fluctuation association tree.
[0010] Furthermore, the process of establishing a person-toy interaction trajectory according to the joint fluctuation association tree and the toy identification point displacement trajectory includes: According to the joint fluctuation association tree, corresponding mutually related displacement trajectory segments of the personnel identification points are intercepted from the displacement trajectory of each personnel identification point, and the displacement trajectory segments of each personnel identification point are grouped according to the number of trajectory fluctuation points at the highest level of the joint fluctuation association tree; At the same time, according to the time correlation interval of the trajectory fluctuation point at the lowest level of the joint fluctuation association tree, the toy identification point displacement trajectory fragment is retrieved from the toy identification point displacement trajectory and recorded as the toy level. Each toy identification point displacement trajectory fragment is associated with the corresponding personnel identification point displacement trajectory fragment, and then several displacement trajectory fragment combinations of each toy interaction event are obtained.
[0011] All displacement trajectory pieces are mapped into the three-dimensional coordinate system respectively, and several interaction time nodes are set. Then, the coordinate points at each interaction time node are extracted from the displacement trajectory segments of the personnel identification points and the displacement trajectory segments of the toy identification points respectively. The coordinate points extracted from each coordinate point are connected in sequence according to the joint level and the toy level, so as to obtain the personnel-toy interaction trajectory at the corresponding interaction time node.
[0012] Furthermore, the process of establishing the interactive action feature trajectory according to the human-toy interaction trajectory includes: According to the time sequence of the first coordinate point of each person-toy interaction trajectory in the joint fluctuation association tree, the person-toy interaction trajectories are arranged in sequence, and according to the position of the trajectory fluctuation point in the joint fluctuation association tree that is simultaneously connected to multiple trajectory fluctuation points at a higher level than itself, the person-toy interaction trajectories are cross-connected, thereby obtaining the interaction action feature trajectory of the corresponding toy interaction event; By establishing the interactive action feature tracks of the toy interactive events as described above, the interactive data analysis module establishes the interactive action feature tracks of each toy interactive event, and marks the corresponding toy interactive event name and toy category name.
[0013] Furthermore, the process of establishing the interactive motion joint point cloud trajectory includes: The interactive scene analysis unit establishes a real-time interactive scene model according to each real-time scene video data, extracts a toy motion joint model and a part of the real-time personnel image model from the real-time interactive scene model established by each real-time scene video data, matches and splices each toy motion joint model to obtain a real-time intelligent toy image model, and matches and splices each part of the real-time personnel image model to obtain a real-time personnel image model; The interactive scene analysis unit marks the motion joint positions on the real-time intelligent toy image model and the real-time personnel image model respectively, and then establishes the interactive motion joint point cloud trajectory according to the running joint positions of the real-time intelligent toy image model and the real-time personnel image model.
[0014] Furthermore, the process of establishing the toy interaction instruction includes: Input the interactive motion joint point cloud trajectory into the three-dimensional coordinate system, and match the corresponding interactive action feature trajectory according to the interaction type name; Set a number of interaction time nodes, and then obtain the real-time human-toy interaction trajectory composed of the coordinate points of each motion joint position of the interactive motion joint point cloud trajectory at each interaction time node; Obtain the overlap between the human-toy interaction trajectory and the real-time human-toy interaction trajectory of each interactive action feature trajectory at each interactive time node; Set a coincidence threshold to determine whether the coincidence between the human-toy interaction trajectory of each interactive action feature trajectory and the real-time human-toy interaction trajectory is greater than or equal to the coincidence threshold, and generate a toy interaction instruction based on the determination result.
[0015] Furthermore, the execution process of the toy interaction instruction includes: The toy interaction decision generation module controls the position of the motion joints by calling the corresponding components of the intelligent toy to execute the toy interaction instructions. During the process of the intelligent toy executing the toy interaction instructions, the toy interaction decision generation module determines in real time whether the overlap between the person-toy interaction trajectory corresponding to each toy interaction instruction and the real-time person-toy interaction trajectory is greater than or equal to the repetition threshold; The toy interaction instruction for the next interaction time node is adjusted according to the judgment result until the interaction between the target person and the smart toy is completed.
[0016] Compared with the prior art, the present invention has the following beneficial effects: The present invention connects each trajectory fluctuation point in sequence to obtain a joint fluctuation association tree, and then matches and associates the joint fluctuation association tree with the toy identification point displacement trajectory to establish an interactive action feature trajectory, establishes a corresponding interactive scene model according to real-time scene video data, and then establishes an interactive motion joint point cloud trajectory according to the interactive scene model, sets a number of interactive time nodes, matches the interactive motion joint point cloud trajectory with each interactive action feature trajectory, generates and executes toy interaction instructions according to the matching, reduces the interference of the scene environment on the interactive action recognition, and compared with the traditional technology, the present invention focuses on the actions of the target person to judge the interactive action, and improves the accuracy of interactive action recognition and the real-time response speed. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention.
[0018] Figure 1 It is a schematic diagram of the present invention. DETAILED DESCRIPTION
[0019] To make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described in detail below. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other implementation methods obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.
[0020] like Figure 1 As shown, a toy interactive action recognition system based on machine vision includes a cloud computing platform, wherein the cloud computing platform is communicatively connected with an interactive data analysis module and a toy interactive decision generation module; The interactive data analysis module is used to establish a toy identification point displacement trajectory and a person identification point displacement trajectory according to several pre-stored historical toy interaction event records, extract several trajectory fluctuation points from the person identification point displacement trajectory, connect each trajectory fluctuation point in sequence according to the human joint position corresponding to each trajectory fluctuation point to obtain a joint fluctuation association tree, and then match and associate the joint fluctuation association tree with the toy identification point displacement trajectory, establish a person-toy interaction trajectory according to the matching association result, and cross-connect each person-toy interaction trajectory to obtain an interactive action feature trajectory; The toy interaction decision generation module is provided with an interaction scene analysis unit and a scene interaction decision generation unit; The interactive scene analysis unit is used to collect real-time scene video data of the interactive scene, establish a corresponding interactive scene model according to the real-time scene video data, and then establish an interactive motion joint point cloud trajectory according to the interactive scene model; The scene interaction decision generation unit is used to set a number of interaction time nodes, match the interactive motion joint point cloud trajectory with each interactive action feature trajectory, generate and execute toy interaction instructions according to the matching, until the target person's interactive action ends.
[0021] Further, the working principle of the present invention is described below by way of examples: The historical toy interaction event record consists of the name of the intelligent toy type, the name of the toy interaction event and a number of historical multi-view interaction video data, wherein each historical multi-view interaction video data is collected at a different time, but all completely correspond to the same toy interaction event; According to the historical multi-view interactive video data, the corresponding historical interactive scene dynamic model is established, and the toy dynamic model and the target person dynamic model are extracted from each historical interactive scene dynamic model through image feature extraction technology; Obtain Num types of smart toy information through the Internet, wherein the smart toy information includes a smart toy name, smart toy component information, and corresponding smart toy component image data, wherein the smart toy component information includes a component name and a component function, for example, a component name is a controller, which is used to receive and execute interactive instructions, and a component name is an actuator, which is used to drive a motor and a wheel according to the interactive instructions, wherein Num is a natural number greater than 0; The interactive data analysis module matches the smart toy component image data of various smart toys with the corresponding toy dynamic models, and marks the component names and component functions on the toy dynamic models according to the matching.
[0022] Furthermore, toy action recognition points and personnel action recognition points are set at the motion joint positions of the toy dynamic model and the target personnel dynamic model. In practical applications, the motion joint positions of the toy dynamic model are, for example, toy mechanical arms and wheels. It should be noted that for circular components such as wheels whose motion joint positions are circular components, the toy action recognition points are set at their edge positions. For the motion joint positions of the target person's dynamic model, such as finger joints, elbow joints, etc.; Respectively obtain the displacement trajectories of each toy action recognition point and the person action recognition point on each toy dynamic model and the target person dynamic model, and record them as the toy recognition point displacement trajectory and the person recognition point displacement trajectory respectively; Establish a three-dimensional coordinate system, map the displacement trajectories of all personnel identification points to the three-dimensional coordinate system in order of their height on the human body, set the curve slope threshold, and then obtain the slope value of each spatial coordinate point on the displacement trajectory of each personnel identification point. If the slope value is greater than or equal to the curve slope threshold, mark the corresponding spatial coordinate point as a trajectory fluctuation point. If the slope value is less than the curve slope threshold, do nothing. The trajectory fluctuation points of the displacement trajectory of each personnel identification point are extracted respectively, and the trajectory fluctuation points are spliced according to the height order of the displacement trajectory of each personnel identification point corresponding to the human joints and the time order of each trajectory fluctuation point to obtain a joint fluctuation association tree; Each trajectory fluctuation point in the joint fluctuation association tree is connected in sequence according to the height order of the corresponding human joints. For example, the human joint corresponding to the highest level trajectory fluctuation point is the shoulder joint, and the human joint corresponding to the lowest level trajectory fluctuation point is the foot joint. A time correlation interval is set for each level of trajectory fluctuation points, and the time correlation intervals of each level are reduced from high to low according to the corresponding trajectory fluctuation point level, where the middle time node of the time correlation interval is the time node of the corresponding trajectory fluctuation point; The trajectory fluctuation points of the same level are connected in parallel. For the trajectory fluctuation points of different levels, if the time nodes corresponding to the lower-level trajectory fluctuation points of two adjacent trajectory fluctuation points are within the time correlation interval of the higher-level trajectory fluctuation points, then the pair of trajectory fluctuation points are connected to each other, otherwise they are not connected; It should be noted that if there is a trajectory fluctuation point that satisfies the connection condition with multiple trajectory fluctuation points at another level, they are also connected to each other at the corresponding trajectory fluctuation points; According to the joint fluctuation association tree, corresponding mutually related displacement trajectory segments of the personnel identification points are intercepted from the displacement trajectory of each personnel identification point, and the displacement trajectory segments of each personnel identification point are grouped according to the number of trajectory fluctuation points at the highest level of the joint fluctuation association tree; At the same time, according to the time correlation interval of the trajectory fluctuation point at the lowest level of the joint fluctuation association tree, the toy identification point displacement trajectory fragment is retrieved from the toy identification point displacement trajectory and recorded as the toy level. Each toy identification point displacement trajectory fragment is associated with the corresponding personnel identification point displacement trajectory fragment, and then several displacement trajectory fragment combinations of each toy interaction event are obtained.
[0023] Furthermore, all displacement trajectory pieces are mapped into a three-dimensional coordinate system, and a number of interaction time nodes are set, and then the coordinate points at each interaction time node are extracted from the displacement trajectory segments of the personnel identification points and the displacement trajectory segments of the toy identification points, and the coordinate points extracted from each coordinate point are sequentially connected according to the joint level and the toy level, so as to obtain the personnel-toy interaction trajectory at the corresponding interaction time node; According to the time sequence of the first coordinate point of each person-toy interaction trajectory in the joint fluctuation association tree, the person-toy interaction trajectories are arranged in sequence, and according to the position of the trajectory fluctuation point in the joint fluctuation association tree that is simultaneously connected to multiple trajectory fluctuation points at a higher level than itself, the person-toy interaction trajectories are cross-connected, thereby obtaining the interaction action feature trajectory of the corresponding toy interaction event; By establishing the interactive action feature tracks of the toy interactive events as described above, the interactive data analysis module establishes the interactive action feature tracks of each toy interactive event, and marks the corresponding toy interactive event name and toy category name.
[0024] Further, the interaction data analysis module sends the interaction action feature trajectories of all toy interaction events to the toy interaction decision generation module; Install multiple cameras on each motion joint of the smart toy, and set a number s1, s2, s3, ..., s m , where m is a natural number greater than 0; It should be noted that the shooting range of each camera on the smart toy includes the position of the motion joint where the next numbered camera is located according to the size relationship of the numbered subscripts, and the combined result of the shooting ranges of each camera is an all-round shooting angle centered on the smart toy; The interactive scene analysis unit in the toy interactive decision generation module communicates with the camera on the smart toy, so that the camera on the smart toy captures real-time scene video data of the interactive scene, and synchronously uploads the real-time scene video data to the interactive scene analysis unit; The interactive scene analysis unit establishes a real-time interactive scene model based on each real-time scene video data. Since the shooting ranges of each camera on the smart toy are cross-related, the corresponding real-time scene video data are partially the same; Therefore, the toy motion joint model and some real-time personnel image models are extracted from the real-time interactive scene model established by each real-time scene video data, and the toy motion joint models are matched and spliced to obtain a real-time intelligent toy image model, and the real-time personnel image model is matched and spliced to obtain a real-time personnel image model.
[0025] Further, the interactive scene analysis unit marks the motion joint positions on the real-time intelligent toy image model and the real-time personnel image model respectively, and then establishes the interactive motion joint point cloud trajectory according to the running joint positions of the real-time intelligent toy image model and the real-time personnel image model, and marks the toy type name, and sends the interactive motion joint point cloud trajectory to the scene interaction decision generation unit; It should be noted that the interactive motion joint point cloud trajectory is updated synchronously with the update of the real-time scene video data; Input the interactive motion joint point cloud trajectory into the three-dimensional coordinate system, and match the corresponding interactive action feature trajectory according to the interaction type name; Set a number of interaction time nodes, and then obtain the real-time human-toy interaction trajectory composed of the coordinate points of each motion joint position of the interactive motion joint point cloud trajectory at each interaction time node; Obtain the overlap between the human-toy interaction trajectory and the real-time human-toy interaction trajectory of each interactive action feature trajectory at each interactive time node; Set a coincidence threshold. If the coincidence degree between a person-toy interaction trajectory of an interactive action feature trajectory and the real-time person-toy interaction trajectory is greater than or equal to the coincidence threshold, then it is determined that at the current time node, the interaction time between the target person and the smart toy is the toy interaction event corresponding to the interactive action feature trajectory; If the overlap is less than the overlap threshold, no operation is performed; If it is determined that at the current time node, the interaction action between the target person and the smart toy matches multiple toy interaction events at the same time, then a toy interaction instruction is generated based on the coordinate points of the motion joints of the smart toy corresponding to the person-toy interaction trajectories at the subsequent two interaction time nodes according to the matched interaction action feature trajectories; If it is determined that at the current time node, the interaction action between the target person and the smart toy matches only one toy interaction event, then a toy interaction instruction is generated based on the coordinate points of the motion joints of the smart toy corresponding to the person-toy interaction trajectories at the subsequent three interaction time nodes according to the matched interaction action feature trajectories; The toy interaction instruction includes the motion joint position for executing the interaction action and the target displacement space position.
[0026] Furthermore, the toy interaction decision generation module controls the position of the motion joints to execute the toy interaction instructions by calling the corresponding components of the intelligent toy; When the smart toy executes the toy interaction command, the toy interaction decision generation module determines in real time whether the human-toy interaction trajectory corresponding to each toy interaction command and the real-time human-toy interaction trajectory have a degree of overlap greater than or equal to a repetition threshold; If it is greater than or equal to, the toy interaction instruction is still executed, otherwise the toy interaction instruction is regenerated and executed at the next interaction time node; If the real-time human-toy interaction trajectory still has a coincidence degree between human-toy interaction trajectories corresponding to multiple toy interaction instructions at the same time that is greater than or equal to the repetition threshold, the toy interaction instruction corresponding to the human-toy interaction trajectory with the largest repetition degree is selected for execution; If there are two or more human-toy interaction trajectories at the same time, and the overlap between them and the real-time human-toy interaction trajectory is the largest, a toy interaction instruction is randomly selected to execute until the target person and the smart toy interaction action is completed.
[0027] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.
Claims
1. A toy interactive action recognition system based on machine vision, including a cloud computing platform, characterized in that: The cloud computing platform is communicatively connected to an interactive data analysis module and a toy interactive decision making module; The interactive data analysis module is used to establish a toy identification point displacement trajectory and a person identification point displacement trajectory according to several pre-stored historical toy interaction event records, extract several trajectory fluctuation points from the person identification point displacement trajectory, connect each trajectory fluctuation point in sequence according to the human joint position corresponding to each trajectory fluctuation point to obtain a joint fluctuation association tree, and then match and associate the joint fluctuation association tree with the toy identification point displacement trajectory, establish a person-toy interaction trajectory according to the matching association result, and cross-connect each person-toy interaction trajectory to obtain an interactive action feature trajectory; The toy interaction decision generation module is provided with an interaction scene analysis unit and a scene interaction decision generation unit; The interactive scene analysis unit is used to collect real-time scene video data of the interactive scene, establish a corresponding interactive scene model according to the real-time scene video data, and then establish an interactive motion joint point cloud trajectory according to the interactive scene model; The scene interaction decision generation unit is used to set a number of interaction time nodes, match the interactive motion joint point cloud trajectory with each interactive action feature trajectory, generate and execute toy interaction instructions according to the matching, until the target person's interactive action ends.
2. A toy interactive action recognition system based on machine vision according to claim 1, characterized in that: The historical toy interaction event record consists of the name of the smart toy type, the name of the toy interaction event and a number of historical multi-view interaction video data; According to the historical multi-view interactive video data, the corresponding historical interactive scene dynamic model is established, and the toy dynamic model and the target person dynamic model are extracted from each historical interactive scene dynamic model through image feature extraction technology.
3. A toy interactive action recognition system based on machine vision according to claim 2, characterized in that: The process of establishing the joint fluctuation association tree includes: Toy motion recognition points and personnel motion recognition points are set at the motion joint positions of the toy dynamic model and the target personnel dynamic model, and displacement trajectories of each toy motion recognition point and personnel motion recognition point on each toy dynamic model and the target personnel dynamic model are obtained respectively, and recorded as toy recognition point displacement trajectory and personnel recognition point displacement trajectory respectively; Establish a three-dimensional coordinate system, map the displacement trajectories of all personnel identification points to the three-dimensional coordinate system in order of their height on the human body structure, set the curve slope threshold, and then obtain the slope value of each spatial coordinate point on the displacement trajectory of each personnel identification point. If the slope value is greater than or equal to the curve slope threshold, mark the corresponding spatial coordinate point as a trajectory fluctuation point. If the slope value is less than the curve slope threshold, do nothing. The trajectory fluctuation points of the displacement trajectory of each personnel identification point are extracted respectively, and the trajectory fluctuation points are spliced according to the height order of the displacement trajectory of each personnel identification point corresponding to the human joints and the time order of each trajectory fluctuation point to obtain the joint fluctuation association tree.
4. A toy interactive action recognition system based on machine vision according to claim 3, characterized in that: The process of establishing a human-toy interaction trajectory according to the joint fluctuation association tree and the displacement trajectory of the toy identification point includes: According to the joint fluctuation association tree, corresponding mutually related displacement trajectory segments of the personnel identification points are intercepted from the displacement trajectory of each personnel identification point, and the displacement trajectory segments of each personnel identification point are grouped according to the number of trajectory fluctuation points at the highest level of the joint fluctuation association tree; At the same time, according to the time correlation interval of the trajectory fluctuation point at the lowest level of the joint fluctuation association tree, the displacement trajectory fragment of the toy identification point is retrieved from the displacement trajectory of the toy identification point and recorded as the toy level, and each toy identification point displacement trajectory fragment is associated with the corresponding personnel identification point displacement trajectory fragment, thereby obtaining a plurality of displacement trajectory fragment combinations of each toy interaction event; All displacement trajectory segments are mapped into the three-dimensional coordinate system respectively, and several interaction time nodes are set. Then, the coordinate points at each interaction time node are extracted from the displacement trajectory segments of the personnel identification points and the displacement trajectory segments of the toy identification points respectively. The coordinate points are connected in sequence according to the joint level and the toy level, so as to obtain the personnel-toy interaction trajectory at the corresponding interaction time node.
5. A toy interactive action recognition system based on machine vision according to claim 4, characterized in that: The process of establishing the interactive action feature trajectory according to the human-toy interaction trajectory includes: According to the time sequence of the first coordinate point of each personnel-toy interaction trajectory in the joint fluctuation association tree, the personnel-toy interaction trajectories are arranged in sequence, and according to the location of the trajectory fluctuation points in the joint fluctuation association tree that are simultaneously connected to multiple trajectory fluctuation points at a higher level than themselves, the personnel-toy interaction trajectories are cross-connected to obtain the interactive action feature trajectory of the corresponding toy interaction event, and the corresponding toy interaction event name and toy type name are marked.
6. A toy interactive action recognition system based on machine vision according to claim 5, characterized in that: The process of establishing the interactive motion joint point cloud trajectory includes: The interactive scene analysis unit establishes a real-time interactive scene model according to each real-time scene video data, extracts a toy motion joint model and a part of the real-time personnel image model from the real-time interactive scene model established by each real-time scene video data, matches and splices each toy motion joint model to obtain a real-time intelligent toy image model, and matches and splices each part of the real-time personnel image model to obtain a real-time personnel image model; The interactive scene analysis unit marks the motion joint positions on the real-time intelligent toy image model and the real-time personnel image model respectively, and then establishes the interactive motion joint point cloud trajectory according to the running joint positions of the real-time intelligent toy image model and the real-time personnel image model.
7. A toy interactive action recognition system based on machine vision according to claim 6, characterized in that: The process of establishing the toy interaction instruction includes: Input the interactive motion joint point cloud trajectory into the three-dimensional coordinate system, and match the corresponding interactive action feature trajectory according to the interaction type name; Set a number of interaction time nodes, and then obtain the real-time human-toy interaction trajectory composed of the coordinate points of each motion joint position of the interactive motion joint point cloud trajectory at each interaction time node; Obtain the overlap between the human-toy interaction trajectory and the real-time human-toy interaction trajectory of each interactive action feature trajectory at each interactive time node; Set a coincidence threshold to determine whether the coincidence between the human-toy interaction trajectory of each interactive action feature trajectory and the real-time human-toy interaction trajectory is greater than or equal to the coincidence threshold, and generate a toy interaction instruction based on the determination result.
8. The toy interactive action recognition system based on machine vision according to claim 7, characterized in that: The execution process of the toy interaction instruction includes: The toy interaction decision generation module controls the position of the motion joints by calling the corresponding components of the intelligent toy to execute the toy interaction instructions. During the process of the intelligent toy executing the toy interaction instructions, the toy interaction decision generation module determines in real time whether the overlap between the person-toy interaction trajectory corresponding to each toy interaction instruction and the real-time person-toy interaction trajectory is greater than or equal to the repetition threshold; The toy interaction instruction for the next interaction time node is adjusted according to the judgment result until the interaction between the target person and the smart toy is completed.
