Image-based generation method and interactive learning apparatus
Patent Information
- Application Number
- TW114120813
- Authority / Receiving Office
- TW · TW
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2024-08-13
- Filing Date
- 2025-06-04
- Publication Date
- 2026-11-11
- Estimated Expiration
- 2045-06-03
AI Technical Summary
There is a lack of mobile devices designed for child users aged 3 to 7 with AI-assisted educational functions.
An image-based generation method and interactive learning device that utilizes a processor to identify a target object in an input image, perform pixel position transformation, and input the transformed object image to an artificial intelligence model to generate derivative objects, leveraging AI models trained on datasets to map input data to corresponding derivative objects based on contour, shape, or pattern.
Enables the conversion of target objects into derivative objects through pixel position conversion, stimulating children's creativity and enhancing cognitive development by providing interactive learning experiences.
Smart Images

Figure TWG2TB001920612_001 
Figure TWG2TB001920612_002 
Figure TWG2TB001920612_003
Abstract
Description
[Technical Field]
[0001] This invention relates to an artificial intelligence-based generation technology, and more particularly to an image-based generation method and interactive learning device. [Previous Technology]
[0002] Currently, there are many artificial intelligence (AI) related products on the market. However, in the existing technology, there is a lack of mobile devices specifically designed for child users (e.g., children aged 3 to 7) and with AI-assisted educational functions. [Summary of the Invention]
[0003] The present invention provides an image-based generation method and an interactive learning device, which can experience the derivative data generated by artificial intelligence through simple interactive operation.
[0004] The image-based generation method of this invention is implemented through a processor. The generation method includes (but is not limited to) the following steps: identifying a target object in an input image; performing pixel position transformation on the target object in the input image to generate a transformed object image; and inputting the transformed object image to an artificial intelligence model, and outputting a derivative object through the artificial intelligence model. The pixel position transformation maps the value corresponding to the first pixel coordinate of the target object in the input image to the second pixel coordinate. The artificial intelligence model is trained on a dataset and learns to map the input data to the corresponding derivative object, and the target object in the transformed object image corresponds to at least one of the contour, shape, or pattern of the derivative object.
[0005] The interactive learning device of this embodiment includes (but is not limited to) a memory and a processor. The memory is used to store program code. The processor is coupled to the memory. The processor is configured to load the program code to execute: identifying a target object in an input image; performing pixel position transformation on the target object in the input image to generate a transformed object image; and inputting the transformed object image to an artificial intelligence model, and outputting derivative objects through the artificial intelligence model. The pixel position transformation maps the value corresponding to a first pixel coordinate of the target object in the input image to a second pixel coordinate. The artificial intelligence model learns the association between the input data and the corresponding derivative objects, and the target object in the transformed object image corresponds to at least one of the contour, shape, or pattern of the derivative object.
[0006] Based on the above, the image-based generation method and interactive learning device of the present invention can convert a target object into a converted object image through pixel position conversion and generate derivative objects corresponding to the target object.
[0007] In order to make the above features and advantages of the present invention more apparent and understandable, specific embodiments are described below, and detailed descriptions are given in conjunction with the accompanying drawings.
Implementation Method
[0009] Figure 1 is a block diagram of an interactive learning device 100 and a server 50 according to an embodiment of the present invention. Referring to Figure 1, the server 50 may be a cloud server, a database, a computing platform, or a combination thereof. In one embodiment, the server 50 runs an artificial intelligence model (e.g., a large language model (LLM) or a generative artificial intelligence model).
[0010] The interactive learning device 100 may be a smartphone, tablet computer, wearable device, laptop computer, desktop computer, all-in-one PC, server, smart home appliance, smart assistant device, in-vehicle system, home game console, personal computer, artificial intelligence personal computer (AI PC) or other electronic device.
[0011] The interactive learning device 100 includes (but is not limited to) an image capturing device 110, an input device 120, a communication transceiver 130, a display 140, a speaker 145, a storage device 150, and a processor 160.
[0012] The image capturing device 110 may be a camera, video camera, or other image capturing device / module / component. In one embodiment, the image capturing device 110 is used to acquire images, that is, images captured through the lens of the image capturing device 110.
[0013] The input device 120 may be a touch panel, keyboard, mouse, handheld controller, microphone, or a combination thereof. In one embodiment, the input device 120 is used to receive user operations. These user operations correspond to drawing, writing, typing, function selection, and / or voice input.
[0014] The communication transceiver 130 may support communication transceiver circuits / transmission interfaces such as Bluetooth, Wi-Fi, mobile network, fiber optic network, Universal Serial Bus (USB), Thunderbolt, or other communication technologies. In one embodiment, the communication transceiver 130 is used to receive data from an external device (e.g., server 50) or transmit data to an external device (e.g., server 50).
[0015] The display 140 may be a screen using LCD, LED, OLED, or other display technologies. In one embodiment, the display 140 is used to display images.
[0016] The loudspeaker 145 (or horn) is used to play voice data or sound data.
[0017] The storage device 150 can be any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid-state drive (SSD), or similar element. In one embodiment, the storage device 150 is used to store code, software modules, configuration settings, data (e.g., image, voice data, or parameters), or files, as detailed in subsequent embodiments.
[0018] The processor 160 is coupled to the image capturing device 110, the input device 120, the communication transceiver 130, the display 140, the speaker 145, and the storage device 150. The processor 160 may be a central processing unit (CPU), a graphics processing unit (GPU), a data processing unit (DPU), a visual processing unit (VPU), a tensor processing unit (TPU), or a neural-network processing unit (NPU), or other programmable general-purpose or special-purpose microprocessors, digital signal processors (DSPs), programmable controllers, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), or other similar components or combinations thereof. In one embodiment, the processor 160 is used to perform all or part of the operations of the interactive learning device 100, and can load and execute one or more software modules, files and / or data stored in the storage 150.
[0019] In some embodiments, processor 160 may run an offline version of an artificial intelligence model (e.g., an LLM or a generative artificial intelligence model).
[0020] Figure 2 is a schematic diagram of an interactive learning device 100 and a server 50 according to an embodiment of the present invention. Referring to Figure 2, the interactive learning device 100 is exemplified by a tablet computer. The user U can hold the interactive learning device 100 in hand. Furthermore, the interactive learning device 100 is connected to the server 50.
[0021] The method described in the embodiments of the present invention will be described below with reference to the apparatus, components and modules shown in FIG1. Each step of this method may be adjusted according to the implementation situation, and is not limited thereto.
[0022] Figure 3 is a flowchart of an image-based generation method according to an embodiment of the present invention. Referring to Figure 3, the processor 160 identifies a target object in the input image (step S310). Specifically, the input image may be an image captured by the image capturing device 110, an image stored in the storage device 150, or an image transmitted by another device. Taking Figure 2 as an example, the user U holds the interactive learning device 100 and takes a picture of an airplane in the sky to obtain the input image.
[0023] In one embodiment, the processor 160 can perform foreground and background separation on the input image and extract a foreground image from the input image. Foreground and background separation can be performed to separate objects or elements located in the foreground from the background based on depth information, edge information, motion information, and / or semantic segmentation. The processor 160 can generate a foreground image based on the image region corresponding to the object or element in the foreground.
[0024] The processor 160 can perform object recognition on the foreground image based on object size to identify target objects. Object recognition can be achieved through image feature (e.g., scale-invariant feature transform (SIFT) or histogram of oriented gradient (HOG) alignment) or inference from deep learning models (e.g., AlexNet, VGG, ResNet, Inception). The processor 160 can pre-store or learn the object size of the target object to facilitate object type identification. For example, the area or outline of an object in the image may occupy a certain position. In some embodiments, the processor 160 can also extract foreground color patches from the foreground image to facilitate object type identification.
[0025] In some embodiments, the processor 160 may transmit input images to the server 50 via the transceiver 130, and the server 50 may identify the target object. The server 50 then transmits relevant information about the target object (e.g., type, location, quantity, and / or size) to the interactive learning device 100.
[0026] The processor 160 performs pixel position transformation on the target object in the input image to generate a transformed object image (step S320). Specifically, the pixel position transformation maps the value corresponding to the first pixel coordinate of the target object in the input image to the second pixel coordinate in the transformed object image. For example, the first pixel coordinate is (75, 65), and the second pixel coordinate is (85, 158). The RGB (red-green-blue) color value at the first pixel coordinate (i.e., the value corresponding to the above pixel coordinate) is (12, 52, 153). After pixel position transformation, the color value at the second pixel coordinate in the transformed object image is (12, 52, 153). That is, the value corresponding to the second pixel coordinate in the transformed object image is replaced by the original value corresponding to the first pixel coordinate in the input image. Similarly, the values corresponding to more pixel coordinates may be replaced by the original values corresponding to other pixel coordinates.
[0027] Pixel position transformation is, for example, image rotation or image segmentation. For example, a 90-degree leftward rotation of an image maps, for example, the value of a pixel position at the top left to the pixel position at the bottom left. Another example is image segmentation, which maps the value of a pixel position in the middle to the pixel position on the left. However, there are other types of pixel position transformations.
[0028] The processor 160 inputs the converted object image to the artificial intelligence model and outputs derivative objects through the artificial intelligence model (step S330). Specifically, the processor 160 can transmit the converted object image to the server 50 through the communication transceiver 130. The server 50 inputs the converted object image to the artificial intelligence model to output derivative objects. Alternatively, the processor 160 can input the converted object image to an offline version of the artificial intelligence model and output derivative objects accordingly.
[0029] An artificial intelligence model (e.g., an LLM or generative artificial intelligence model) learns the association between input data and corresponding derivative components. Input data may be, for example, images of transformed objects and / or question information (e.g., text content used to specify the association conditions for the output derivative components). The artificial intelligence model is trained on a large dataset (e.g., images of known objects, known text content used for questions, and known types of derivative components) and learns to map input data to corresponding derivative components (e.g., establishing an association between input data and output derivative components), thereby learning to predict the probability of the next word or symbol in the sequence. After training, the artificial intelligence model can receive input data (e.g., an image, the beginning of one or more sentences, one or more questions, one or more descriptions) and, based on the language rules it has learned, progressively generate subsequent text content (e.g., the type of derivative component).
[0030] The target object and the derivative object in the transformed object image correspond to at least one of the contour, shape, or pattern. Contour refers to the edge lines of an object in the image. Shape refers to the overall form and appearance of an object in the image. Pattern refers to a combination of lines, shapes, or textures that appear on the surface of an object in the image. In one embodiment, the artificial intelligence model can identify the derivative object in the transformed object image based on at least one of the contour, shape, or pattern. The transformed object image is generated by transforming the image region corresponding to the target object through pixel position conversion. The artificial intelligence model can find similar objects (as derivative objects) that match the contour, shape, and / or pattern of the target object after image rotation. Alternatively, the artificial intelligence model can find similar objects (as derivative objects) that match the contour, shape, and / or pattern of a portion of the target object after image segmentation.
[0031] In one embodiment, the type of the target object is different from the type of the derivative object. For example, the target object is a whale, while the derivative object is a goose with a similar outline to the target object in the converted object image.
[0032] In one embodiment, the processor 160 may use an artificial intelligence model to replace a target object in a converted object image with a derivative object. The server 50 or storage 150 pre-stores images of multiple objects. The artificial intelligence model may composite the images of the derivative objects into the converted object image to generate a new composite image.
[0033] The following are examples of application scenarios.
[0034] Figure 4 is a schematic diagram of image rotation-based generation according to an embodiment of the present invention. Referring to Figure 4, the user holds the interactive learning device 100 and takes a picture to obtain an input image IM1 (step S410). The interactive learning device 100 identifies the target object O1 in the input image IM1 (step S420). In some embodiments, the processor 160 can generate guiding text information based on the target object through an artificial intelligence model and provide the guiding text information through a display 140 or a speaker 145. The guiding text information includes text content related to the type, size, and / or color of the target object. For example, "What do you think it looks like?" or "What kind of fish do you think it is?"
[0035] The interactive learning device 100 rotates the input image IM1 by 90 degrees (step S430) to generate a transformed object image TOI1. The target object O1 corresponds to the first reference axis RL1 (e.g., the horizontal axis) in the input image IM1. After the image rotation, the target object O1 corresponds to the second reference axis RL2 (e.g., the vertical axis) in the transformed object image TOI1. At this time, the first reference axis RL1 is different from the second reference axis RL2.
[0036] The interactive learning device 100 transmits the converted object image TOI1 to the server 50 and interprets the converted object image TOI1 through an artificial intelligence model (step S440). The artificial intelligence model generates a new composite image SI1 (step S450). The new composite image SI1 includes the image of the derivative object DO1. The display 140 can display this new composite image SI1.
[0037] In one embodiment, the processor 160 may also generate guidance text information based on derivative components through an artificial intelligence model, and provide the guidance text information through the display 140 or the speaker 145. For example, "Do you think he looks like a goose?" or "Do you think he looks like a goose?"
[0038] Figure 5 is a schematic diagram of image segmentation-based generation according to an embodiment of the present invention. Referring to Figure 5, the interactive learning device 100 performs image segmentation on the input image IM1 (step S510) to generate three transformed object images IM11, IM12, and IM13. The input device 120 can receive a user's selection operation on one or more of the three transformed object images IM11, IM12, and IM13 (step S520).
[0039] For the transformed object image IM11 corresponding to the selection operation, the artificial intelligence model interprets the transformed object image IM11 (step S530). Next, the artificial intelligence model generates a new composite image SI2. The new composite image SI2 includes the image of the derivative object DO2. The display 140 can display this new composite image SI2. In addition, the display 140 or the speaker 145 provides guiding text information. For example, "Does it look like a brush?" or "Do you think it looks like a brush?"
[0040] For the transformed object image IM12 corresponding to the selection operation, the artificial intelligence model interprets the transformed object image IM12 (step S540). Next, the artificial intelligence model generates a new composite image SI3. The new composite image SI3 includes the image of the derivative object DO3. The display 140 can display this new composite image SI3. In addition, the display 140 or the speaker 145 provides guiding text information. For example, "Does it look like a potato chip?" or "Do you think it looks like a potato chip?"
[0041] For the transformed object image IM13 corresponding to the selection operation, the artificial intelligence model interprets the transformed object image IM13 (step S550). Next, the artificial intelligence model generates a new composite image SI4. The new composite image SI4 includes the image of the derivative object DO4. The display 140 can display this new composite image SI4. In addition, the display 140 or the speaker 145 provides guiding text information. For example, "Does it look like a bow and arrow?" or "Do you think it looks like a bow and arrow?"
[0042] In one embodiment, the processor 160 may also add a packaging layer to the derivative item (e.g., package it as a gift) and display the derivative item according to the user operation (e.g., click, scratch card) received by the input device 120.
[0043] In one embodiment, the processor 160 may retrieve recommendation information matching the derivative item from a database (located in storage 150 or server 50). The recommendation information is related to products, articles, or multimedia content. For example, storybooks, toys, or movies related to the derivative item.
[0044] Figure 6 is a flowchart of a trajectory-based generation method according to an embodiment of the present invention. Referring to Figure 6, the processor 160 can generate generation condition information based on input trajectory information (step S610). Specifically, the input trajectory information includes the positions of one or more trajectory points and / or the path formed based on one or more trajectory points. The input device 120 (e.g., a touch panel, mouse, or keyboard) receives the user's swiping operation (i.e., drawing or writing action) on the image presented on the display 140 and stores the trajectory points and / or paths corresponding to the swiping operation.
[0045] The generated condition information includes at least one of the following: object type parameters, object movement parameters, or user preference parameters. The processor 160 can identify the type based on the contour corresponding to the input trajectory information and generate object type parameters. For example, the object type parameter might be that dinosaurs and crocodiles belong to the same genus. The input device 120 (e.g., a touch panel, mouse, or keyboard) can further receive a second swipe operation (i.e., a drawing or handwriting action) from the user on the image displayed on the monitor 140, and the processor 160 generates object movement parameters based on the trajectory points and / or paths corresponding to the second swipe operation. Object movement parameters might be, for example, changing its own shape, such as adding wings or fins, or adding the use of tools, such as a boat, rocket, or hot air balloon. User preference parameters might be, for example, the user's historical drawing data or browsing history.
[0046] The processor 160 can output a second derivative component through the artificial intelligence model based on the generated condition information (step S620). Specifically, the processor 160 can convert the generated condition information into text content and use this text content as question information for the artificial intelligence model. The artificial intelligence model can generate a second derivative component based on the question information. For example, it can generate the type, size, color, and / or related information of the second derivative component.
[0047] In one embodiment, the artificial intelligence model can identify a second derivative object corresponding to the input trajectory information. For example, the artificial intelligence model can find similar objects (as second derivative objects) that match the contour of the input trajectory information.
[0048] In one embodiment, the processor 160 may retrieve recommendation information matching the second derivative item from a database (located in storage 150 or server 50). The recommendation information is related to products, articles, or multimedia content. For example, storybooks, toys, or movies related to the second derivative item.
[0049] The following are examples of application scenarios.
[0050] Figure 7 is a schematic diagram of trajectory-based generation according to an embodiment of the present invention. Referring to Figure 7, the user holds the interactive learning device 100 and takes a picture to obtain the input image IM2 (step S710). The interactive learning device 100 can instruct the artificial intelligence model to "guide a child aged 5 to 7 to create based on the scene content of the first three largest recognition ranges in the input image IM2" (step S720). The artificial intelligence model generates guiding text information and provides the guiding text information through the display 140 or the speaker 145. For example, "Do you think there are elves hidden under the lake?", "What fun things are there on the grass?", and "What's on the mountaintop?", where the scene content of the first three largest recognition ranges is a lake, a grassy area, and a mountain.
[0051] Input device 120 receives the user's swipe operation (i.e., drawing or writing gesture) and stores the trajectory points and / or paths corresponding to the swipe operation to generate input trajectory information T1 (step S740). Processor 150 records the process of the swipe operation (step S750). For example, it stores the time points of the trajectory points corresponding to the swipe operation and redraws the corresponding trajectory points or paths sequentially on the interface presented on display 140 according to these time points. Processor 160 retrieves recommendation information for second derivative items that match the input trajectory information T1. For example, it presents storybooks related to the second derivative items (step S760). Alternatively, it presents the product categories that the user is most interested in and least contacted with based on user preference parameters (step S770).
[0052] Figure 8 is a schematic diagram of trajectory-based generation according to an embodiment of the present invention. Referring to Figure 8, the user holds the interactive learning device 100 and takes a picture to obtain the input image IM2 (step S810). The artificial intelligence model analyzes the image features of the input image IM2 and generates guiding text information. The interactive learning device 100 can display or play the guiding text information (step S820).
[0053] The input device 120 receives the user's swiping operation (i.e., hand-drawing or handwriting action) and stores the trajectory points and / or paths corresponding to the swiping operation to generate input trajectory information T2 (step S825). The interactive learning device 100 generates generation condition information (e.g., object type parameters of the same type as the object corresponding to trajectory information T2) based on the input trajectory information T2, and outputs the guiding text information corresponding to the second derivative object through the artificial intelligence model (step S830). For example, "Dinosaurs fight crocodiles", "Dinosaurs explore downstream of the river by boat", or "Dinosaurs are captured by collars".
[0054] In addition, the artificial intelligence model can also generate new synthetic images with the same style and drawing format and interact (battle) with the original input image IM2 (step S840). Alternatively, the processor 160 can retrieve recommended information (e.g., insect battle books) matching dinosaurs from a database (located in storage 150 or server 50) (step S850).
[0055] Figure 9 is a schematic diagram of trajectory-based generation according to an embodiment of the present invention. Referring to Figure 9, the input device 120 receives a user's swipe operation (i.e., a hand-drawn or handwritten gesture) and stores the trajectory points and / or paths corresponding to the swipe operation to generate input trajectory information T2 (step S910). Next, the input device 120 receives a second swipe operation (i.e., a hand-drawn or handwritten gesture) from the user and stores the trajectory points and / or paths corresponding to the second swipe operation to generate second input trajectory information T3 (e.g., adding wings or fins, or adding a boat, rocket, or hot air balloon) (step S920). The processor 160 generates object movement parameters based on the trajectory points and / or paths corresponding to the second input trajectory information T3.
[0056] The processor 160 can output a new synthetic image IM3 corresponding to the object movement parameters through an artificial intelligence model based on the object movement parameters (step S930). Alternatively, the processor 160 can retrieve recommended information (e.g., cruise ship picture book) that matches the object movement parameters from a database (located in storage 150 or server 50) (step S940).
[0057] Figure 10 is a schematic diagram of trajectory-based generation according to an embodiment of the present invention. Referring to Figure 10, the user preference parameter HR is, for example, liking to play with pets, reading, swimming, science fiction novels and science fiction movies. The processor 160 can output a new synthetic image IM4 corresponding to the user preference parameter HR through an artificial intelligence model based on the user preference parameter HR (step S1010). Alternatively, the processor 160 can retrieve recommendation information (e.g., a custom Pokémon) that matches the user preference parameter HR from a database (located in storage 150 or server 50) (step S1020).
[0058] The following are application scenarios. Through the connection of sound, it is possible to extend the recommendation information (e.g., products or services).
[0059] Figure 11 is a schematic diagram of speech-based generation according to an embodiment of the present invention. Referring to Figure 11, the user holds the interactive learning device 100 and takes a picture to obtain an input image (step S1110). The artificial intelligence model analyzes the image features of the input image and determines that the target object is a hand-drawn creation (step S1120). The interactive learning device 100 can instruct the artificial intelligence model to "guide a child, for example, aged 3 to 5, to add sound to the creation based on the scene content of the first three largest recognition ranges in the input image". The artificial intelligence model generates guiding text information and provides the guiding text information through the display 140 or the speaker 145 (step S1130). For example, "How do you think a dog barks?" or "Tell your parents what the story you drew is about."
[0060] The input device 120 (taking a microphone as an example) can receive user voice data. For example, "Woof woof woof, do you think it sounds like me?" The processor 160 can store the user voice data in the storage 150 and make it available for later playback. Alternatively, an artificial intelligence model can generate dog bark data, and the speaker 145 can play the dog bark data (step S1140).
[0061] In addition, the processor 160 retrieves recommended information that matches dogs. For example, it presents dog-related storybooks. Alternatively, it presents product categories that the user is most interested in and least familiar with based on user preference parameters (step S1150).
[0062] Figure 12 is a schematic diagram of speech-based generation according to an embodiment of the present invention. Referring to Figure 12, the input device 120 (taking a microphone as an example) can receive user voice data (step S1210). For example, the user imitating the sound of a dog barking. The processor 160 retrieves recommendation information matching the dog (step S1220). For example, recommending teaching aids with different animal sounds.
[0063] The processor 160 can generate transformation information based on the input image and user voice data (step S1230). For example, the type information of the target object in the input image and the type information in the text data of the user voice data can be used to generate type information of similar objects (as transformation information). The processor 160 can generate a third derivative object based on the transformation information (step S1240). For example, the interactive learning device 100 can instruct the artificial intelligence model to "generate other objects of the same style based on the type of the target object in the input image and the dog". The display 140 can display a puppy (i.e., the third derivative object).
[0064] The input device 120 (using a microphone as an example) can receive user voice data (step S1230). For example, the sound of a user humming while watching a story. The processor 160 retrieves recommendation information that matches the user voice data (step S1240). For example, presenting a music-related storybook. The processor 160 can generate transformation information based on the input image (e.g., generated by taking a screenshot of the image displayed on the display 140) and the user voice data. For example, the type information of the target object in the input image and the type information in the text data of the user voice data can be used to generate type information of similar objects (as transformation information). The processor 160 can change the multimedia content (e.g., images, music, or videos) corresponding to the input trajectory information based on the transformation information (step S1260). For example, the interactive learning device 100 can instruct the artificial intelligence model to "generate an image based on the style of the background and the humming melody in the input image". The interactive learning device 100 can change the story atmosphere in the displayed image (e.g., jazz and psychedelic) based on the user's singing style.
[0065] Figure 13 is a schematic diagram illustrating the relationship between input data and generated data according to an embodiment of the present invention. Referring to Figure 13, the input data for the artificial intelligence model may be obtained through photography, video recording, drawing, dialogue, or historical records, and set as user preference parameters. The artificial intelligence model can identify / classify: for example, reading and literature, educational games and toys, music and performing arts (e.g., liking to narrate insects in filming; liking role-playing; describing stories in dialogue), technology and software programs, sports and outdoor activities (e.g., photographing insects; drawing many plants; high frequency of time spent outdoors; high frequency of watching football videos), cooking and life skills, and handicrafts and creative expression.
[0066] In one embodiment, the processor 160 can import the generated results and creation process, and re-select and present these contents according to weights. The weight order is, for example, as follows: 1. Time: from morning to evening 2. GPS: home, outing location 3. Newly encountered category (e.g., animal exploration) 4. Favorite type (science-related) 5. Content with the most interaction with AI 6. User-created content 7. Photos: the most diverse content 8. The longest production time 9. The most viewed. The processor 160 can know the user's preferences and learning level, and provide an analysis of the most interesting and the most needed missing categories (if identified as the same type, the weight is finally determined based on the time spent). In this way, the generated results of the artificial intelligence model can match the user's preferences and learning level.
[0067] For example, FIG14 is a schematic diagram showing the relationship between source data and generated data according to an embodiment of the present invention. Referring to FIG14, the source data is, for example, the aforementioned input image, and the image content in the generated data is derived from the content of the input image.
[0068] In summary, in the image-based generation method and interactive learning device of the present invention, derivative objects can be generated based on target objects transformed by pixel position, second derivative objects can be generated based on input trajectory information, and third derivative objects or multimedia content can be generated based on user voice data. Therefore, the present invention can read and analyze drawing, image, voice, and other data provided by children, and stimulate the creativity of child users through interactive methods such as AI question-and-answer guidance. The present invention can provide corresponding product information for parents to refer to when making purchases. Furthermore, when the target user is a child, it helps to improve cognitive development and learning characteristics.
[0069] Although the present invention has been disclosed above by way of embodiments, it is not intended to limit the present invention. Anyone skilled in the art can make some modifications and refinements without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims. [Simplified Explanation of the Diagram]
[0008] Figure 1 is a block diagram of an interactive learning device and server according to an embodiment of the present invention. Figure 2 is a schematic diagram of an interactive learning device and server according to an embodiment of the present invention. Figure 3 is a flowchart of an image-based generation method according to an embodiment of the present invention. Figure 4 is a schematic diagram of image rotation-based generation according to an embodiment of the present invention. Figure 5 is a schematic diagram of image segmentation-based generation according to an embodiment of the present invention. Figure 6 is a flowchart of a trajectory-based generation method according to an embodiment of the present invention. Figure 7 is a schematic diagram of trajectory-based generation according to an embodiment of the present invention. Figure 8 is a schematic diagram of trajectory-based generation according to an embodiment of the present invention. Figure 9 is a schematic diagram of trajectory-based generation according to an embodiment of the present invention. Figure 10 is a schematic diagram of trajectory-based generation according to an embodiment of the present invention. Figure 11 is a schematic diagram of speech-based generation according to an embodiment of the present invention. Figure 12 is a schematic diagram of speech-based generation according to an embodiment of the present invention. Figure 13 is a schematic diagram of the association between input data and generated data according to an embodiment of the present invention. Figure 14 is a schematic diagram of the association between source data and generated data according to an embodiment of the present invention.
Claims
1. An image-based generation method, implemented through a processor, the generation method comprising: Identify a target object in an input image; The target object in the input image undergoes a one-pixel position transformation to generate a transformed object image. The pixel position transformation maps the value corresponding to the first pixel coordinate of the target object in the input image to the second pixel coordinate. The pixel position transformation includes image rotation. The target object corresponds to a first reference axis in the input image and a second reference axis in the transformed object image. The first reference axis is different from the second reference axis. The transformed object image is input to an artificial intelligence model, and a derivative object is output through the artificial intelligence model. The artificial intelligence model is trained on a dataset and learns to map input data to corresponding derivative objects. The target object in the transformed object image and the derivative object correspond to at least one of contour, shape, or pattern. The type of the target object is different from the type of the derivative object.
2. The image-based generation method as described in claim 1, wherein the pixel position transformation includes image segmentation, and the transformed object image is a portion of the input image corresponding to the target object.
3. The image-based generation method as described in claim 1, wherein the step of outputting the derived component through the artificial intelligence model includes: The target object in the transformed object image is replaced with the derivative object.
4. The image-based generation method as described in claim 1, wherein the step of identifying the target object in the input image includes: Perform foreground-background separation on the input image and extract a foreground image; And to identify an object based on an object size in the foreground image, wherein the object size of the target object is pre-stored or learned.
5. The image-based generation method as described in claim 1 further includes: Generate guiding text information based on the target object or its derivative object; And provide the guiding text information.
6. The image-based generation method as described in claim 1 further includes: Retrieve a recommendation that matches the derivative item, wherein the recommendation is related to a product, an article, or multimedia content.
7. The image-based generation method as described in claim 1 further includes: Generate generation condition information based on an input trajectory information, wherein the input trajectory information includes the position of at least one trajectory point and / or the path formed based on the at least one trajectory point; and output a second derivative component through the artificial intelligence model based on the generation condition information.
8. The image-based generation method as described in claim 8, wherein the generation condition information includes at least one of the following: object type parameters; object movement parameters; or user preference parameters.
9. The image-based generation method as described in claim 1, further comprising: A change information is generated based on the input image and a user's voice data; And based on the change information, generate a third derivative component or change a multimedia content.
10. An interactive learning device, comprising: A storage device for storing code; The processor, coupled to the storage and configured to load the code for execution, includes: identifying a target object in an input image; performing a pixel position transformation on the target object in the input image to generate a transformed object image, wherein the pixel position transformation maps the value corresponding to a first pixel coordinate of the target object in the input image to a second pixel coordinate, and the pixel position transformation includes image rotation, the target object corresponding to a first reference axis in the input image, the target object corresponding to a second reference axis in the transformed object image, and the first reference axis being different from the second reference axis; and inputting the transformed object image to an artificial intelligence model and outputting a derivative object through the artificial intelligence model, wherein the artificial intelligence model is trained on a dataset and learns to map input data to corresponding derivative objects, the target object in the transformed object image and the derivative object corresponding to at least one of contour, shape, or pattern, and the type of the target object being different from the type of the derivative object.
11. The interactive learning apparatus of claim 10, wherein the pixel position transformation includes image segmentation, and the transformed object image is a portion of the input image corresponding to the target object.
12. The interactive learning apparatus as claimed in claim 10, wherein the processor further performs: replacing the target object in the transformed object image with the derivative object.
13. The interactive learning apparatus of claim 10, wherein the processor further performs: performing a foreground-background separation on the input image and capturing a foreground image; and performing an object recognition on the foreground image based on an object size to identify the target object, wherein the object size of the target object is pre-stored or learned.
14. The interactive learning device as claimed in claim 10, wherein the processor further performs: generating guidance text information based on the target object or the derivative object; and providing the guidance text information.
15. The interactive learning device as claimed in claim 10, wherein the processor further performs: retrieving a recommendation that matches the derivative component, wherein the recommendation is related to a product, an article, or multimedia content.
16. The interactive learning device of claim 10, wherein the processor further performs: generating generation condition information based on an input trajectory information, wherein the input trajectory information includes the position of at least one trajectory point and / or a path formed based on the at least one trajectory point; and outputting a second derivative component through the artificial intelligence model based on the generation condition information.
17. The interactive learning device as claimed in claim 16, wherein the generation condition information includes at least one of the following: object type parameters; object movement parameters; or user preference parameters.
18. The interactive learning device as claimed in claim 16, wherein the processor further performs: generating change information based on the input image and a user voice data; and generating a third derivative component or changing multimedia content based on the change information.
Citation Information
Patent Citations
Image generation method, device and system
CN113706370A
Image generation model training method and device, electronic equipment and storage medium
CN115861462A
Image generation method and device, medium, computing equipment and program product
CN118469797A
