Application method and device of digital textbook, equipment, storage medium and computer program product

Abstract

The invention discloses a digital textbook application method and device, equipment, a storage medium and a computer program product, and relates to the technical field of AR, and the method comprises the steps: receiving a target picture sent by a client, the target picture being obtained by the client through AR recognition; performing feature recognition on the target picture to obtain a feature recognition result of the target picture; and obtaining a target resource corresponding to the target picture based on the feature recognition result, sending the target resource to the client, and providing the target resource to a target user by the client. According to the invention, based on the scanned target picture, picture recognition is carried out through the AR technology to obtain the target resource, and the use flexibility of the digital textbook can be improved.

Description

Technical Field

[0001] This application relates to the field of AR technology, and particularly to an application method, device, equipment, storage medium, and computer program product for digital textbooks. Background Art

[0002] Currently, the development of digital textbooks has shown a diverse trend, but there are certain limitations in the two main existing forms.

[0003] The first type of digital textbook still relies on traditional paper textbooks and uses QR codes as a bridge to guide readers to relevant digital resources. Most of these resources are mainly audio and video, with a relatively single form, making it difficult to fully display the uniqueness and richness of different courses. In addition, the resources linked by QR codes often lack flexibility, are inconvenient for timely updating and adjustment, and are also difficult to effectively record the learning progress and achievements of learners, which poses challenges to personalized learning and teaching effect evaluation.

[0004] Another type of digital textbook completely abandons the form of paper textbooks and relies entirely on a specific APP for learning. The advantage of this form is that it can record detailed data of learners and provide strong support for teaching evaluation and learning analysis. However, its popularization and application are restricted to a certain extent. On the one hand, learners need to download and install a specific APP, which undoubtedly increases the usage threshold; on the other hand, the compatibility and user experience differences between different APPs may also affect learners' enthusiasm and learning effects.

[0005] The above content is only used to assist in understanding the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0006] The main purpose of this application is to provide an application method, device, equipment, storage medium, and computer program product for digital textbooks, aiming to solve the technical problem of the low flexibility in the use of existing digital textbooks.

[0007] To achieve the above object, this application proposes an application method for digital textbooks. The method is applied to a management platform and includes:

[0008] Receiving a target image sent by a client, where the target image is obtained by the client through augmented reality (AR) recognition;

[0009] Performing feature recognition on the target image to obtain a feature recognition result of the target image;

[0010] Based on the feature recognition result, obtain the target resource corresponding to the target picture, send the target resource to the client, and the client provides the target resource to the target user.

[0011] In one embodiment, the step of performing feature recognition on the target picture to obtain the feature recognition result of the target picture includes:

[0012] Preprocess the target picture;

[0013] Extract features from the preprocessed target picture to obtain a feature extraction result;

[0014] Based on a preset feature label, identify and classify the feature extraction result to obtain a feature recognition result.

[0015] In one embodiment, the step of obtaining the target resource corresponding to the target picture based on the feature recognition result includes:

[0016] Compare the feature recognition result with a preset resource mapping table to obtain a resource mapping result of the feature recognition result;

[0017] Based on the resource mapping result, obtain the target resource corresponding to the target picture from a preset resource database.

[0018] In one embodiment, the method is applied to a client, and the method includes:

[0019] Obtain a target picture through augmented reality (AR) recognition;

[0020] Send the target picture to a management platform, so that the management platform performs feature recognition on the target picture to obtain a feature recognition result, obtains the target resource of the target picture based on the feature recognition result, and sends the target resource to the client;

[0021] Receive the target resource sent by the management platform and provide the target resource to the target user.

[0022] In one embodiment, when the target resource in the step of providing the target resource to the target user is a virtual simulation resource, after the step of providing the target resource to the target user, the following steps are further included:

[0023] Capture the interaction operation of the target user in the virtual simulation scenario;

[0024] Analyze the interaction operation to obtain a target recognition result of the interaction operation;

[0025] Based on the target recognition result, generate a text label corresponding to the target recognition result;

[0026] Based on the text tags and a preset event mechanism, update the text tags in the virtual simulation scenario.

[0027] In one embodiment, the method further includes:

[0028] Generate a data progress record of the target user according to the text tags in the virtual simulation scenario, and send the data progress record to the management platform.

[0029] In addition, to achieve the above object, the present application further provides an application device for a digital textbook, and the application device for the digital textbook includes:

[0030] An image receiving module, configured to receive a target picture sent by a client, where the target picture is obtained by the client through augmented reality (AR) recognition;

[0031] A feature recognition module, configured to perform feature recognition on the target picture to obtain a feature recognition result of the target picture;

[0032] A resource acquisition module, configured to acquire a target resource corresponding to the target picture based on the feature recognition result, and send the target resource to the client, and the client provides the target resource to a target user.

[0033] In addition, to achieve the above object, the present application further provides an application device for a digital textbook, and the device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, where the computer program is configured to implement the steps of the application method for the digital textbook as described above.

[0034] In addition, to achieve the above object, the present application further provides a storage medium, where the storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium, and when the computer program is executed by a processor, the steps of the application method for the digital textbook as described above are implemented.

[0035] In addition, to achieve the above object, the present application further provides a computer program product, where the computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the application method for the digital textbook as described above are implemented.

[0036] One or more technical solutions proposed by the present application have at least the following technical effects:

[0037] The present application discloses an application method, device, equipment, storage medium and computer program product for digital textbooks, including: receiving a target picture sent by a client, where the target picture is obtained by the client through augmented reality (AR) recognition; performing feature recognition on the target picture to obtain a feature recognition result of the target picture; based on the feature recognition result, obtaining a target resource corresponding to the target picture, and sending the target resource to the client, and the client provides the target resource to a target user. Based on the scanned target picture, the present application performs picture recognition and generates an AR scene through AR technology, which can improve the flexibility of using digital textbooks. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present application and used together with the specification to explain the principles of the present application.

[0039] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0040] Figure 1 It is a schematic flowchart provided for the first embodiment of the application method of the digital textbook of the present application;

[0041] Figure 2 It is a schematic module structure diagram of the application device of the digital textbook in the embodiment of the present application;

[0042] Figure 3 It is a schematic device structure diagram of the hardware operating environment involved in the application method of the digital textbook in the embodiment of the present application.

[0043] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the drawings. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0044] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0045] To better understand the technical solutions of the present application, the following will be described in detail in combination with the drawings in the specification and the specific embodiments.

[0046] The main solution of the embodiment of the present application is as follows: The method is applied to a management platform, and the method includes: receiving a target picture sent by a client, where the target picture is obtained by the client through augmented reality (AR) recognition; performing feature recognition on the target picture to obtain a feature recognition result of the target picture; based on the feature recognition result, obtaining a target resource corresponding to the target picture, and sending the target resource to the client, and the client provides the target resource to a target user. The method is applied to the client, and the method includes: obtaining a target picture through AR recognition; sending the target picture to the management platform, so that the management platform performs feature recognition on the target picture to obtain a feature recognition result, obtains a target resource of the target picture based on the feature recognition result, and sends the target resource to the client; receiving the target resource sent by the management platform and providing the target resource to the target user.

[0047] In this embodiment, for the convenience of description, the following takes the application device for recognizing digital textbooks as the execution subject for elaboration.

[0048] Currently, the development of digital textbooks has shown a diverse trend, but there are certain limitations in the two main existing forms.

[0049] The first type of digital textbook still relies on traditional paper textbooks and uses two-dimensional codes as a bridge to guide readers to relevant digital resources. Most of these resources are mainly audio and video, with a relatively single form, and it is difficult to fully display the uniqueness and richness of different courses. In addition, the resources linked by two-dimensional codes often lack flexibility, are inconvenient for timely updating and adjustment, and it is also difficult to effectively record the learning progress and achievements of learners, which poses challenges to personalized learning and teaching effect evaluation.

[0050] Another type of digital textbook completely abandons the form of paper textbooks and completely relies on a specific APP for learning. The advantage of this form is that it can record detailed data of learners and provides strong support for teaching evaluation and learning analysis. However, its popularization and application are restricted to a certain extent. On the one hand, learners need to download and install a specific APP, which undoubtedly increases the usage threshold; on the other hand, the compatibility and user experience differences between different APPs may also affect the enthusiasm and learning effect of learners.

[0051] The present application provides a solution. Based on the scanned target picture, target resources can be obtained through AR technology for picture recognition, which can improve the usage flexibility of digital textbooks.

[0052] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication, and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, an application device of a digital textbook, etc. that can implement the above functions. Hereinafter, taking the application device of the digital textbook as an example, this embodiment and the following embodiments will be described.

[0053] Based on this, the embodiment of the present application provides a method for applying a digital textbook. Refer to Figure 1 , Figure 1 which is a schematic flowchart of the first embodiment of the method for applying the digital textbook of the present application.

[0054] In this embodiment, the method for applying the digital textbook is applied to a management platform, and the method for applying the digital textbook includes steps S11 to S13:

[0055] Step S11, receiving a target picture sent by a client, where the target picture is obtained by the client through augmented reality (AR) recognition.

[0056] It should be noted that the management platform is an integrated software system for storing, managing, and real-time updating digital resources. The platform not only has resource management functions, such as uploading, classifying, retrieving, and deleting digital resources, but also has data analysis functions, and can record and analyze students' learning data. These data include but are not limited to the frequency, duration, completion degree, and interaction situation of students accessing resources. The management platform helps teachers understand students' learning progress and mastery by providing personalized teaching feedback, so as to adjust teaching strategies. The management platform is usually deployed on a server and can be accessed through the Internet.

[0057] In addition, it should be noted that the client is a portable electronic device with image recognition function, capable of running a specific application program and connecting to the Internet. These devices include but are not limited to intelligent devices such as smart phones and tablet computers. The intelligent device captures and recognizes the target picture in the textbook through the built-in camera, and uses AR (augmented reality) picture recognition technology to associate these pictures with the digital resources pre-stored in the cloud or local server, thereby triggering and displaying these digital resources.

[0058] In addition, it should be noted that the target picture is used as a sign for AR recognition, combined with AR (augmented reality) picture recognition technology, to become a bridge connecting the paper textbook and digital resources. When the user scans these pictures through the intelligent device, it can quickly trigger the rich digital resources associated with them, such as audio explanations, video demonstrations, interactive games, and virtual simulations, thus greatly expanding the interactivity and functionality of the textbook.

[0059] Specifically, the management platform continuously monitors network requests from the client. When receiving a data packet sent by the client, it parses the data packet to extract the target picture, performs integrity verification on the received picture data to ensure that the picture data is not damaged, and performs format conversion on the pictures that need format conversion into a data picture format that the management platform can perform feature recognition on.

[0060] Step S12: Perform feature recognition on the target picture to obtain the feature recognition result of the target picture.

[0061] Specifically, preprocess the target picture; perform feature extraction on the preprocessed target picture to obtain a feature extraction result; perform recognition and classification on the feature extraction result based on a preset feature label to obtain a feature recognition result.

[0062] Step S13: Based on the feature recognition result, obtain the target resource corresponding to the target picture, send the target resource to the client, and the client provides the target resource to the target user.

[0063] It should be noted that the target resources refer to a series of learning materials and educational contents that exist in digital form and can be accessed through intelligent devices. These resources are closely designed around the knowledge points and themes in the textbooks and are presented in multimedia forms such as audio, video, interactive games, virtual simulations, etc., enriching the learning experience of students and enhancing the learning effect. Corresponding displays are made according to the types of resources. For example, when the resource is audio or video, the audio or video is played; when the resource is in the form of an interactive game, the game interface is loaded for display; when the resource is a virtual simulation resource, it is provided to the target user in the form of a virtual simulation model, allowing the user to experience a three-dimensional immersive experience. The target resources are uploaded by teachers through the teacher port in the background of the management platform. Teachers can also update the target resources through the teacher port.

[0064] Specifically, compare the feature recognition result with a preset resource mapping table to obtain the resource mapping result of the feature recognition result; based on the resource mapping result, obtain the target resource corresponding to the target picture from a preset resource database. Send the target resource to the client. After receiving the target resource, the client provides the target resource to the target user in its corresponding form, and the forms of the target resource include audio, video, interactive games, virtual simulations, etc.

[0065] In this embodiment, through the above solution, the target image sent by the client is received, and the target image is obtained by the client through augmented reality (AR) recognition; the target image is subjected to feature recognition to obtain the feature recognition result of the target image; based on the feature recognition result, the target resource corresponding to the target image is obtained, and the target resource is sent to the client, and the client provides the target resource to the target user. Based on the scanned target image, this application uses AR technology to identify images and then trigger target resources, which can improve the flexibility of using digital textbooks; and the forms of target resources are diverse, turning textbooks from static text and pictures into a dynamic interactive learning platform, enhancing the user experience; at the same time, the carrier of AR image recognition is the picture in the textbook, so traditional textbooks that have been compiled and printed can be digitally transformed.

[0066] Based on the above implementation solution, in a feasible implementation manner, the step of performing feature recognition on the target image to obtain the feature recognition result of the target image includes S21 to S23:

[0067] Step S21, preprocess the target image.

[0068] Specifically, after the management platform receives the target image, it preprocesses the target image. The preprocessing of the image mainly includes image adjustment, image enhancement, and denoising processing. Image adjustment includes adjusting the size, resolution, and color balance of the image to ensure that the image quality meets the recognition requirements. Image enhancement refers to enhancing the image, such as sharpening, contrast adjustment, etc., to improve the clarity and recognition effect of the image. Denoising processing refers to using filtering algorithms (such as Gaussian filtering, median filtering, etc.) to remove noise and interference factors in the image, which can maintain the smoothness and coherence of the image and reduce the possibility of misrecognition. According to the specific actual situation, the preprocessing also includes performing color space conversion on the target image, converting the image from the RGB color space to the grayscale space or other suitable color spaces to reduce the calculation amount and meet the requirements of the AR recognition model.

[0069] Step S22, perform feature extraction on the preprocessed target image to obtain the feature extraction result.

[0070] Specifically, according to the characteristics and recognition requirements of the target image, a suitable feature extraction algorithm is selected. Common feature extraction algorithms include SIFT (Scale-Invariant Feature Transform), SURF (Speeded-Up Robust Features), ORB (Oriented FAST and Rotated BRIEF), etc. The selected feature extraction algorithm is used to process the preprocessed image. Key points (i.e., feature points) and descriptors (i.e., feature vectors) in the image are extracted as the feature extraction result.

[0071] Furthermore, the paper textbook pictures mainly contain elements such as text, charts, and a small number of pictures. In an embodiment of the present application, the SURF (Speeded Up Robust Features) algorithm is selected as the feature extraction algorithm. The SURF algorithm is a feature point detection algorithm based on the Hessian matrix. It detects feature points by performing Gaussian blur processing on the image and calculating the determinant value of the Hessian matrix at different scales. Then, the feature vectors are calculated for the regions around each feature point for subsequent matching and recognition. The SURF algorithm has high robustness and computational efficiency and is suitable for feature extraction of book textbook pictures.

[0072] More specifically, the preprocessed image is subjected to feature extraction through the SURF (Speeded Up Robust Features) algorithm. First, feature points are detected in the image, and these feature points are usually corner points, edge points, or regions with significant texture in the image. Then, for each detected feature point, the feature vector around it is calculated; the feature vector is usually a multi-dimensional numerical vector used to describe the local texture information of the feature point. In the SURF algorithm, the key points and descriptors are closely related. The key points refer to the positions of the feature points in the image, while the descriptors are the mathematical representations of the texture information of the regions around these feature points. The feature extraction result is the key points and feature vectors.

[0073] Finally, the extracted feature vectors are encoded for subsequent matching and recognition. The encoding method can adopt binary encoding, floating-point encoding, etc. In this embodiment, the floating-point encoding method is adopted, and each feature vector is represented as a multi-dimensional floating-point array; the encoded feature vectors are stored in the database for subsequent use in matching and recognition. When storing, a unique picture ID is assigned to each target picture, and the feature vectors are associated with the corresponding image IDs so that the corresponding feature vectors can be quickly retrieved when needed.

[0074] Step S23, based on the preset feature tags, identify and classify the feature extraction result to obtain the feature recognition result.

[0075] It should be noted that the preset feature tags refer to a set of feature tags predefined according to the actual application scenario and training data set for identification and classification. The feature tags usually represent specific features, object categories, or attributes in the image. The feature tags include but are not limited to feature categories and tag names. In an embodiment of the present application, the feature tags represent the identification pictures for AR picture recognition in the textbook. A set of feature tags represents a picture in the textbook. The textbook pictures selected as feature tags are representative pictures that not only have visual attractiveness but also can accurately reflect the core content and characteristics of the course.

[0076] Specifically, according to specific application requirements and training data, a feature label library containing various types of feature labels is constructed. Each group of labels in the feature label library corresponds to one or a category of features. The obtained feature extraction results are matched with the feature labels in the feature label library, and the matching degree between the feature extraction results and the feature labels is calculated through a matching algorithm, such as calculating the distance or similarity between the two. According to the matching results, the feature extraction results are classified under the group of feature labels with the best match. A preset determination threshold is set. If the similarity is higher than the preset determination threshold, it is considered that the recognition is successful, and this feature label is used as the feature recognition result.

[0077] Based on the above implementation solutions, in a feasible implementation manner, the step of obtaining the target resource corresponding to the target picture based on the feature recognition result includes S31 - S32:

[0078] Step S31: Compare the feature recognition result with a preset resource mapping table to obtain the resource mapping result of the feature recognition result.

[0079] It should be noted that the resource mapping table is used to store the correspondence between the feature recognition results (feature labels) and the target resources. This correspondence enables the system to quickly locate and obtain the corresponding target resources according to the feature recognition results. The resource mapping table includes, but is not limited to, feature labels and resource identifiers. The resource identifier is used to uniquely identify the target resources stored in the resource database.

[0080] Additionally, it should be noted that the resource mapping result is the matching information about the target resources corresponding to the features of the target picture obtained after comparing the feature recognition result with the preset resource mapping table.

[0081] Specifically, traverse and compare the feature recognition result of the target picture with the preset resource mapping table to find the resource mapping result that matches the features of the target picture.

[0082] Step S32: Based on the resource mapping result, obtain the target resource corresponding to the target picture from the preset resource database.

[0083] It should be noted that the resource database is a storage system that stores various types of digital teaching resources. The resources can be images, audio, video, documents, or any other form of data files, such as interactive games, virtual simulations, etc.

[0084] Specifically, according to the resource mapping result, retrieve and obtain the target resource corresponding to the target picture from the resource database based on the mapping relationship in the resource mapping table.

[0085] In this embodiment, through the above solution, by comparing the feature recognition result with the preset resource mapping table, the resource mapping result corresponding to the target picture can be quickly located, and then the target resource can be obtained from the resource database, significantly improving the data processing efficiency.

[0086] Based on the above implementation solution, in a feasible implementation manner, the method is applied to a client, and the method includes steps S41 to S43:

[0087] Step S41, obtaining a target picture through augmented reality (AR) recognition.

[0088] It should be noted that augmented reality (AR) is a technology that superimposes virtual information such as computer-generated images, sounds, videos, and haptics onto the real-world environment. In this application, the client uses AR image recognition technology to scan and identify the target picture in the textbook. By comparing the captured image with the images in the database, the client can determine which picture the user has scanned, and then trigger the corresponding target resource.

[0089] Specifically, start the AR application on the client, initialize the AR environment, and use the camera function of the AR application to capture the target picture in the textbook in real time.

[0090] Step S42, sending the target picture to a management platform so that the management platform performs feature recognition on the target picture to obtain a feature recognition result, obtains the target resource of the target picture based on the feature recognition result, and sends the target resource to the client.

[0091] Specifically, send the obtained target picture to the management platform. After receiving the target picture, the management platform first performs preprocessing such as image adjustment, image enhancement, and denoising on the image. Then, a feature extraction algorithm is used to extract features from the preprocessed image to obtain the feature points and feature vectors of the target picture, that is, the feature extraction result of the target picture. Then, the obtained feature extraction result is matched with the feature tags in the feature tag library, and the matching degree between the feature extraction result and the feature tags is calculated through a matching algorithm. According to the matching degree, the feature extraction result is classified under the most matching group of feature tags, and this feature tag is used as the feature recognition result. Finally, the feature recognition result of the target picture and the preset resource mapping table are traversed and compared to find the resource mapping result that matches the features of the target picture. According to the resource mapping result, the target resource corresponding to the target picture is retrieved and obtained from the resource database based on the mapping relationship in the resource mapping table, and the target resource is sent to the client.

[0092] Step S43, receiving the target resource sent by the management platform and providing the target resource to the target user.

[0093] Specifically, the client receives the target resource from the management platform, decodes and processes the target resource data, and prepares for resource display. In an embodiment of the present application, when the target resource is a virtual simulation resource, the target resource is provided to the user through a virtual simulation scenario.

[0094] Furthermore, in an embodiment of the present application, a small program is used as the client. The lightweight, installation-free, and highly flexible characteristics of the small program greatly improve its convenience.

[0095] In this embodiment, through the above solution, the target picture is obtained by augmented reality (AR) recognition; the target picture is sent to the management platform so that the management platform performs feature recognition on the target picture to obtain a feature recognition result, and based on the feature recognition result, the target resource of the target picture is obtained, and the target resource is sent to the client; the target resource sent by the management platform is received, and the target resource is provided to the target user. Through the AR technology, the present application can identify pictures and trigger target resources, which can improve the flexibility of using digital textbooks; and the forms of target resources are diverse, turning the textbook from static text and pictures into a dynamic interactive learning platform, enhancing the user experience.

[0096] Based on the above implementation scheme, in a feasible implementation manner, when the target resource in the step of providing the target resource to the target user is a virtual simulation resource, after the step of providing the target resource to the target user, the following steps S51 to S54 are further included:

[0097] Step S51, capturing the interaction operations of the target user in the virtual simulation scenario.

[0098] It should be noted that the interaction operation refers to various actions or instructions performed by the target user in the virtual simulation scenario through input devices (such as touch screens, gesture recognition systems, voice recognition systems, etc.), and these actions or instructions are intended to interact with the elements in the virtual simulation scenario. Interaction operations may include, but are not limited to, clicking, swiping, voice commands, gesture recognition, etc.

[0099] Specifically, an interaction capture module is integrated in the client to monitor the user's operations in real time; necessary permissions and settings are configured to allow the application to access the user's input devices (such as touch screens, cameras, microphones, etc.); the interaction capture module is used to monitor the user's operations in real time, such as clicking, swiping, voice commands, etc. When the user interacts with the virtual simulation scenario (such as clicking, dragging, inputting, etc.), the capture mechanism will collect the data of these operations, including the operation type, timestamp, location information, etc.

[0100] Step S52: Parse the interaction operation to obtain the target recognition result of the interaction operation.

[0101] It should be noted that the target recognition result refers to the result obtained after the client parses the interaction operation performed by the target user in the virtual simulation scenario. This result is obtained by analyzing and matching the characteristics of the interaction operation based on a preset recognition algorithm or model, and it represents the specific target of the user's intention or operation. The target recognition result may be a specific object, location, action, or instruction, etc.

[0102] Specifically, perform preprocessing such as denoising and smoothing on the captured interaction operation data, perform target recognition on the preprocessed data to determine the specific object or area targeted by the user operation, such as clicking a button or dragging an object, extract the characteristics of the operation data as the target recognition result, for example, the object, location, action, or instruction of the interaction operation, etc. After the target recognition is completed, output the recognition result in a structured form, including information such as the type, location, and status of the recognized object.

[0103] Step S53: Generate a text label corresponding to the target recognition result based on the target recognition result.

[0104] It should be noted that the text label refers to a short text description used to identify or describe specific elements or objects in the virtual simulation scenario. These labels are usually generated based on the target recognition result and are used to provide users with information about the elements or objects in the virtual simulation scenario. The text label can include the name, description, status, prompt information, etc.

[0105] Specifically, pre-define a set of mapping rules to map the target recognition result to the corresponding text label; generate the corresponding text label according to the target recognition result and the mapping rules.

[0106] Step S54: Update the text label in the virtual simulation scenario based on the text label and the preset event mechanism.

[0107] It should be noted that the preset event mechanism refers to a set of rules and processes defined in the virtual simulation scenario for handling events triggered by user interaction operations. These events can be click events, swipe events, voice command events, etc. The event mechanism includes links such as event capture, parsing, processing, and feedback. It determines how the client responds to the user's interaction operation and updates the elements or states in the virtual simulation scenario according to the preset rules. The event mechanism is usually associated with text labels and other UI elements to achieve a rich interaction experience.

[0108] Specifically, a set of event mechanisms are defined in the virtual simulation scenario to handle various possible user interaction events. These events may include user clicks, drags, inputs, etc., as well as state changes triggered by these operations. When a user's interaction operation triggers an event, corresponding response operations are executed according to the preset event mechanism. In this step, the key is to combine the generated text labels with the event mechanism to ensure that the text labels in the virtual simulation scenario can be correctly updated when the event occurs. During the event response process, according to the generated text labels, the corresponding elements in the virtual simulation scenario are updated. For example, the text display on the interface is modified, and the status indicator is updated, etc.

[0109] Through the above solution, in this embodiment, by obtaining digital teaching resources in the virtual simulation scenario, the immersion and interactivity of the user are enhanced; the text labels in the virtual simulation scenario are dynamically updated according to the user's interaction operations, making the virtual simulation scenario more vivid and interesting, and capable of better reflecting the user's operation intention and results. At the same time, this dynamic nature not only improves the attractiveness of the virtual simulation scenario but also helps to maintain the user's interest and participation.

[0110] Based on the above implementation solution, in a feasible implementation manner, the method further includes step S61:

[0111] Step S61, generate a data progress record of the target user according to the text labels in the virtual simulation scenario, and send the data progress record to the management platform.

[0112] Specifically, after each interaction operation, the client obtains the latest text labels in the virtual simulation scenario. The latest text labels reflect the user's current interaction state. Compare the latest text labels with the preset event mechanism to generate a data progress record. The progress record represents the user's learning progress in the current target resource. Send the data progress record to the management platform.

[0113] Furthermore, the data progress of digital media resources such as audio and video played on the client, as well as interactive games and virtual simulation resources, will be synchronously sent to the management platform. The management platform will count the learning records of student users, such as learning time, learning progress, learning courses, etc., and provide personalized teaching feedback for teacher users.

[0114] Through the above solution, in this embodiment, by synchronizing the data progress record to the management platform for statistics by the management platform, it helps teachers to immediately understand the learning progress and status of students, formulate teaching plans that better meet the needs of students, provide personalized teaching feedback, and thus improve the teaching effect.

[0115] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the application method of the digital textbook of this application. Based on this technical concept, more forms of simple transformation are within the protection scope of this application.

[0116] This application also provides an application device for a digital textbook. Please refer to Figure 2 , the application device for the digital textbook includes:

[0117] An image receiving module 201, configured to receive a target picture sent by a client, where the target picture is obtained by the client through augmented reality (AR) recognition;

[0118] A feature recognition module 202, configured to perform feature recognition on the target picture to obtain a feature recognition result of the target picture;

[0119] A resource acquisition module 203, configured to acquire a target resource corresponding to the target picture based on the feature recognition result, send the target resource to the client, and the client provides the target resource to a target user.

[0120] The application device for the digital textbook provided by this application adopts the application method of the digital textbook in the above embodiment, and can solve the technical problem of low flexibility in using existing digital textbooks. Compared with the prior art, the beneficial effects of the application device for the digital textbook provided by this application are the same as those of the application method of the digital textbook provided in the above embodiment, and other technical features in the application device for the digital textbook are the same as those disclosed in the method of the above embodiment, and will not be elaborated here.

[0121] This application provides an application device for a digital textbook. The application device for the digital textbook includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the application method of the digital textbook in the first embodiment above.

[0122] Next, refer to Figure 3, which shows a schematic structural diagram of an application device suitable for implementing the digital teaching materials of the embodiments of the present application. The application devices of the digital teaching materials in the embodiments of the present application may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistant), PADs (Portable Application Description: tablet computers), PMPs (Portable Media Player: portable multimedia players), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3 The application device of the digital teaching materials shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present application.

[0123] As Figure 3 shown, the application device of the digital teaching materials may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which may perform various appropriate actions and processes according to the program stored in the read-only memory (ROM: Read Only Memory) 1002 or the program loaded from the storage device 1003 into the random access memory (RAM: Random Access Memory) 1004. In the RAM 1004, various programs and data required for the operation of the application device of the digital teaching materials are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. The input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems may be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the application device of the digital teaching materials to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows an application device of the digital teaching materials having various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or had alternatively.

[0124] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by a processing device 1001, the above functions defined in the methods of the embodiments disclosed in the present application are executed.

[0125] The application device of the digital textbook provided by the present application adopts the application method of the digital textbook in the above embodiments, and can solve the technical problem of low flexibility in using existing digital textbooks. Compared with the prior art, the beneficial effects of the application device of the digital textbook provided by the present application are the same as those of the application method of the digital textbook provided in the above embodiments, and other technical features in the application device of the digital textbook are the same as the features disclosed in the method of the previous embodiment, and will not be elaborated here.

[0126] It should be understood that each part disclosed in the present application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0127] As described above, the above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, and all of them should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0128] The present application provides a computer-readable storage medium, having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the application method of the digital textbook in the above embodiments.

[0129] The computer-readable storage medium provided by this application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. The program code contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0130] The above computer-readable storage medium can be included in the application device of the digital textbook; or it can exist separately and not be assembled into the application device of the digital textbook.

[0131] The above computer-readable storage medium carries one or more programs. When the one or more programs are executed by the application device of the digital textbook, the application device of the digital textbook is caused to: receive a target picture sent by a client, where the target picture is obtained by the client through augmented reality (AR) recognition; perform feature recognition on the target picture to obtain a feature recognition result of the target picture; based on the feature recognition result, obtain a target resource corresponding to the target picture, and send the target resource to the client, and the client provides the target resource to the target user.

[0132] Computer program code for performing the operations of this application can be written in one or more programming languages or combinations thereof. The above-mentioned programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0133] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of the code, and this module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutively represented blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0134] The modules described in the embodiments of this application can be implemented in software or in hardware. Among them, the name of the module does not constitute a limitation on the unit itself in some cases.

[0135] The readable storage medium provided by this application is a computer-readable storage medium. The computer-readable storage medium stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned application method of the digital textbook, and can solve the technical problem of the low flexibility in using existing digital textbooks. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by this application are the same as those of the application method of the digital textbook provided in the above embodiments, and will not be elaborated here.

[0136] The present application also provides a computer program product, including a computer program, which when executed by a processor implements the steps of the application method of the digital textbook as described above.

[0137] The computer program product provided by the present application can solve the technical problem of low flexibility in the use of existing digital textbooks. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as those of the application method of the digital textbook provided in the above embodiments, and will not be elaborated herein.

[0138] The above are only some embodiments of the present application, and thus do not limit the patent scope of the present application. Any equivalent structural transformation made under the technical concept of the present application by using the content of the specification and drawings of the present application, or any direct / indirect application in other related technical fields is included in the patent protection scope of the present application.

Claims

1. A method for applying a digital textbook, characterized in that, The method is applied to a management platform, and the method includes: Receiving a target picture sent by a client, where the target picture is obtained by the client through augmented reality (AR) recognition; Performing feature recognition on the target picture to obtain a feature recognition result of the target picture; Based on the feature recognition result, obtaining a target resource corresponding to the target picture, and sending the target resource to the client, where the client provides the target resource to a target user.

2. The method according to claim 1, wherein The step of performing feature recognition on the target picture to obtain a feature recognition result of the target picture includes: Performing preprocessing on the target picture; Performing feature extraction on the target picture after the preprocessing is completed to obtain a feature extraction result; Based on a preset feature label, performing recognition and classification on the feature extraction result to obtain a feature recognition result.

3. The method according to claim 1, characterized in that The step of obtaining a target resource corresponding to the target picture based on the feature recognition result includes: Comparing the feature recognition result with a preset resource mapping table to obtain a resource mapping result of the feature recognition result; Based on the resource mapping result, obtaining the target resource corresponding to the target picture from a preset resource database.

4. A method for applying a digital teaching material, characterized in that, The method is applied to a client, and the method includes: Obtaining a target picture through augmented reality (AR) recognition; Sending the target picture to the management platform so that the management platform performs feature recognition on the target picture to obtain a feature recognition result, obtains the target resource of the target picture based on the feature recognition result, and sends the target resource to the client; Receiving the target resource sent by the management platform and providing the target resource to the target user.

5. The method according to claim 4, characterized in that, When the target resource in the step of providing the target resource to the target user is a virtual simulation resource, after the step of providing the target resource to the target user, the method further includes: Capturing an interaction operation of the target user in the virtual simulation scenario; Analyzing the interaction operation to obtain a target recognition result of the interaction operation; Based on the target recognition result, generating a text label corresponding to the target recognition result; Based on the text label and a preset event mechanism, updating the text label in the virtual simulation scenario.

6. The method according to claim 5, characterized in that The method further includes: Generating a data progress record of the target user according to the text label in the virtual simulation scenario, and sending the data progress record to the management platform.

7. An application device for a digital teaching material, characterized in that, The device includes: An image receiving module, configured to receive a target picture sent by a client, where the target picture is obtained by the client through augmented reality (AR) recognition; A feature recognition module, configured to perform feature recognition on the target picture to obtain a feature recognition result of the target picture; A resource obtaining module, configured to obtain a target resource corresponding to the target picture based on the feature recognition result, send the target resource to the client, and the client provides the target resource to a target user.

8. An application device for a digital textbook, characterized in that, The device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the application method of the digital textbook as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the application method of the digital textbook as described in any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that, The computer program product includes a computer program. When the computer program is executed by a processor, the steps of the application method of the digital textbook as described in any one of claims 1 to 6 are implemented.

Images