A paper-screen interactive method based on learning cards and related products

Abstract

This application discloses a paper screen interaction method based on learning cards, comprising: performing grid layout detection on valid images of the baseboard using a preset grid layout detection model to construct a grid layout dataset, and comparing the layout with the grid layout datasets corresponding to adjacent previous valid images of the baseboard to determine the position of the newly added learning card and its corresponding grid; extracting a local image corresponding to the newly added learning card from the valid images of the baseboard, and performing content recognition on the local image corresponding to the newly added learning card using a preset card content recognition model, while associating and binding the content corresponding to the newly added learning card with the grid displayed on the screen to construct a learning card layout association dataset; and continuously removing learning cards based on the learning card layout association dataset, and determining the content matching of the continuously removed learning cards to generate corresponding interactive feedback. This solution improves card positioning stability through grid layout comparison, achieving accurate mapping between physical cards and screen grids, reducing computational load and improving interactive smoothness; combined with noise-resistant convolution and splicing judgment, it enhances the recognition effect of low-quality images, expands pattern categories without retraining the model, and comprehensively optimizes the accuracy and adaptability of paper-screen interactive learning.

Description

Technical Field

[0001] This application relates to the fields of educational informatization and human-computer interaction technology, specifically to a paper-screen interaction method based on learning cards and related products. Background Technology

[0002] In the basic education stage, English vocabulary learning is a core part of students' language accumulation. The paper-screen interactive learning method, which combines physical learning cards with electronic screens, has become a mainstream auxiliary form of vocabulary learning for young students due to its fun hands-on operation and low visual burden. This method strengthens the correspondence between words and real objects through gamified interaction, which can effectively improve students' learning participation.

[0003] Existing paper-screen interactive word learning solutions mostly acquire card images directly through image acquisition devices and then perform content recognition. They rely on traditional convolutional algorithms to extract image features and classify cards, using fixed multi-classification models to directly determine the category of the patterned cards. These solutions suffer from weak recognition stability and scenario adaptability, insufficient model expansion flexibility, and low efficiency in matching and coordinating physical cards with screen displays. This makes it difficult to consistently guarantee the recognition effect and interactive experience of paper-screen interaction, and ultimately fails to adequately meet the needs of real-world learning scenarios. Summary of the Invention

[0004] To address the aforementioned technical problems, this application provides a paper-screen interactive method and related products based on learning cards, thereby alleviating these issues.

[0005] The technical solutions provided in this application are as follows:

[0006] A paper-screen interaction method based on learning cards includes: performing grid layout detection on valid images of the baseboard using a preset grid layout detection model to construct a grid layout dataset, and comparing the layout with the grid layout datasets corresponding to adjacent valid images of the baseboard to determine the position of the newly added learning card and its corresponding grid; extracting a local image corresponding to the newly added learning card from the valid images of the baseboard, and performing content recognition on the local image corresponding to the newly added learning card using a preset card content recognition model, while associating and binding the content corresponding to the newly added learning card with the grid displayed on the screen to construct a learning card layout association dataset; and continuously removing learning cards based on the learning card layout association dataset, and determining the content matching of the continuously removed learning cards to generate corresponding interactive feedback.

[0007] Optionally, according to the preset grid layout rules, the learning cards are placed on a base plate with the same grid layout outside the screen, and a valid image of the base plate containing the base plate and the laid-out learning cards is obtained according to the preset rules.

[0008] Optionally, the effective images of the baseboard, including the baseboard and the laid-out learning cards, are obtained according to preset rules. This includes: continuously acquiring multiple frames of original baseboard images containing the baseboard and the laid-out learning cards, and performing standardized preprocessing on the multiple frames of original baseboard images to obtain multiple frames of standard baseboard images; scoring the image clarity of the multiple frames of standard baseboard images based on a preset clarity judgment model, and selecting the single frame of standard baseboard images with the highest clarity score as the effective baseboard images.

[0009] Optionally, the image sharpness of the multi-frame standard images of substrate placement is scored, and the single-frame standard image with the highest sharpness score is selected as the valid image for substrate placement according to the image sharpness score ranking. Then, the following steps are taken: performing standardization preprocessing on the single-frame standard image of substrate placement to form a single-frame standardized substrate placement image; and performing background removal and interference suppression processing on the single-frame standardized substrate placement image based on a preset target region extraction model to obtain a single-frame interference-free substrate image as the valid image for substrate placement.

[0010] Optionally, a grid layout detection model is used to detect the effective images of the base plate placement to construct a grid layout dataset. Specifically, this includes: based on the grid layout detection model, performing grid occupancy and grid position detection on the effective images of the base plate placement to generate grid occupancy and grid position data corresponding to each grid; and performing positional association and structured processing on the grid occupancy and grid position data corresponding to each grid according to the grid positional mapping mechanism to form a grid layout dataset.

[0011] Optionally, the preset card content recognition model includes a pattern card recognition structure layer and a text card recognition structure layer. Correspondingly, the preset card content recognition model is used to perform content recognition on the local image corresponding to the newly added learning card. Specifically, this includes: classifying the newly added learning card to determine whether it belongs to the pattern card type or the text card type; if the newly added learning card belongs to the pattern card type, then performing content recognition on the newly added learning card based on the pattern card recognition structure layer; if the newly added learning card belongs to the text card type, then performing content recognition on the newly added learning card based on the text card recognition structure layer.

[0012] Optionally, if the newly added learning card is a pattern card type, then the content recognition of the newly added learning card is performed based on the pattern card recognition structure layer, specifically including: splicing the newly added learning card with multiple standard pattern images to form multiple spliced ​​image pairs; and performing content consistency recognition on the multiple spliced ​​image pairs to determine the pattern content corresponding to the newly added learning card.

[0013] Optionally, content consistency recognition is performed on the multiple sets of spliced ​​image pairs to determine the pattern content corresponding to the newly added learning card. This includes: performing noise reduction preprocessing and image feature extraction on each set of spliced ​​image pairs based on a noise-resistant gradient convolution algorithm to obtain the image gradient and structural association features corresponding to each set of spliced ​​image pairs; performing content consistency recognition on each set of spliced ​​image pairs based on the extracted image gradient and structural association features corresponding to each set of spliced ​​image pairs to output the content consistency judgment result corresponding to each set of spliced ​​image pairs; and determining the standard pattern consistent with the content of the newly added learning card based on the content consistency judgment result to determine the pattern content corresponding to the newly added learning card.

[0014] Optionally, based on the learning card layout association dataset, learning cards are continuously removed, and the content matching of the continuously removed learning cards is determined to generate corresponding interactive feedback. This includes: obtaining valid images of the background removal according to preset rules; performing grid layout detection on the valid images of the background removal using a preset grid layout detection model to obtain a grid layout dataset containing grid position and grid occupancy information; comparing this dataset with the grid layout dataset corresponding to the adjacent previous valid images of the background removal to determine the removed learning cards; extracting the card content of the continuously removed learning cards based on the card content bound to the screen display grid corresponding to the removed learning cards; and generating corresponding interactive feedback based on the content matching of the continuously removed learning cards.

[0015] Optionally, obtaining valid images for substrate removal according to preset rules includes: continuously acquiring multiple frames of original substrate removal images, and performing standardized preprocessing on the multiple frames of original substrate removal images to obtain multiple frames of standard substrate removal images; based on a preset sharpness determination model, scoring the image sharpness of the multiple frames of standard substrate removal images, and selecting the single frame of standard substrate removal image with the highest sharpness score as a valid image for substrate removal according to the image sharpness score ranking.

[0016] A paper-screen interactive device based on learning cards includes: a determination module, which performs grid layout detection on valid images of a baseboard using a preset grid layout detection model to construct a grid layout dataset, and compares the layout with the grid layout datasets corresponding to adjacent valid images of the baseboard to determine the position of a new learning card and its corresponding grid; a construction module, which extracts a local image corresponding to the new learning card from the valid images of the baseboard, performs content recognition on the local image corresponding to the new learning card using a preset card content recognition model, and associates and binds the content corresponding to the new learning card with the grid displayed on the screen to construct a learning card layout association dataset; and an interactive feedback module, which continuously removes learning cards based on the learning card layout association dataset, determines the content matching of the continuously removed learning cards, and generates corresponding interactive feedback.

[0017] An electronic device includes: a memory for storing executable program code; and a processor for calling and running the executable program code from the memory, causing the electronic device to perform a paper screen interaction method based on learning cards as described in any of the preceding claims.

[0018] A computer program product or computer program comprising computer instructions that, when executed by a processor, implement a paper-screen interactive method based on learning cards as described in any of the preceding claims.

[0019] The technical solution provided in this application has the following technical advantages: This solution utilizes a pre-defined grid layout detection model to perform layout detection, dataset construction, and before-and-after sequence comparison. It first identifies card positions and layout changes, improving card positioning stability and reducing layout recognition errors, especially in scenarios with poor image quality or card misalignment, compared to directly recognizing card content. Based on this, it extracts local card images to complete content recognition and associates the card content with the screen display grid to construct a layout association dataset. This achieves precise mapping between physical cards and screen displays, making the correspondence between physical operations and screen feedback clearer, thereby improving the matching efficiency of paper-screen interaction. Simultaneously, relying on this layout... The joint dataset performs card removal and content matching verification without repeatedly performing full card content recognition, effectively reducing the computational load of the interactive process, improving the smoothness of the interactive flow, and alleviating interactive lag issues. In addition, the proposed solution combines grid layout comparison, type recognition, splicing consistency judgment, and noise-resistant gradient convolution, which not only enhances the recognition stability under low-quality images, but also eliminates the need to retrain the model due to the expansion of pattern card categories. This significantly improves the scenario adaptability and expansion flexibility of the solution, optimizing the overall user experience of paper screen interactive word learning in terms of recognition accuracy, interactive smoothness, and scenario applicability. Attached Figure Description

[0020] Figure 1 A flowchart of a paper screen interaction method based on learning cards in one embodiment provided in this application; Figure 2 A block diagram of a paper screen interactive device based on learning cards in one embodiment of this application; Figure 3 This is a structural block diagram of an electronic device provided in one embodiment of this application. Detailed Implementation

[0021] like Figure 1 As shown in the figure, this application provides a paper screen interaction method based on learning cards, including: performing grid layout detection on valid images of the baseboard using a preset grid layout detection model to construct a grid layout dataset, and comparing the layout with the grid layout datasets corresponding to adjacent previous valid images of the baseboard to determine the position of the newly added learning card and its corresponding grid; extracting a local image corresponding to the newly added learning card from the valid images of the baseboard, and performing content recognition on the local image corresponding to the newly added learning card using a preset card content recognition model, while associating and binding the content corresponding to the newly added learning card with the grid displayed on the screen to construct a learning card layout association dataset; and continuously removing learning cards based on the learning card layout association dataset, and determining the content matching of the continuously removed learning cards to generate corresponding interactive feedback.

[0022] Optionally, according to the preset grid layout rules, the learning cards are placed on a base plate with the same grid layout outside the screen, and a valid image of the base plate containing the base plate and the laid-out learning cards is obtained according to the preset rules.

[0023] This application acquires and standardizes images of learning cards placed on the outer base of the screen to obtain valid images that retain only the base and the learning cards. Based on the base corner point positioning and layered detection, grid area division, and point-by-point comparison of the occupancy status according to the physical sequence of the grid, the application locates the grid position of newly added learning cards on the base and completes the content parsing of the newly added learning cards. It establishes a one-to-one binding relationship between the content of the learning cards and the corresponding display grids on the screen to form complete layout association data. Based on this layout association data, the application tracks the grid position and verifies the content matching of learning cards that are continuously removed from the base. Based on the verification results, it generates corresponding screen interaction feedback. This approach differs from the traditional paper screen interaction method that only performs single image recognition and simple command feedback. Through the coordinated cooperation of base corner point positioning and layered detection, grid area division, point-by-point comparison of the occupancy status according to the physical sequence of the grid, content recognition, and binding verification, the application improves the stability of image parsing and the rationality of content matching during the placement and removal of learning cards.

[0024] Specifically, in this application, learning cards are placed one by one on a base plate outside the screen according to the grid layout rules displayed on the screen. The number of grids on the base plate is consistent with the number of grids displayed on the screen, so that each physical placement position on the base plate corresponds to each display position on the screen. Multiple frames of original images containing the base plate and the placed learning cards are continuously acquired. Size normalization and pixel value standardization are performed on these multiple frames of original images to obtain multiple standard images with regular pixel distribution. These multiple standard images are input into a designed sharpness determination model, which includes a shallow feature extraction layer and a sharpness classification layer. The shallow feature extraction layer extracts the pixel gradient features and texture distribution features of the standard images. The sharpness classification layer outputs the sharpness confidence score corresponding to each frame of the standard image based on the extracted features. The frame of the standard image with the highest sharpness confidence score is selected as the valid image for base plate placement. This valid image retains only the valid image areas of the base plate and learning cards, eliminating interference information from irrelevant background areas.

[0025] Specifically, in this application, the effective image of the baseboard placement is input into the designed grid layout detection model, which includes a baseboard corner point localization layer, a grid region segmentation layer, and a card placement detection layer. The baseboard corner point localization layer locates the coordinates of the four corner points of the baseboard in the effective image, and based on these coordinates, crops and separates the main body area of ​​the baseboard from the effective image, eliminating interference from redundant image edge information. The grid region segmentation layer divides the main body area of ​​the baseboard into multiple independent grid regions based on its size and layout rules, with each grid region corresponding to a physical placement position on the baseboard. The card placement detection layer detects the pixel features and texture gradient features within each grid region, determining whether a learning card is placed in each grid region, and generating placement status data and position coordinate data for each grid region. Following the arrangement order of the grid regions, the placement status data and position coordinate data of all grid regions are structurally integrated to form a grid layout dataset. The current grid layout dataset is compared point by point with the grid layout dataset corresponding to the previous frame of valid background placement image. The occupancy status change of each grid area is compared with the matching degree of position coordinates. Grid areas whose occupancy status changes from zero to one are selected and identified as the grid positions corresponding to the new learning cards, thus completing the localization of the new learning cards.

[0026] Specifically, in this application, based on the grid position coordinates corresponding to the newly added learning card, a partial image containing only the newly added learning card is cropped from the effective image of the base placement. This partial image retains the complete image features of the newly added learning card while eliminating image interference from surrounding grid areas and the background. The partial image of the newly added learning card is input into a designed card content recognition model, which includes a card classification layer and a hierarchical content analysis layer. The card classification layer detects the feature distribution of the partial image to determine whether the newly added learning card belongs to the pattern card type or the text card type. For partial images of pattern card types, the hierarchical content analysis layer concatenates the partial image with all standard pattern images within the level, forming multiple sets of concatenated image pairs. Each set of concatenated image pairs contains the partial image of the newly added learning card and one standard pattern image. For each pair of stitched images, subtraction gradient convolution is performed. This convolution process extracts image boundary features and texture gradient features by calculating the difference between local pixel values ​​and neighboring pixel values. This differs from traditional convolution, which directly weights pixel values, thus improving the stability of feature extraction in low-quality images. Based on the extracted boundary and texture gradient features, content consistency is compared for each pair of stitched images, and the content consistency confidence score for each pair is output. The pattern content corresponding to the standard pattern image with the highest content consistency confidence score is selected as the pattern content for the new learning card. For local images of text card types, a hierarchical content parsing layer extracts and matches the text character features of the local image character by character. The extracted text character sequence is compared with a preset text character library, and the text content corresponding to the new learning card is output.

[0027] Specifically, in this application, the effective image of the base plate is analyzed pixel by pixel to identify whether a new learning card is placed in each physical grid on the base plate. Based on image feature matching, the placement position of the new learning card in the physical grid on the base plate is located. The pattern or text content corresponding to the new learning card is associated and bound one-to-one with the screen display grid at the corresponding position on the screen, establishing a three-element mapping relationship of base plate physical grid - screen display grid - learning card content. This ensures that the screen display grid synchronously loads the content information corresponding to the new learning card and synchronously updates the content loading status and placeholder status of the screen display grid, ensuring that after the new learning card is placed in the physical grid on the base plate, the screen display grid completes content matching and status synchronization without delay. Following the row and column arrangement order of the physical grid on the base, the information corresponding to all the learning cards that have been bound is traversed. The physical grid position code, card content type code, card content data, screen display grid position code, and binding status code corresponding to each learning card are encapsulated in an orderly manner according to a preset data frame format to form a standardized learning card layout association data frame. Multiple ordered data frames are combined according to the arrangement order of the physical grid on the base to form a complete learning card layout association dataset. This dataset fully records the physical placement position of the learning card in the physical grid on the base, the card content data, the virtual display position of the screen display grid, and the binding status information, providing standardized data support for subsequent learning card removal verification. When a learning card is detected to have been removed from the physical grid of the baseboard based on a valid image of the baseboard placement, the system quickly matches the baseboard physical grid code, screen display grid code, and card content data corresponding to the removed learning card using the learning card layout association dataset. This data item is marked as pending verification, and the content of the screen display grid is simultaneously cleared and its status updated. At the same time, the feature data corresponding to the removal operation is compared and verified with the learning card layout association dataset to determine whether the removal operation conforms to the preset interaction rules. This provides accurate data basis for subsequent interaction feedback and data updates. Unlike traditional paper-screen interaction that only displays single content without standardized layout data support, this system improves the continuity and stability of paper-screen interaction during the placement and removal of learning cards through precise mapping between the baseboard physical grid and the screen display grid, orderly encapsulation of learning card content, and full-process verification of layout data.

[0028] Optionally, the effective images of the baseboard, including the baseboard and the laid-out learning cards, are obtained according to preset rules. This includes: continuously acquiring multiple frames of original baseboard images containing the baseboard and the laid-out learning cards, and performing standardized preprocessing on the multiple frames of original baseboard images to obtain multiple frames of standard baseboard images; scoring the image clarity of the multiple frames of standard baseboard images based on a preset clarity judgment model, and selecting the single frame of standard baseboard images with the highest clarity score as the effective baseboard images.

[0029] Optionally, the image sharpness of the multi-frame standard images of substrate placement is scored, and the single-frame standard image with the highest sharpness score is selected as the valid image for substrate placement according to the image sharpness score ranking. Then, the following steps are taken: performing standardization preprocessing on the single-frame standard image of substrate placement to form a single-frame standardized substrate placement image; and performing background removal and interference suppression processing on the single-frame standardized substrate placement image based on a preset target region extraction model to obtain a single-frame interference-free substrate image as the valid image for substrate placement.

[0030] In this application, effective images of the baseboard and the pre-laid learning cards are obtained. Unlike the traditional method of directly acquiring and using single-frame images, this method first performs standardized preprocessing on multiple consecutive original images, then uses a designed sharpness judgment model to perform layered sharpness scoring and sorting, and subsequently performs secondary standardized preprocessing and background interference removal on the selected images. Finally, effective images of the baseboard without interference are obtained. The entire process improves image quality and the stability of subsequent grid layout detection through progressive processing of multi-frame image fusion, layered sharpness judgment, and background interference removal.

[0031] Preferably, continuous image acquisition is first performed on the learning cards already placed on the base plate. The acquired continuous multi-frame images all include the entire area of ​​the base plate and all the learning cards already placed on the base plate grid. The continuously acquired multi-frame images are determined as the original images of the base plate placement. Then, each frame of the original base plate placement image is subjected to size normalization and data normalization processing. The pixel values ​​of each frame of the original image are adjusted to within a preset range, and the size of each frame of the original image is scaled to a uniform specification. After processing, multiple frames of standard base plate placement images are obtained. The size specifications and data format of the multiple frames of standard base plate placement images are consistent, providing a unified basic data for subsequent clarity determination.

[0032] Specifically, in this application, the sharpness determination algorithm includes a two-layer feature processing structure. The first layer is an image basic feature extraction structure, and the second layer is a sharpness confidence output structure. The image basic feature extraction structure first extracts pixel-level texture gradient features and edge continuity features for each frame of the standard image of the substrate placement. The extracted features include pixel gradient features, edge continuity features, and region uniformity features of each frame of the image. Then, the three types of basic features are input into the sharpness confidence output structure. The sharpness confidence output structure performs weighted fusion processing on each type of basic feature and outputs the sharpness confidence value corresponding to each frame of the standard image of the substrate placement. The magnitude of the sharpness confidence value is used to characterize the sharpness of the corresponding image. Images with higher sharpness confidence values ​​have higher image sharpness, and images with lower sharpness confidence values ​​have lower image sharpness. Then, the sharpness confidence values ​​corresponding to multiple frames of standard images of the substrate placement are sorted in descending order, and the standard image of the substrate placement corresponding to the first sharpness confidence value in the sorted order is selected as the preliminary screened standard image of the substrate placement.

[0033] Preferably, the initial screened standard images of the baseboard placement are first subjected to secondary standardization preprocessing. The secondary standardization preprocessing includes image grayscale value normalization and image contrast normalization, adjusting the image grayscale value and contrast to within a preset range. After processing, a single-frame standardized baseboard placement image is obtained. Then, the single-frame standardized baseboard placement image is input into a preset target region extraction model. The target region extraction model includes a baseboard contour positioning structure and a non-target region removal structure. The baseboard contour positioning structure first performs baseboard contour positioning on the single-frame standardized baseboard placement image to determine the overall contour range of the baseboard and the layout range of the baseboard grid. The image area outside the baseboard contour range is determined as the non-target region. The non-target region removal structure performs pixel-by-pixel removal processing on the pixels in the non-target region, and at the same time, it performs suppression processing on the interference pixels in the non-target region. Interference pixels include ambient reflection pixels, clutter shadow pixels, and irrelevant edge pixels. After the removal and suppression processing is completed, a single-frame interference-free baseboard image containing only the baseboard and the already laid-out learning cards is obtained. This single-frame interference-free baseboard image is determined as the valid baseboard placement image.

[0034] Optionally, a grid layout detection model is used to detect the grid layout of the valid images of the base plate placement to construct a grid layout dataset. Specifically, this includes: based on the grid layout detection model, performing grid occupancy and grid position detection on the valid images of the base plate placement to generate grid occupancy and grid position data corresponding to each grid; and performing positional association and structured processing on the grid occupancy and grid position data corresponding to each grid according to the grid positional mapping mechanism to form a grid layout dataset.

[0035] Preferably, the effective image of the base plate placement is input into the designed grid layout detection model. This model is composed of a base plate corner point positioning layer, a grid area segmentation layer, and a grid occupancy detection layer connected in series. The base plate corner point positioning layer performs edge contour feature extraction and corner point feature matching processing on the effective image of the base plate placement, extracting the base plate corner point coordinate features in the effective image of the base plate placement. Based on the base plate corner point coordinate features, the main body area of ​​the base plate is completely segmented from the effective image of the base plate placement, and the background redundant area at the edge of the image is removed to obtain the main image of the base plate containing only the base plate and the learning cards.

[0036] Specifically, in this application, for the grid region segmentation process, the base plate main image is input into the grid region segmentation layer. The grid region segmentation layer performs proportional division processing on the overall size of the base plate main image according to the preset grid layout rules, and splits the base plate main image into multiple independent grid region images according to the row and column distribution. Each grid region image corresponds to a physical grid unit on the base plate. Multiple grid region images are arranged in a fixed row and column order to form a continuous grid region image sequence.

[0037] Preferably, the grid region image sequence is input into the grid occupancy detection layer. The grid occupancy detection layer performs texture gradient feature extraction and pixel distribution feature analysis on each grid region image. Based on the texture gradient features and pixel distribution features, it determines whether a learning card is placed in each grid region, and generates grid occupancy status features corresponding to each grid region. At the same time, it performs coordinate positioning processing on each grid region image to extract the grid position coordinate features corresponding to each grid region. The grid occupancy status features and grid position coordinate features are combined to form the grid occupancy and grid position data corresponding to each grid.

[0038] Specifically, in this application, regarding the execution process of the grid position mapping mechanism, the grid occupancy and position data corresponding to all grids are used as the processing basis. According to the row and column arrangement order of the grid area on the base plate, a unique grid position code is assigned to the grid occupancy and position data of each grid. The grid position code is used to represent the physical position order of the grid in the overall layout. The grid occupancy and position data with grid position codes are ordered and associated. The data is structured and encapsulated according to the coding order. Redundant feature information in the data is removed to form a regular and directly callable grid layout dataset. The grid layout dataset completely records the physical position, occupancy status and position information of each grid, providing standardized data support for the subsequent addition of learning card positioning.

[0039] Optionally, the preset card content recognition model includes a pattern card recognition structure layer and a text card recognition structure layer. Correspondingly, the preset card content recognition model is used to perform content recognition on the local image corresponding to the newly added learning card. Specifically, this includes: classifying the newly added learning card to determine whether it belongs to the pattern card type or the text card type; if the newly added learning card belongs to the pattern card type, then performing content recognition on the newly added learning card based on the pattern card recognition structure layer; if the newly added learning card belongs to the text card type, then performing content recognition on the newly added learning card based on the text card recognition structure layer.

[0040] Preferably, the local image corresponding to the newly added learning card is first input into the category recognition layer of the card content recognition model. This category recognition layer has a built-in feature extraction sublayer and a category determination sublayer. The feature extraction sublayer performs targeted feature extraction on the local image of the newly added card, focusing on extracting the differentiated features of the two types of cards—texture gradient features and contour continuity features of pattern cards, and character region features and character grayscale difference features of text cards, generating a card category recognition feature map. This feature map clearly distinguishes the core features of the two types of cards and weakens irrelevant interference. The category determination sublayer performs feature classification on the card category recognition feature map. It uses feature threshold comparison logic to compare the extracted features with preset feature thresholds for pattern cards and text cards. If the proportion of texture gradient features and contour continuity features in the card category recognition feature map is higher than the preset threshold, the newly added learning card is determined to be a pattern card. If the proportion of character region features and character grayscale difference features is higher than the preset threshold, the newly added learning card is determined to be a text card. A clear card category determination result is generated, which is directly used for switching subsequent recognition structure layers to ensure the targeting of the recognition logic.

[0041] Preferably, the pattern card recognition structure layer includes a stitching processing sublayer, a noise-resistant feature extraction sublayer, and a consistency determination sublayer, with each sublayer working together to complete the recognition. The stitching processing sublayer receives the card category determination result and a local image of a newly added card. It then retrieves all standard pattern images from the current learning level and stitches the local image of the newly added card with each standard pattern image of the same size, maintaining pixel alignment between the newly added card and the standard pattern images during stitching to avoid feature distortion caused by stitching deviations. This results in multiple sets of stitched image pairs (for example, if the current level has 8 standard pattern images, then 8 sets of stitched image pairs are generated; this is just an example and does not limit the scope of protection). The noise-resistant feature extraction sublayer uses a designed noise-resistant gradient convolution algorithm to perform noise reduction preprocessing on each set of stitched image pairs, filtering out noise interference and lighting deviations generated during image acquisition. Then, it extracts features from the denoised stitched image pairs to obtain the image gradient features and structural association features corresponding to each set of stitched image pairs. These two types of features completely preserve the edge details and overall structural information of the pattern, improving the reliability of subsequent consistency recognition. The consistency determination sublayer performs dimensional alignment and weighted fusion on the extracted image gradient features and structural association features to generate a feature fusion vector for the spliced ​​image pair. Then, it compares the cosine similarity of this vector with the preset standard matching vector and outputs the content consistency determination result for each spliced ​​image pair. The standard pattern with the highest matching degree in the consistency determination result is selected to determine the pattern content corresponding to the newly added learning card.

[0042] Preferably, in the specific implementation of content recognition based on the text card recognition structure layer, the text card recognition structure layer includes a text region localization sublayer, a character feature extraction sublayer, and a character recognition sublayer, designed to meet the character recognition requirements of text cards (word cards). The text region localization sublayer accurately locates the text region in the newly added card's local image, using local threshold segmentation logic to distinguish the character region from the card's background region, removing background interference, and generating a cropped text region image to ensure that the cropped image contains only complete word characters. The character feature extraction sublayer extracts character features from the cropped text region image, extracting the contour features, stroke structure features, and grayscale distribution features of each character, generating a character feature sequence. This sequence is arranged according to the order of characters in the word, preserving the positional relationship between characters. The character recognition sublayer employs a lightweight character recognition logic to recognize character feature sequences character by character. It combines word spelling rules for error correction and verification to avoid overall recognition errors caused by single character recognition deviations. It generates text content (i.e., word content) corresponding to newly added learning cards. This text content corresponds to the pattern content of the pattern cards, providing data support for content matching of subsequently removed cards.

[0043] Specifically, in this application, the learning cards include two categories: images and text. The pre-defined card content recognition model features a dual-branch structure layer that can optimize the recognition logic for the features of both types of cards, unlike traditional single-recognition models which have to consider various image features, resulting in low recognition efficiency and large biases. Simultaneously, the design of differentiated feature extraction in the category recognition layer ensures the accuracy of category determination and avoids errors in switching recognition structure layers due to confusion between the features of the two types of cards, further improving overall recognition reliability. Furthermore, this dual-branch structure can flexibly adapt to the expansion of card categories, eliminating the need to retrain the entire recognition model for adding image or text card types; only the standard features of the corresponding recognition structure layer need to be updated, enhancing the scalability of the solution.

[0044] Optionally, if the newly added learning card is a pattern card type, then the content recognition of the newly added learning card is performed based on the pattern card recognition structure layer, specifically including: splicing the newly added learning card with multiple standard pattern images to form multiple spliced ​​image pairs; and performing content consistency recognition on the multiple spliced ​​image pairs to determine the pattern content corresponding to the newly added learning card.

[0045] Preferably, the pattern card recognition structure layer first receives the card category determination result. After confirming that the newly added learning card is a pattern card type, it retrieves all the preset standard pattern images in the current learning level. These standard pattern images correspond one-to-one with the pattern cards in the learning cards and are stored in the built-in standard pattern database. The database can flexibly add new standard pattern images according to the needs of the learning scenario without retraining the entire recognition model. After retrieving the local image corresponding to the newly added learning card and all standard pattern images, the two types of images are preprocessed. The local image of the newly added card and each standard pattern image are adjusted to the same pixel size to ensure pixel alignment during splicing and avoid feature distortion caused by size deviation. Then, the edge alignment splicing method is used to splice the local image of the newly added card and a single standard pattern image into a complete spliced ​​image. The splicing point retains a clear edge transition to avoid obscuring the core features of the two types of images. This process is repeated with all standard pattern images to form multiple spliced ​​image pairs (for example, if there are 8 standard pattern images in the current level, 8 spliced ​​image pairs are generated; this is just an example and does not limit the scope of protection). Each spliced ​​image pair contains the newly added card pattern and one standard pattern, providing a comparison basis for subsequent content consistency recognition.

[0046] Preferably, the pattern card recognition structure layer incorporates a noise-resistant feature extraction sublayer, a feature fusion sublayer, and a consistency determination sublayer, with each sublayer working collaboratively to complete the recognition. After receiving multiple sets of stitched image pairs, the noise-resistant feature extraction sublayer uses a designed noise-resistant gradient convolution algorithm to perform noise reduction preprocessing on each set of stitched image pairs, filtering out noise interference, light reflection deviations, and residual background interference generated during image acquisition, generating denoised stitched image pairs. Feature extraction is then performed on the denoised stitched image pairs, focusing on extracting image gradient features and structural association features. Image gradient features are used to capture the edge contour details of the pattern, while structural association features are used to capture the overall layout and texture distribution of the pattern. These two types of features together constitute the core recognition features of the stitched image pairs, generating image gradient features and structural association features corresponding to each set of stitched image pairs, ensuring the completeness and anti-interference capabilities of feature extraction.

[0047] Preferably, the image gradient features and structural association features corresponding to each pair of spliced ​​images are first input into a feature fusion sublayer. This sublayer performs dimensional alignment processing on the two types of features to eliminate dimensional differences and avoid feature distortion during fusion. Then, a weighted fusion logic is used to assign different weights to the two types of features according to their importance in pattern recognition. The aligned image gradient features and structural association features are integrated to generate a spliced ​​image pair feature fusion vector. This vector integrates the edge details and overall structural information of the pattern, improving the reliability of consistency recognition. The spliced ​​image pair feature fusion vector is input into a consistency judgment sublayer. This sublayer has a built-in preset feature matching threshold and uses cosine similarity calculation logic to compare the feature fusion vector of each pair of spliced ​​images with a preset standard matching vector. The standard matching vector is generated by feature fusion of a standard pattern image and corresponds one-to-one with each pair of spliced ​​images. After comparison, the content consistency judgment result corresponding to each pair of spliced ​​images is output. This result is presented in the form of a matching degree value. The higher the matching degree value, the higher the content consistency between the newly added learning card and the corresponding standard pattern. The consistency judgment results of all spliced ​​image pairs are sorted, and the spliced ​​image pair with the highest matching degree value is selected. The standard pattern image corresponding to the spliced ​​image pair is retrieved, and the pattern content corresponding to the standard pattern image is determined as the pattern content corresponding to the new learning card. At the same time, the pattern content is synchronized to the construction module for association and binding with the screen display grid, providing data support for the content matching of subsequent continuous removal of cards.

[0048] Specifically, unlike traditional pattern recognition models that employ multi-classification, which require pre-training features for all pattern categories and retraining the entire model when adding a new category, resulting in poor scalability, this application uses a stitching comparison method. When adding a new pattern category, only the corresponding standard pattern image needs to be added to the standard pattern database, eliminating the need to retrain the recognition model. This significantly improves the scalability and flexibility of the solution, perfectly meeting the needs of scenarios where learning cards requires continuous updates to pattern categories. Furthermore, the application of a noise-resistant gradient convolution algorithm, compared to traditional convolution algorithms, can more accurately capture the edge and texture features of patterns, maintaining high recognition reliability even with low image quality and interference.

[0049] Optionally, content consistency recognition is performed on the multiple sets of spliced ​​image pairs to determine the pattern content corresponding to the newly added learning card. This includes: performing noise reduction preprocessing and image feature extraction on each set of spliced ​​image pairs based on a noise-resistant gradient convolution algorithm to obtain the image gradient and structural association features corresponding to each set of spliced ​​image pairs; performing content consistency recognition on each set of spliced ​​image pairs based on the extracted image gradient and structural association features corresponding to each set of spliced ​​image pairs to output the content consistency judgment result corresponding to each set of spliced ​​image pairs; and determining the standard pattern consistent with the content of the newly added learning card based on the content consistency judgment result to determine the pattern content corresponding to the newly added learning card.

[0050] This application addresses the problem in interactive learning card scenarios where spliced ​​images are easily affected by factors such as lighting conditions during shooting and tabletop reflections, leading to inaccurate feature extraction and content consistency recognition deviations. It designs a noise-resistant gradient convolution algorithm and corresponding recognition logic. Unlike traditional convolution algorithms that only perform weighted summation of pixels, this algorithm improves the extraction accuracy of pattern edge and texture features through pixel difference convolution. Simultaneously, it integrates noise reduction preprocessing and feature fusion judgment, solving the pain point of image quality fluctuations in interactive learning cards while ensuring the stability of content consistency recognition. It adapts to the scenario requirements of flexible expansion of learning card pattern types, achieving accurate matching between newly added pattern cards and standard patterns.

[0051] The noise-resistant gradient convolution calculation logic designed in this application first performs a difference operation between each adjacent pixel value and the central target pixel value, then multiplies it by the corresponding convolution weights, sums all the weighted difference results, and finally adds the bias parameters to complete the convolution operation. The newly added convolution method can be expressed as: f(x) = (w1(x1-x) + w2) (x²-x)+w³ (x³-x)+w⁴ (x4-x)+w5 (x5-x)+w6 (x6-x)+w7 (x7-x)+w8 (x8-x))+b. Here, w1 to w8 are preset convolution weights used to adjust the influence of different adjacent pixels on the convolution result, adapting to the extraction requirements of pattern card texture features; x1 to x8 are the gray values ​​of the eight adjacent pixels of the target pixel in the stitched image pair, and x is the gray value of the target pixel, all derived from the pixel matrix of the stitched image pair after noise reduction; b is the bias parameter used to correct the systematic bias of the convolution operation, ensuring the accuracy of feature extraction. This difference-based convolution calculation method replaces the traditional convolution method that only performs weighted summation of pixels, better meeting the texture feature extraction requirements of pattern cards. Noise-resistant gradient convolution can amplify the differences in pixel gray-level changes, more accurately capturing the boundaries and texture features of the pattern.

[0052] Preferably, multiple sets of stitched image pairs are first input into the designed noise reduction preprocessing unit. This unit targets common shooting noises in paper screen interactive scenarios (such as light reflection, edge blurring, and environmental noise) and uses neighborhood pixel mean filtering logic to perform point-by-point traversal processing on the pixel matrix of each set of stitched image pairs, filtering out noise pixels with abrupt changes in pixel values, and generating noise-reduced stitched image pairs. The noise-reduced stitched image pairs retain the core texture and structural information of the newly added learning cards and standard patterns, and remove irrelevant interference pixels. (For example, when there are desktop reflection noises in the stitched image pairs, neighborhood pixel mean filtering will perform mean correction on the pixel values ​​of the reflection area, making the transition between the reflection area and the surrounding pixels smooth and avoiding noise interference to subsequent feature extraction). The noise-resistant gradient convolution feature extraction unit is designed to input the denoised and stitched image. The core of this unit is the noise-resistant gradient convolution structure, which calculates the difference between each pixel and its neighboring pixels to perform convolution operations, replacing the traditional convolution method that only performs weighted summation of pixels. Its processing logic fits the texture feature extraction requirements of pattern cards - the core distinguishing point of pattern cards lies in the edge contour and texture details. The noise-resistant gradient convolution can amplify the differences in pixel grayscale changes and more accurately capture the boundary and texture features of the pattern. This unit performs pixel-by-pixel convolution processing on the denoised stitched image pairs. First, it determines the adjacent pixel regions of each target pixel and calculates the grayscale difference between the target pixel and its adjacent pixels. Then, it weights and integrates all grayscale differences using preset convolution weights to generate image gradient features and structural association features corresponding to each stitched image pair. The image gradient features characterize the edge pixel change trends of the newly added learning card and the standard pattern in the stitched image pair. The structural association features characterize the overall layout and element distribution correspondence of the two patterns in the stitched image pair. These two types of features together constitute the core basis for content consistency recognition of the stitched image pair, ensuring the pertinence and completeness of feature extraction.

[0053] Preferably, the image gradient features and structural association features corresponding to each group of stitched image pairs are first input into the designed feature fusion unit. This unit performs dimensional alignment processing on the two types of features to eliminate the dimensional differences between the two types of features. Then, weighted fusion logic is used to integrate the aligned image gradient features and structural association features to generate a stitched image pair feature fusion vector. The stitched image pair feature fusion vector integrates the edge details and overall structural information of the pattern, improving the reliability of content consistency recognition. The feature fusion vector of the spliced ​​image is input into the content consistency determination unit. This unit has a built-in preset feature matching threshold and uses cosine similarity calculation logic to compare the feature fusion vector of the spliced ​​image with the preset standard matching vector, generating the content consistency determination result for each set of spliced ​​image pairs. The content consistency determination result is presented in the form of matching degree value. The higher the matching degree value, the higher the content consistency between the new learning card and the corresponding standard pattern. (For example, when the new learning card is an apple pattern and the corresponding standard pattern is also an apple pattern, the comparison result of the feature fusion vector of the spliced ​​image and the standard matching vector will show a high matching degree value; if the new learning card is an apple pattern and the standard pattern is a banana pattern, the matching degree value will be low.)

[0054] Preferably, in a given scenario, the content consistency judgment results of all spliced ​​image pairs are first sorted, and the content consistency judgment results with higher matching scores are selected. Then, the standard pattern image corresponding to the content consistency judgment result is retrieved, and the pattern content bound to the standard pattern image in the paper screen interaction scenario database is confirmed. This pattern content is then used as the pattern content corresponding to the newly added learning card, completing the content recognition of the pattern card. Simultaneously, to adapt to the flexible expansion of card pattern types in the scenario, this process synchronously records the feature matching data of this recognition. When adding a new standard pattern later, there is no need to retrain the feature extraction unit and judgment unit; only the new standard pattern image and corresponding standard matching vector need to be added. This significantly improves scenario adaptability, unlike the problem of high expansion difficulty in traditional multi-class recognition models. Furthermore, if the content consistency judgment results of all spliced ​​image pairs are lower than a preset matching threshold, the operation of re-extracting the local image of the newly added learning card is triggered, the spliced ​​image pair is regenerated, and the above content consistency recognition process is repeated to ensure the effectiveness of the recognition results, meeting the actual scenario requirements of card placement angle deviations and poor shooting quality in paper screen interaction.

[0055] Optionally, based on the learning card layout association dataset, learning cards are continuously removed, and the content matching of the continuously removed learning cards is determined to generate corresponding interactive feedback. This includes: obtaining valid images of the background removal according to preset rules; performing grid layout detection on the valid images of the background removal using a preset grid layout detection model to obtain a grid layout dataset containing grid position and grid occupancy information; comparing this dataset with the grid layout dataset corresponding to the adjacent previous valid images of the background removal to determine the removed learning cards; extracting the card content of the continuously removed learning cards based on the card content bound to the screen display grid corresponding to the removed learning cards; and generating corresponding interactive feedback based on the content matching of the continuously removed learning cards.

[0056] This application addresses the issues of misjudgment, delayed content matching, and feedback that doesn't align with user habits when continuously removing learning cards in interactive learning card scenarios. It designs an integrated method for continuous removal judgment, grid dataset comparison, and content matching feedback. Unlike traditional methods that judge each removal individually, this method accurately identifies the card to be removed by comparing adjacent frame grid datasets. Combined with a learning card layout-related dataset, it achieves rapid content matching. A feedback mechanism tailored to the interactive scenario is also designed. This solves the pain point of misjudgment when users continuously remove cards, while ensuring timely and targeted feedback. It adapts to the complete "placement-removal-matching-feedback" operation flow in interactive learning cards, while also connecting with the previous card placement and recognition logic to form a seamless technical chain.

[0057] Preferably, a pre-constructed learning card layout association dataset is first retrieved. This dataset contains the binding relationship between each grid position and the corresponding learning card content, as well as the positional mapping relationship between the screen display grid and the base grid, serving as the basis for continuous removal judgment and content matching. The process of collecting and filtering valid images for base removal is initiated according to preset rules, ensuring that the collected images accurately reflect the true state after card removal. Then, the grid layout detection model completes grid information detection, compares it with the preceding dataset to determine the removed cards, extracts the content of the removed cards and completes the matching, and finally generates corresponding interactive feedback. The entire process of "image acquisition - card judgment - content matching - interactive feedback" is executed seamlessly, closely matching the actual operation scenario of paper-screen interaction.

[0058] Preferably, multiple frames of original images of the baseboard and remaining learning cards after baseboard removal are continuously acquired. The acquisition process adapts to the user's continuous card-taking rhythm to avoid missing card removal actions due to excessively long acquisition intervals (for example, the acquisition interval is set to a preset duration, specifically 100-300 milliseconds, which is only an example and does not limit the protection scope). Standardization preprocessing is performed on the acquired multiple frames of original baseboard removal images. Preprocessing operations include pixel size normalization and image data standardization to eliminate image deviations caused by different acquisition angles and light intensities, generating multiple frames of standard baseboard removal images. These standard images are then input into a designed sharpness determination model. This model employs a lightweight feature extraction structure, unlike traditional fuzzy detection algorithms that only output a binary judgment of sharpness. This model calculates pixel gradients for each frame of standard baseboard removal images and outputs a quantified sharpness score. A higher sharpness score indicates higher recognizability of grid boundaries and card outlines in the image. The standard images for removing the card are sorted in descending order of sharpness score. The single-frame standard image with the highest sharpness score is selected as the valid image for card removal. This ensures the accuracy of subsequent grid layout detection and card removal determination, avoids grid recognition deviation caused by image blur, and adapts to scenarios such as camera shake and lighting changes that may occur when users remove cards.

[0059] Preferably, the valid image of the removed baseboard is first input into the grid layout detection model. This model uses the core structure of the previous grid detection and optimizes the grid occupancy judgment logic for the removal scenario. It can accurately distinguish between empty white squares and the grid areas left after the removal of cards, avoiding misjudging empty white squares without cards as removed areas. The grid layout detection model performs grid occupancy and grid position detection on the valid image of the removed baseboard, extracts the coordinate information and occupancy status of each grid (occupancy status is divided into two types: with cards and without cards), and generates the current grid layout dataset containing grid position and grid occupancy information. The system retrieves the preceding grid layout dataset corresponding to the effective images of the adjacent background removal. It then performs a grid-by-grid comparison between the current grid layout dataset and the preceding grid layout dataset, comparing grid position coordinates and grid occupancy status. Grids whose occupancy status changes from "card present" to "card absent" are identified as the grid corresponding to the removed learning card. The system extracts the grid's position information and grid identifier to generate a removed learning card identifier, thus completing the accurate identification of the removed learning card. To avoid misjudgment from a single comparison, a two-frame comparison verification mechanism is added. If the occupancy status of the same grid changes from "card present" to "card absent" in two consecutive frames, the card corresponding to that grid is confirmed as the removed learning card. If only a single frame shows a change in occupancy status, it is considered a shooting deviation and is not recorded as a removed card, further improving the reliability of the determination.

[0060] Preferably, based on the removed learning card identifier, the screen display grid information corresponding to that grid identifier in the learning card layout association dataset is retrieved first. Then, the card content bound to the screen display grid is extracted to generate the removed learning card content information. The removed learning card content information includes the card category (picture card or text card) and specific content (the word corresponding to the picture or the word content of the text card). A continuous removal counting mechanism is activated to accumulate the count of the removed learning cards. When the accumulated count reaches a preset number of consecutive removals (for example, the preset number of consecutive removals is 2, which is only an example and does not limit the scope of protection), the counting stops and the content information of the two consecutively removed learning cards is extracted. The matching of the two sets of content information is then determined. The matching logic aligns with the learning scenario of paper-screen interaction. Its core is determining whether the contents of two consecutively removed cards correspond to each other (one a picture card, the other a text card with a corresponding word). If the contents correspond, a matching feedback command is generated, triggering a grid elimination animation on the screen and generating positive feedback information that affirms the user's action, aligning with the user's motivational needs during learning. If the contents do not correspond, a matching failure feedback command is generated, triggering a prompt animation in the corresponding grid and generating guidance information to guide the user to return the removed card to its original position. After the user returns the card, the initial card placement recognition process is initiated, re-acquiring a valid image of the baseboard placement, repeating the grid layout detection, card recognition, and association binding steps, and updating the learning card layout association dataset to ensure the accuracy of subsequent consecutive removal judgments, forming a complete operational loop. If the number of cards removed consecutively does not reach the preset number, the count continues to accumulate, waiting for subsequent removal operations by the user until the preset number is reached before executing the content matching and feedback process. This adapts to the user's habit of continuously removing cards and avoids frequent feedback affecting the user experience.

[0061] Optionally, obtaining valid images for substrate removal according to preset rules includes: continuously acquiring multiple frames of original substrate removal images, and performing standardized preprocessing on the multiple frames of original substrate removal images to obtain multiple frames of standard substrate removal images; based on a preset sharpness determination model, scoring the image sharpness of the multiple frames of standard substrate removal images, and selecting the single frame of standard substrate removal image with the highest sharpness score as a valid image for substrate removal according to the image sharpness score ranking.

[0062] This application addresses the issues of blurry images, light interference, and mismatch between the acquisition rhythm and the operation rhythm that users often encounter when continuously removing cards in interactive card-playing scenarios. It designs a complete process solution for acquiring, preprocessing, and filtering valid images for card removal. Unlike traditional single-frame acquisition and simple binary clarity judgment methods, this solution utilizes a multi-frame acquisition strategy adapted to continuous card removal operations, targeted standardized preprocessing, and a lightweight clarity judgment model to achieve accurate filtering of valid images for card removal. This ensures the accuracy of subsequent grid layout detection and card removal judgment, adapts to users' continuous card removal habits in interactive card-playing scenarios, and connects with the acquisition logic of valid images for initial card placement, forming a unified image acquisition and filtering standard. This solves the pain point of judgment deviations caused by poor image quality in traditional acquisition methods.

[0063] Preferably, image acquisition is initiated first. Based on the user's continuous removal of learning cards, an appropriate continuous acquisition strategy is set to continuously acquire multiple frames containing the original image of the baseboard and the remaining learning cards after baseboard removal. During the acquisition process, both acquisition efficiency and image integrity are taken into account to avoid missing card removal actions due to excessively long acquisition intervals, and to avoid image redundancy and reduced processing efficiency due to excessively short acquisition intervals. (For example, the acquisition interval is set to a preset duration, specifically 100-300 milliseconds, and the number of acquisition frames is set to a preset number of frames, specifically 5-10 frames. This is just an example and does not limit the scope of protection.) After acquisition, standardized preprocessing is performed on each of the multiple frames of original images with the base removed. The preprocessing operation is specifically designed to address common acquisition deviations in paper screen interactive scenarios. First, the pixel size of each frame of original images with the base removed is normalized to adjust all images to a uniform pixel size, ensuring the consistency of subsequent processing. Then, image data standardization is performed on the normalized images to map pixel values ​​to a preset range, eliminating pixel grayscale deviations caused by different acquisition lighting and shooting angles. At the same time, isolated pixel noise in the images is filtered out to generate multiple frames of standard images with the base removed. The standard images with the base removed retain the core information of the base grid boundary and the outline of the remaining cards, while eliminating irrelevant acquisition deviations and minor noise interference.

[0064] Preferably, the standard images of the background removed from multiple frames are first input into the designed sharpness determination model one by one. This model adopts a lightweight feature extraction structure, which is different from the traditional fuzzy detection algorithm that only outputs a binary judgment result of "sharp" or "unsharp", and also different from the binary classification confidence output of the traditional MobileNetV3 Small model. This model is specifically designed for the sharpness determination of the background grid and card outline, and includes a feature extraction layer, a gradient calculation layer and a scoring output layer. Each layer works together to complete the sharpness measurement scoring. The feature extraction layer extracts grid boundaries and card outline features from the input standard image for background removal, generating a grid outline feature map. This map focuses on the edge lines of the grid and the outline details of the cards, ignoring irrelevant background areas. The gradient calculation layer calculates pixel gradients on the grid outline feature map, statistically analyzing the pixel grayscale changes of the grid boundaries and card outlines. Larger grayscale changes indicate clearer grid boundaries and card outlines, while smaller changes indicate blurred outlines. The scoring output layer weights and integrates the pixel gradient calculation results to generate a quantified clarity score. The clarity score ranges from a preset interval; a higher score indicates higher recognizability of the grid boundaries and card outlines, making it more suitable for subsequent grid layout detection and card removal determination. The clarity scores of multiple frames of standard images for background removal are sorted, and the single frame with the highest clarity score is selected as the valid image for background removal. If multiple standard images of card removal have the same and highest clarity scores, the frame with the closest acquisition time to the current card removal operation will be selected as the valid image for card removal. This ensures that the image accurately reflects the state of the card after removal, providing high-quality image input for subsequent grid layout detection and card removal determination. It avoids grid recognition deviations and misjudgments of removed cards due to image blurring, and adapts to scenarios such as camera shake and lighting changes that may occur when users continuously remove cards.

[0065] Specifically, in this application, the core design of the feature extraction layer for the clarity assessment model is tailored to the recognition requirements of the background grid and card outlines. It employs local feature enhancement logic to strengthen the feature extraction of the grid boundaries and card outline areas, while weakening the feature response of the background area. Compared to traditional feature extraction methods, this approach can more accurately capture the core features required for clarity assessment, further improving the accuracy of the clarity score. Simultaneously, the coordinated design of standardized preprocessing and the clarity assessment model enables the selection of valid background removal images even in low-light conditions or with slight camera shake. This overcomes the drawback of traditional acquisition and screening methods that have high requirements for the shooting environment, improving the adaptability to paper screen interaction scenarios and ensuring the smooth execution of subsequent card removal assessment and content matching.

[0066] like Figure 2As shown in the illustration, this application also provides a paper screen interactive device based on learning cards, comprising: a determination module, which performs grid layout detection on valid images of the baseboard using a preset grid layout detection model to construct a grid layout dataset, and compares the layout with the grid layout datasets corresponding to adjacent previous valid images of the baseboard to determine the position of the newly added learning card and its corresponding grid; a construction module, which extracts the local image corresponding to the newly added learning card from the valid images of the baseboard, performs content recognition on the local image corresponding to the newly added learning card using a preset card content recognition model, and associates and binds the content corresponding to the newly added learning card with the grid displayed on the screen to construct a learning card layout association dataset; and an interactive feedback module, which continuously removes learning cards based on the learning card layout association dataset, and determines the content matching of the continuously removed learning cards to generate corresponding interactive feedback.

[0067] This application addresses the issues of poor coordination, insufficient recognition accuracy, and limited adaptability among the various modules in interactive card-based learning scenarios. It designs an integrated, interconnected structure comprising a determination module, a construction module, and an interactive feedback module. Unlike traditional devices where modules operate independently and data interaction is inefficient, this design achieves seamless integration through standardized data interfaces. Each module is designed with its own processing logic tailored to the specific needs of interactive card-based learning scenarios. The determination module focuses on the precise positioning of newly added cards; the construction module achieves accurate binding of card content recognition to the screen grid; and the interactive feedback module adapts to real-time judgment and feedback for continuous removal operations. These three modules work together to form a complete technical chain of "positioning-recognition-binding-feedback," solving the pain points of traditional devices such as card positioning errors, weak anti-interference capabilities in content recognition, and feedback that doesn't align with user operating habits. It also improves the device's adaptability to expanding card types and changing operating scenarios.

[0068] Preferably, the determination module has a built-in preset grid layout detection model and a data caching unit. The data caching unit stores the grid layout dataset corresponding to adjacent previous valid base card placement images, ensuring the continuity of temporal comparison. The determination module first receives externally input valid base card placement images. These images are single-frame interference-free base card images after preprocessing, background removal, and interference suppression, containing complete information about the base card grid and the placed learning cards. The valid base card placement images are then input into the grid layout detection model. This model adopts a layered structure design, including an image preprocessing layer, a grid feature extraction layer, and a occupancy and position detection layer. Each layer works together to complete the grid layout detection. The image preprocessing layer performs secondary standardization on the input base plate placement image, including pixel size normalization and grayscale correction, eliminating slight deviations left over from the previous processing and generating a standardized grid detection image. The grid feature extraction layer uses local feature enhancement logic to specifically extract features from the base plate grid boundaries and card outlines, weakening background interference and generating a grid outline feature map. The grid outline feature map clearly shows the boundary lines and occupancy status of each grid. The occupancy and position detection layer performs grid-by-grid analysis on the grid outline feature map, identifying the coordinate information (i.e., grid position data) and occupancy status (divided into two categories: with card and without card), while also labeling the preliminary category of the occupancy grids (pattern card, text card, empty white grid), generating grid occupancy and grid position data for each grid. According to the preset grid position mapping mechanism, the grid occupancy and grid position data corresponding to each grid are associated and structured, and the grid position coordinates are matched one-to-one with the grid position of the grid displayed on the screen to form a structured grid layout dataset. The grid layout dataset contains core information such as grid position, grid position coordinates, grid occupancy status, and preliminary category of the occupancy grid. The system retrieves the preceding grid layout dataset corresponding to the valid images of adjacent previous baseboard placements stored in the data cache unit. It then performs a grid-by-grid time-series comparison between the currently generated grid layout dataset and the preceding grid layout dataset, including the grid occupancy status and grid position coordinates. Grids whose occupancy status changes from "no card" to "card" are selected and identified as the grid corresponding to the new learning card. The system extracts the grid's position coordinates, grid order, and preliminary category to generate the new learning card and its corresponding grid position information. Simultaneously, the current grid layout dataset is updated to the data cache unit, overwriting the preceding grid layout dataset. This provides a comparison basis for the next new card determination, ensuring the continuity and accuracy of new card positioning and adapting to users' habit of placing cards in batches.

[0069] Preferably, the construction module and the determination module are linked through a data interface. The construction module receives the newly added learning cards and their corresponding grid positions output by the determination module, and simultaneously receives the previously processed valid images of the baseboard placement. The construction module first extracts a local image corresponding to the newly added learning card from the valid baseboard placement image based on the grid position coordinates. The extraction process employs precise cropping logic, using the grid position coordinates as a reference to define the cropping area, ensuring that the cropped local image only contains the complete content of the newly added learning card, removing irrelevant areas such as grid boundaries and adjacent cards, thus generating a local image of the newly added card. This local image is then input into a preset card content recognition model, which includes a category recognition layer and a branch recognition layer. The category recognition layer first performs category recognition on the local image of the newly added card, determining whether the newly added learning card belongs to the pattern card type or the text card type by extracting the texture and contour features of the local image, generating a card category determination result. If the card category is determined to be a pattern card, the local image of the new card is input into the pattern card recognition sub-model in the branch recognition layer. This sub-model uses a splicing comparison recognition logic to splice the local image of the new card with multiple standard pattern images in the current level, forming multiple spliced ​​image pairs. Then, a noise-resistant gradient convolution algorithm is used to perform noise reduction preprocessing and feature extraction on each spliced ​​image pair to obtain the image gradient and structural association features corresponding to each spliced ​​image pair. Based on the feature comparison, the content consistency judgment result is output, and the standard pattern that matches the content of the local image of the new card is selected to determine the pattern content corresponding to the new learning card. If the card category is determined to be a text card, the local image of the new card is input into the text card recognition sub-model in the branch recognition layer. This sub-model uses a lightweight text recognition structure to locate and recognize the text region in the local image of the card, and outputs the text content (i.e., word content) corresponding to the new learning card. After recognizing the content of the newly added learning cards, the construction module retrieves the grid position information corresponding to the newly added learning card output by the determination module, obtains the screen display grid position corresponding to that grid position, and associates and binds the content (image content or text content) of the newly added learning card with the screen display grid, generating grid-card content binding data. Simultaneously, it integrates all bound grid-card content data, grid position data, and card category data to form a structured learning card layout association dataset. This dataset is synchronized to the interactive feedback module in real time, providing core data support for subsequent continuous removal judgment of learning cards, content matching, and interactive feedback. Furthermore, the construction module has a built-in data update unit that updates the learning card layout association dataset in real time when the user adjusts the card placement, adds, or removes cards, ensuring that the dataset remains consistent with the actual card placement and screen display status.

[0070] Preferably, in one scenario, the interactive feedback module integrates an image acquisition and control unit, a grid comparison unit, a content matching unit, and a feedback generation unit. Its core relies on the learning card layout association dataset synchronized by the construction module to complete the entire process of continuous removal of learning cards and interactive feedback. The interactive feedback module first initiates the acquisition process of valid images for base removal through the image acquisition and control unit. It controls an external image acquisition device to continuously acquire multiple frames of original base removal images containing the base and remaining learning cards. These multiple frames of original base removal images undergo standardized preprocessing to generate multiple frames of standard base removal images. Then, a preset clarity judgment model is used to score the clarity of these standard base removal images, and the single frame with the highest clarity score is selected as the valid base removal image. The valid image of the baseboard removal is input into the grid comparison unit. The grid comparison unit calls the preset grid layout detection model to perform grid layout detection on the valid image of the baseboard removal, extracts the position information and occupancy status of each grid, generates the current grid layout dataset, and simultaneously retrieves the previous grid layout dataset corresponding to the adjacent valid image of the baseboard removal. The current grid layout dataset and the previous grid layout dataset are compared grid by grid to filter out the grids whose occupancy status changes from "with card" to "without card", determine the grid position and grid order corresponding to the removed learning card, and generate the removed learning card identifier. The content matching unit retrieves the screen display grid bound content corresponding to the grid position in the learning card layout association dataset based on the removed learning card identifier, extracts the specific content (picture content or text content) and card category of the removed learning card, starts the continuous removal counting mechanism, accumulates the count of the removed learning cards, and stops counting when the accumulated count reaches the preset number of consecutive removals (for example, the preset number is 2, which is only an example and does not limit the scope of protection). The content information and category information of the two consecutively removed learning cards are then extracted to determine the content matching. If one of the two learning cards removed consecutively is a picture card and the other is a text card with a corresponding word, meaning their contents are related, the content matching unit generates a matching success signal. Upon receiving this signal, the feedback generation unit generates an interactive feedback instruction indicating successful matching. This instruction triggers the screen to display a grid elimination animation and simultaneously generates positive prompts that fit the learning scenario. If the two learning cards removed consecutively are not related, the content matching unit generates a matching failure signal. Upon receiving this signal, the feedback generation unit generates an interactive feedback instruction indicating failed matching. This instruction triggers the screen to display a prompt animation for the corresponding grid and simultaneously generates guidance prompts to guide the user to return the removed card to its original grid position.After the user places a card back into its original grid, the interaction feedback module sends a data update signal to the determination and construction modules. This triggers the determination module to re-detect the grid layout and determine if a new card will be added, while the construction module re-identifies and associates card content, updating the learned card layout association dataset to ensure the accuracy of subsequent continuous removal decisions. If the number of consecutively removed cards does not reach the preset number, the interaction feedback module continues to accumulate the count, waiting for subsequent removal operations from the user until the preset number is reached before executing the content matching and feedback process. This adapts to the user's habit of continuously removing cards and avoids frequent feedback that could negatively impact the user experience.

[0071] An electronic device includes: a memory for storing executable program code; and a processor for calling and running the executable program code from the memory, causing the electronic device to perform a paper screen interaction method based on learning cards as described in any of the preceding claims.

[0072] A computer program product or computer program comprising computer instructions that, when executed by a processor, implement a paper-screen interactive method based on learning cards as described in any of the preceding claims.

[0073] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for learning card-based paper screen interaction, characterized by, include: The grid layout detection model is used to detect the grid layout of the valid images of the base plate placement to construct a grid layout dataset. The grid layout is then compared with the grid layout datasets corresponding to the valid images of the adjacent previous base plate placement to determine the newly added learning cards and their corresponding grid positions. Extract the local image corresponding to the newly added learning card from the effective image of the base plate placement, and perform content recognition on the local image corresponding to the newly added learning card through a preset card content recognition model. At the same time, associate and bind the content corresponding to the newly added learning card with the screen display grid to construct a learning card layout association dataset. Based on the learning card layout association dataset, learning cards are continuously removed, and the content matching of the continuously removed learning cards is determined to generate corresponding interactive feedback.

2. A method of paper screen interaction based on learning cards according to claim 1, characterized in that, Also includes: According to the preset grid layout rules, the learning cards are placed on a base plate with the same grid layout outside the screen, and the effective image of the base plate containing the base plate and the learning cards is obtained according to the preset rules.

3. A method of paper screen interaction based on learning cards according to claim 2, characterized in that, Obtain valid images of the baseboard and the already laid-out learning cards according to preset rules, including: Multiple frames of original images of the base plate and the layout of the learning cards are continuously acquired, and the multiple frames of original images of the base plate are standardized preprocessed to obtain multiple frames of standard images of the base plate. Based on a preset sharpness determination model, the image sharpness of the multi-frame standard images of the base plate placement is scored, and the single-frame standard image of the base plate placement with the highest sharpness score is selected as the valid image of the base plate placement according to the image sharpness score sorting.

4. A method of paper screen interaction based on learning cards according to claim 3, characterized in that, The image sharpness of the multi-frame standard images of the substrate placement is scored, and the single-frame standard image with the highest sharpness score is selected as the valid image for substrate placement according to the image sharpness score. The process then includes: The single-frame base plate placement standard image is subjected to standardization preprocessing to form a single-frame standardized base plate placement image; Based on a preset target region extraction model, the single-frame standardized base plate placement image is subjected to background removal and interference suppression processing to obtain a single-frame interference-free base plate image as an effective image for base plate placement.

5. The method of claim 1, wherein the learning card-based paper screen interaction is based on a card game. A pre-defined grid layout detection model is used to detect the grid layout of valid base plate placement images to construct a grid layout dataset, which specifically includes: Based on the preset grid layout detection model, the effective image of the base plate placement is subjected to grid occupancy and grid position detection to generate grid occupancy and grid position data corresponding to each grid. According to the grid position mapping mechanism, the grid occupancy and grid position data corresponding to each grid are correlated and structured to form a grid layout dataset.

6. The method of claim 1, wherein the learning card-based paper screen interaction is based on a card game. The preset card content recognition model includes a pattern card recognition structure layer and a text card recognition structure layer. Correspondingly, the preset card content recognition model is used to perform content recognition on the local image corresponding to the newly added learning card, specifically including: The newly added learning cards are categorized to determine whether they belong to the image card type or the text card type. If the newly added learning card is a pattern card, then the content of the newly added learning card is recognized based on the pattern card recognition structure layer; If the newly added learning card is a text card type, then the content of the newly added learning card is recognized based on the text card recognition structure layer.

7. The method of claim 6, wherein, If the newly added learning card is a pattern card type, then content recognition is performed on the newly added learning card based on the pattern card recognition structure layer, specifically including: The newly added learning cards are spliced ​​together with multiple standard pattern images to form multiple spliced ​​image pairs; Content consistency recognition is performed on the multiple sets of spliced ​​image pairs to determine the pattern content corresponding to the newly added learning card.

8. The method for paper-screen interaction based on learning cards according to claim 7, characterized in that, Content consistency recognition is performed on the multiple sets of spliced ​​image pairs to determine the pattern content corresponding to the newly added learning card, including: Based on the noise-resistant gradient convolution algorithm, noise reduction preprocessing and image feature extraction are performed on each group of stitched image pairs to obtain the image gradient and structural correlation features corresponding to each group of stitched image pairs. Based on the extracted image gradient and structural association features corresponding to each group of spliced ​​image pairs, content consistency recognition is performed on each group of spliced ​​image pairs to output the content consistency judgment result corresponding to each group of spliced ​​image pairs. Based on the content consistency determination result, a standard pattern that is consistent with the content of the newly added learning card is identified to determine the pattern content corresponding to the newly added learning card.

9. The method for paper-screen interaction based on learning cards according to claim 1, characterized in that, Based on the learning card layout association dataset, learning cards are continuously removed, and the content matching of the continuously removed learning cards is determined to generate corresponding interactive feedback, including: According to the preset rules, the effective images of the base removal are obtained. The grid layout detection model is used to detect the grid layout of the effective images of the base removal to obtain a grid layout dataset containing grid position and grid occupancy information. The grid layout dataset is then compared with the grid layout dataset corresponding to the adjacent previous effective images of the base removal to determine the learning card to be removed. Based on the card content bound to the screen display grid corresponding to the removed learning card, extract the card content of consecutively removed learning cards, and generate corresponding interactive feedback based on the content matching of consecutively removed learning cards.

10. The method for paper-screen interaction based on learning cards according to claim 9, characterized in that, Obtain valid images for base plate removal according to preset rules, including: Multiple frames of original images of substrate removal are continuously acquired, and the original images of substrate removal are standardized preprocessed to obtain multiple frames of standard images of substrate removal. Based on a preset sharpness determination model, the image sharpness of the multi-frame substrate removal standard images is scored, and the single-frame substrate removal standard image with the highest sharpness score is selected as the effective substrate removal image according to the image sharpness score ranking.

11. A paper-screen interactive device based on learning cards, characterized in that, include: The determination module uses a preset grid layout detection model to detect the grid layout of valid base plate images to construct a grid layout dataset. It then compares the grid layout dataset with the grid layout datasets corresponding to the valid base plate images in the preceding sequence to determine the newly added learning cards and their corresponding grid positions. The construction module extracts the local image corresponding to the newly added learning card from the effective image of the base plate placement, and performs content recognition on the local image corresponding to the newly added learning card through a preset card content recognition model. At the same time, it associates and binds the content corresponding to the newly added learning card with the screen display grid to construct a learning card layout association dataset. The interactive feedback module, based on the learning card layout association dataset, continuously removes learning cards and determines the content matching of the continuously removed learning cards in order to generate corresponding interactive feedback.

12. An electronic device, characterized in that, The electronic device includes: a memory for storing executable program code; and a processor for calling and running the executable program code from the memory, causing the electronic device to perform a paper screen interaction method based on learning cards as described in any one of claims 1 to 10.

13. A computer program product or computer program, characterized in that, The computer program product or computer program includes computer instructions that, when executed by a processor, implement a paper screen interaction method based on learning cards as described in any one of claims 1-10.

Images