An information display method, apparatus, electronic device, and storage medium

Abstract

This application discloses an information display method, apparatus, electronic device, and storage medium. The information display method includes: acquiring a first image upon receiving a first trigger operation; performing a first image processing on the first image to obtain a second image and displaying the second image; performing a second image processing on the second image to obtain incremental data, wherein the processing complexity of the second image processing is higher than that of the first image processing; and refreshing the currently displayed content based on the incremental data to replace the second image. This solution solves the technical problem of long waiting times for display results. By performing simple first image processing on the first image and immediately displaying the corresponding second image, and then continuously refreshing the currently displayed content based on the incremental data obtained from the second image processing, the final input result is gradually refreshed and displayed, significantly improving the speed of displaying information input results and avoiding long waiting times for users.

Description

Technical Field

[0001] This application relates to the field of intelligent recognition technology, and in particular to an information display method, device, electronic device and storage medium. Background Technology

[0002] In the development of modern educational informatization and intelligentization, the efficient and accurate display of entered information and intelligent recognition results has become a core element in improving work, study, and management efficiency. Especially in the field of educational support, error entry systems can transform paper-based questions into digital content, which is of paramount importance for building personal error notebooks, achieving precise learning analysis, and providing personalized exercise recommendations. In existing information entry display solutions, after acquiring the image taken or uploaded by the user, multiple intelligent recognition and processing tasks are required before the corresponding entry result is displayed. The waiting time from entry to display of the result is often long, resulting in a poor user experience. Summary of the Invention

[0003] This application provides an information display method, device, electronic device, and storage medium that can solve the technical problem of long waiting time for displaying information input results. By performing simple first image processing on the first image, the corresponding second image is immediately displayed. Then, based on the incremental data obtained from the second image processing, the current display content is continuously refreshed to gradually refresh and display the final input result, which significantly improves the speed of displaying information input results, avoids long waiting time for users, and thus improves the user experience.

[0004] In a first aspect, embodiments of this application provide an information display method, including: Upon receiving the first trigger operation, acquire the first image; Perform a first image processing step on the first image to obtain a second image, and then display the second image. The second image is processed to obtain incremental data, wherein the processing complexity of the second image processing is higher than that of the first image processing. The currently displayed content is refreshed based on incremental data to replace the second image.

[0005] Furthermore, the first image is subjected to a first image processing step to obtain the second image, including: The first image is processed by target object detection and cropping to obtain the corresponding target region image; The corrected image is obtained by performing image correction processing on the target region image; The corrected image is processed by target content recognition and cropping to obtain a second image containing the target content.

[0006] Furthermore, the incremental data includes target version tags; Accordingly, the currently displayed content is refreshed based on incremental data, including: Get the current version flag of the currently displayed content; If the target version marker is higher than the current version marker, refresh the currently displayed content based on the incremental data and update the current version marker.

[0007] Furthermore, the second image undergoes a second image processing step to obtain incremental data, including: The third image is obtained by enhancing the second image. Accordingly, the currently displayed content is refreshed based on incremental data, including: The third image is used to replace the currently displayed second image using a smooth image transition method.

[0008] Furthermore, after enhancing the second image to obtain the third image, the process also includes: The text content is obtained by performing optical character recognition (OCR) on the third image. Accordingly, after replacing the currently displayed second image with the third image using a smooth image transition method, the process also includes: The text content is refreshed to the preset text area of ​​the current display interface through the preset display component refresh mechanism.

[0009] Furthermore, after enhancing the second image to obtain the third image, the process also includes: The third image is categorized and labeled to obtain tag information; Accordingly, after replacing the currently displayed second image with the third image using a smooth image transition method, the process also includes: The label information is refreshed to the preset label area of ​​the current display interface through the preset display component refresh mechanism.

[0010] Furthermore, after performing optical character recognition (OCR) processing on the third image to obtain the text content, the process also includes: Matching information is obtained by searching for similar content in a preset database based on the text content; Accordingly, after refreshing the text content to the preset text area of ​​the current display interface through the preset display component refresh mechanism, it also includes: The matching information is refreshed to the preset matching information area of ​​the current display interface through the preset display component refresh mechanism.

[0011] In a second aspect, embodiments of this application provide an information display device, comprising: The image acquisition module is used to acquire the first image upon receiving the first trigger operation; The first processing module is used to perform a first image processing on the first image to obtain a second image; The display module is used to display the second image; The second processing module is used to perform second image processing on the second image to obtain incremental data, wherein the processing complexity of the second image processing is higher than that of the first image processing. The refresh module is used to refresh the currently displayed content based on incremental data to replace the second image.

[0012] In a third aspect, embodiments of this application provide an information display device, including: Memory and one or more processors; Memory, used to store one or more programs; When one or more programs are executed by one or more processors, the one or more processors implement the information display method as described in the first aspect.

[0013] In a fourth aspect, embodiments of this application provide a storage medium for storing computer-executable instructions, which, when executed by a computer processor, are used to perform the information display method as described in the first aspect.

[0014] This embodiment of the application, during information entry, performs a first image processing on the acquired first image to obtain a second image and displays the second image. Then, it performs a more complex second image processing on the second image to obtain incremental data, and refreshes the currently displayed content based on the incremental data. Using the above technical means, the second image can be obtained by performing a simple first image processing on the first image, and then displayed immediately. This avoids the problem of long waiting times caused by all processing steps having to be completed before content can be displayed. This embodiment can quickly display the second image after the first image is entered, significantly improving the display speed of information entry results and thus enhancing the user experience. Then, the more complex second image processing on the second image obtains corresponding incremental data, and the currently displayed content is continuously refreshed using the incremental data, achieving a gradual display of intelligent recognition results. This results in a smooth display interface without noticeable lag, significantly improving the display speed of information entry results and further enhancing the user experience.

[0015] The beneficial effects of the information display device, electronic device, and storage medium provided above can be referenced to the beneficial effects of the information display method. Attached Figure Description

[0016] Figure 1 This is a flowchart of an information display method provided in an embodiment of this application; Figure 2 This is a flowchart of another information display method provided in an embodiment of this application; Figure 3 This is a schematic diagram of a first image provided in an embodiment of this application; Figure 4 This is a schematic diagram of the target area image provided in an embodiment of this application; Figure 5 This is a flowchart of yet another information display method provided in the embodiments of this application; Figure 6 This is a flowchart of another information display method provided in an embodiment of this application; Figure 7 This is a first schematic diagram of a display interface provided in an embodiment of this application; Figure 8 This is a second schematic diagram of a display interface provided in an embodiment of this application; Figure 9 This is a flowchart of yet another information display method provided in the embodiments of this application; Figure 10 This is a third schematic diagram of a display interface provided in an embodiment of this application; Figure 11 This is a flowchart of another information display method provided in an embodiment of this application; Figure 12 This is a fourth schematic diagram of a display interface provided in an embodiment of this application; Figure 13 This is a schematic diagram of the structure of an information display device provided in an embodiment of this application; Figure 14 This is a schematic diagram of the structure of an information display device provided in an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, specific embodiments of this application will be described in further detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely for explaining this application and not for limiting it. It should also be noted that, for ease of description, only the parts relevant to this application are shown in the drawings, not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but may also have additional steps not included in the drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0018] In existing information entry and display solutions, after acquiring an image taken or uploaded by the user, multiple intelligent recognition and processing tasks need to be performed before the corresponding entry result can be displayed. The long waiting time from the start of entry to the display of results results in a poor user experience. Taking a wrong question entry system as an example, after receiving an image submitted by the user, the system needs to perform multiple processing steps, including book detection, image correction, question cropping, OCR (Optical Character Recognition) recognition, question labeling and classification, and original question search and matching. Only after all processing steps are completed is the final result displayed at once. Lightweight processing such as book detection, image correction, and question cropping only takes 200-500 milliseconds, but computationally intensive tasks such as image enhancement, OCR recognition, question labeling and classification, and original question search and matching take 2.2 seconds, 1 second, 1.5 seconds, and 1.84 seconds respectively, with the total processing time reaching 5-8 seconds. This processing mode results in users waiting a long time to see the input results, and the interface remains unresponsive for extended periods after the operation (waiting for the results of multiple processing steps), creating a noticeable lag and a poor user experience. Furthermore, in scenarios where multiple incorrect questions are entered consecutively, users must wait for the previous question to be processed before each entry, frequently interrupting the operation rhythm, significantly reducing input efficiency, and further impacting the user experience.

[0019] Based on this, the present application provides an information display method, apparatus, electronic device, and storage medium. During information input, a first image is processed to obtain a second image, which is then displayed. Subsequently, a more complex second image processing is performed on the second image to obtain incremental data, and the currently displayed content is refreshed based on the incremental data. Using the above technical means, a second image can be obtained by performing simple first image processing on the first image, and then immediately displayed. Compared to existing methods that require waiting for multiple processing steps to complete before displaying content, this embodiment can quickly display the second image after inputting the first image, significantly improving the speed of information input result display and thus enhancing the user experience. Furthermore, the more complex second image processing is performed on the second image to obtain corresponding incremental data, and the currently displayed content is continuously refreshed based on the incremental data. Compared to existing methods that display only after all processing steps are completed, this embodiment, while quickly displaying the second image, achieves a gradual display of the results of each processing step, resulting in a smooth display interface without noticeable lag, significantly improving the display speed of information input results and further enhancing the user experience.

[0020] Figure 1A flowchart of an information display method provided in this application embodiment is given. The information display method provided in this embodiment can be executed by an information display device, which can be implemented by software and / or hardware. The information display device can consist of two or more physical entities, or it can consist of a single physical entity. Generally, the information display device can be a terminal device, such as a PC (personal computer), interactive whiteboard, tablet computer, smart TV, learning machine, or smartphone. This embodiment can be applied to various application scenarios such as error entry, homework correction, test paper entry, document scanning, certificate entry, product entry, medical record entry, or report entry.

[0021] The following description uses a terminal device as the main entity executing the information display method. (Refer to...) Figure 1 The specific methods for displaying this information include: S11. Upon receiving the first trigger operation, acquire the first image.

[0022] In one embodiment, upon receiving a first trigger operation, a first image is acquired. The first trigger operation can be understood as an operation that triggers information entry, such as clicking a "Enter" control or an "Upload" control; this click operation is the first trigger operation. The first image can be acquired by the user taking a photo in real-time or capturing a snapshot, or it can be uploaded by the user. For example, a user opens a target application installed on a terminal device (e.g., a smartphone) and clicks the "Photo Entry" button on the target application's interface, thus executing the first trigger operation. The target application then accesses the system camera or photo album to acquire an image taken or selected by the user, and uses this image as the first image.

[0023] In one embodiment, taking the recording of incorrect questions as an example, the user opens the error notebook application installed on the terminal device (such as a smartphone), clicks the "Photo Recording" button in the error notebook application, and performs the first trigger operation. The error notebook application calls the system camera or album to obtain an original photo taken or selected by the user that contains the page containing the incorrect question, and uses this photo as the first image. For example, when a student user is doing a math workbook and finds that they have made a mistake on a geometry problem, they can open the error notebook application on the terminal device (such as a smartphone), click the "Photo Recording" button in the error notebook application, point the camera at the incorrect question in the workbook and press the shutter, obtaining a photo containing the content of the incorrect question, i.e., the first image. It should be noted that the first image at this time may contain redundant information such as desktop background, finger shadows, and multiple question contents.

[0024] As described above, the information entry and intelligent recognition process is initiated through the first trigger operation, and the corresponding first image is obtained. The acquisition of the original image can be completed through simple interactive operations, which improves the convenience of information entry.

[0025] S12. Perform first image processing on the first image to obtain the second image, and display the second image.

[0026] After obtaining the first image in S11, the first image is processed to obtain the second image, which is then displayed. The first image processing can be understood as image processing with relatively low complexity and short processing time, such as object detection, image correction, and target content cropping. For example, in application scenarios such as incorrect question entry, homework correction, and test paper entry, the corresponding first image processing could be book detection, image correction, and question cropping; in document scanning applications, it could be document detection, image correction, and document content cropping; in document entry applications, it could be document detection, image correction, and document cropping; and in product entry applications, it could be product detection, image correction, and product cropping. The total processing time for the first image is controlled between 200 and 500 milliseconds. After the first image processing is completed, the second image is immediately displayed on the target application's display interface.

[0027] In one embodiment, taking the entry of incorrect questions as an example, after obtaining the first image in S11, the first image is sequentially processed by book detection (identifying the workbook page area), image correction (correcting skewed pages), and question cropping (automatically selecting the geometry question the user answered incorrectly). The entire process takes approximately 400 milliseconds. The moment the first image processing is completed, the cropped image (i.e., the image containing the geometry question), i.e., the second image, is immediately displayed on the terminal device screen. The second image contains only the question content itself, without any extra background.

[0028] In one embodiment, a first image is processed to obtain a second image and a corresponding base version marker, such as level 0. Each image processing step completes with a corresponding version marker, providing foundational data for subsequent data entry and display updates. After obtaining the second image, it is displayed on the terminal device's interactive interface. At this point, the current version marker of the currently displayed content is the base version marker level 0. By marking each image processing step with a version marker, subsequent asynchronous background or server-side image processing with higher complexity can also be performed with the same version marker, facilitating orderly updates based on the version markers and improving the overall orderliness and reliability of information display.

[0029] As described above, users can see the cropped and corrected second image (such as an image of a wrong answer) in less than half a second after pressing the camera button, without having to wait for a long time. This extremely fast response eliminates the interface lag and lack of feedback of traditional solutions, significantly improves the display speed of the input results, and makes users feel that the operation is smooth and the response is fast, thus significantly improving the user experience.

[0030] S13. Perform second image processing on the second image to obtain incremental data, wherein the processing complexity of the second image processing is higher than that of the first image processing.

[0031] After obtaining the second image in S12, while displaying the second image, a second image processing step is performed on the second image to obtain incremental data. The complexity of the second image processing is higher than that of the first image processing. The second image processing can be understood as image processing with relatively high complexity and relatively long processing time, such as image enhancement, optical character recognition, classification and tagging, and similar content search and matching. The second image processing can be performed in the background of the terminal device, or the second image can be transmitted to a server, where the server performs the corresponding second image processing. For example, the second image processing may include multiple processing tasks, such as image enhancement, optical character recognition, classification and tagging, and similar content search and matching. Priority levels can be preset for multiple tasks, and the corresponding processing tasks are executed sequentially from high to low priority. Each processing task is executed independently, and each generates corresponding incremental data upon completion. In one embodiment, each incremental data corresponds to a version tag.

[0032] In one embodiment, taking the input of incorrect questions and the execution of second image processing in the background of the terminal device as an example, after obtaining the cropped image (i.e., the second image) in the aforementioned S12, at the same moment the user sees the cropped image on the terminal device (e.g., a smartphone), the terminal device (e.g., the smartphone) performs second image processing on the cropped image (i.e., the second image) to obtain corresponding incremental data. For example, the cropped image (i.e., the second image) is beautified to obtain a third image with clearer text and higher contrast. Then, optical character recognition processing and question classification and labeling processing are performed, and finally, original question search and matching processing is performed. After each processing task is completed, corresponding incremental data is obtained. Each incremental data includes a corresponding version tag, such as level 1, level 2, etc. The version tags can be stacked sequentially according to the completion time, or the corresponding version tags can be marked according to a preset priority. The displayed content can be refreshed according to the incremental data.

[0033] In one embodiment, taking the input of incorrect questions and the server's execution of second image processing as an example, after obtaining the cropped image (i.e., the second image) in S12, at the same moment the user sees the cropped image on their terminal device (e.g., a smartphone), the terminal device (e.g., the smartphone) sends the cropped image (i.e., the second image) to the server. The server performs second image processing on the cropped image (i.e., the second image) to obtain corresponding incremental data. For example, the server performs image enhancement processing on the cropped image (i.e., the second image) to obtain a third image with clearer text and higher contrast, then performs optical character recognition processing and question classification and labeling processing, and finally performs original question search and matching processing. Each completed processing task yields corresponding incremental data, and each incremental data includes a corresponding version tag. Version tags can be added sequentially according to the completion time, or the corresponding version tags can be marked according to a preset priority. The server immediately returns each incremental data to the terminal device, and the terminal device can subsequently refresh the displayed content based on the received incremental data.

[0034] As described above, by placing the complex and time-consuming second image processing in the background of the terminal device or on the server for asynchronous execution, the user interface is not blocked. Users do not need to wait for these processing tasks to complete before proceeding to the next step, thereby improving the efficiency of information entry and thus enhancing the overall user experience.

[0035] S14. Refresh the currently displayed content based on the incremental data to replace the second image.

[0036] After obtaining incremental data through the second image processing in S13, the currently displayed content is refreshed based on the incremental data to replace the second image. For example, after obtaining the incremental data, different refresh mechanisms can be used depending on the data type of the incremental data to gradually replace the currently displayed content. For instance, when the incremental data is an image, a smooth image transition method (such as fade-in / fade-out) is used to replace the second image; when the incremental data is supplementary information, the supplementary information is refreshed to the corresponding target area of ​​the current interface to add relevant supplementary information, such as category tags, similarity matching information, etc. Throughout the process, a version marking mechanism can be used to ensure the correct update order, and an incremental update mechanism can be used to replace only the changed parts without reloading the entire interface.

[0037] In one embodiment, taking the entry of incorrect questions as an example for explanation, after the cropped image (i.e., the second image) is displayed in S12, through the second image processing in S13-S14 and the refresh of incremental data, the cropped image (i.e., the second image) displayed on the terminal device (such as a smartphone) will suddenly become bright and clear in a soft fade-in and fade-out manner (the result of image beautification processing); after a while, the corresponding question text area automatically displays the complete question content (the result of optical character recognition processing); immediately afterward, corresponding tag information appears next to the question content (the result of classification and tagging processing), such as mathematics (subject tag), intermediate (difficulty tag), and proof question (question type tag), etc.; after a while, two similar question recommendations appear at the bottom of the screen (similar to the result of search matching processing). Throughout the entire process, the user does not perform any interactive operations, and the interface content of the terminal device (such as a smartphone) gradually becomes richer, finally presenting complete incorrect question information and recommended content.

[0038] As described above, from taking a photo to seeing the final complete result, the perceived waiting time for the user is only a few hundred milliseconds during the initial image processing. Subsequent incremental page refreshes based on the incremental data after the second image processing make the user feel that the system response is extremely fast and the operation is smooth. At the same time, the quality of the information obtained is the same as that of traditional solutions. The visual effect improves the display speed of the input results, thereby improving the overall user experience.

[0039] As described above, during information entry, the first image is processed to obtain a second image, which is then displayed. Subsequently, the second image undergoes more complex image processing to obtain incremental data, and the currently displayed content is refreshed based on this incremental data. Using this technique, the second image can be obtained through simple first image processing and immediately displayed. Compared to existing methods that require waiting for multiple processing steps to complete before displaying content, this embodiment can quickly display the second image after the first image is entered, significantly improving the speed of information entry result display and thus enhancing the user experience. Furthermore, the second image undergoes more complex image processing to obtain corresponding incremental data, and the currently displayed content is continuously refreshed based on this incremental data. Compared to existing methods that display only after all processing steps are completed, this embodiment, while quickly displaying the second image, also progressively displays the results of each processing step, resulting in a smooth display interface without noticeable lag, significantly improving the display speed of information entry results and further enhancing the user experience.

[0040] Figure 2 This is a flowchart of another information display method provided in the embodiments of this application, referred to as... Figure 2 The specific methods for displaying this information include: S121. Perform target object detection and cropping processing on the first image to obtain the corresponding target region image.

[0041] During the first image processing in S12, the first image can be processed by target object detection and cropping to obtain the corresponding target region image. For example, after acquiring the first image taken by the user, the terminal device can call a locally deployed lightweight target detection model to detect and locate the target object in the first image. The target object detection model performs target object detection processing to obtain the bounding box of the corresponding target object. Based on this bounding box, the target region image is cropped from the first image to filter out background interference. The specific target object can be determined according to the actual application scenario. For example, in application scenarios such as error entry, homework correction, and test paper entry, the first image is a photo of a student's exercise book, notebook, or test paper, and the target object is a page of the book or test paper. The target detection model identifies the four corner points of the page, and the area containing only the page content is cropped to obtain the target region image, thus excluding background elements such as desktop surfaces, fingers, or shadows. In document scanning applications, the first image is a photographed document, such as a contract, invoice, or data. The target object is a rectangular area of ​​the document page. An object detection model identifies this rectangular area, and cropping yields a document image with the background removed—this is the target area image. In document entry applications, the first image is a photograph of an ID card, passport, or driver's license. The target object is the document itself. An object detection model identifies a rectangular area containing the complete document, and cropping yields an image containing only the document—this is the target area image. In product entry applications, the first image is a photograph of a product in an e-commerce setting. The target object is the product itself, such as a mobile phone or clothing. An object detection model identifies a rectangular area containing the product itself, and cropping yields the product area—this is the target area image, removing cluttered backgrounds. In medical record or report entry applications, the first image is a photograph of a medical record or test report. The target object is the paper area of ​​the medical record or report. An object detection model identifies the paper area of ​​the medical record or report, and cropping yields a report image—this is the target area image.

[0042] Figure 3 This is a schematic diagram of a first image provided in an embodiment of this application. Figure 4 This is a schematic diagram of the target area image provided in the embodiments of this application, with reference to... Figure 3-4 The first image 10 is processed by target object detection and cropping to obtain the corresponding target area image 101, so as to filter out the corresponding interference areas such as fingers, desktop and shadows.

[0043] As described above, by performing target object detection and cropping on the first image to obtain the corresponding target region image, a large amount of visual noise unrelated to the target object (such as the content of the question) can be quickly removed, reducing the computational load of subsequent processing.

[0044] S122. Perform image correction processing on the target area image to obtain a corrected image.

[0045] After obtaining the target region image in S121, image correction processing is performed on the target region image to obtain a corrected image, thereby solving the perspective distortion problem caused by the shooting angle. For example, the bounding box of the target object obtained in the target object detection stage (such as the coordinates of the four corner points) can be used, combined with a preset target rectangle, such as the desired book page being a regular rectangle with an aspect ratio of A4 or the actual proportion of an exercise book, to obtain a perspective transformation matrix (such as a 3×3 perspective transformation matrix) by solving a system of perspective transformation equations. Each pixel in the previously cropped target region image is mapped to the new coordinates according to the inverse perspective transformation matrix to generate the corrected image. During the matrix mapping transformation process, bilinear interpolation can be used to ensure image smoothness. For the corrected image, black areas may appear at the edges; these invalid areas can be automatically detected and cropped to obtain the final corrected image.

[0046] In one embodiment, in applications such as incorrect question entry, homework correction, and test paper entry, students often take photos at an angle, resulting in a trapezoidal page shape. Image correction can transform the page into a right-angled rectangle, restoring horizontal text lines and neat page margins, facilitating subsequent question cropping and optical character recognition (OCR) processing. In document scanning applications, photographed contract or book pages may exhibit perspective distortion. After correction, the document page is square, and the text is not skewed, improving the reading and recognition experience. In document entry applications, document photos may be tilted or have some perspective distortion. Correction yields a frontal rectangular document image, conforming to the standard input format for document recognition. In product entry applications, product photos may be taken at an incorrect angle. Correction yields a frontal, well-formed product image, improving display quality and the accuracy of subsequent feature extraction. In medical record or report entry applications, medical records or reports are often on A4 paper, and trapezoidal distortion is common after photography. Correction yields a straight report image, facilitating quick reading by users or the system.

[0047] As mentioned above, image correction significantly improves the accuracy of subsequent target content recognition and second image processing; moreover, the correction process is performed entirely locally without needing to be transmitted to the server, which significantly improves processing speed; in addition, the corrected image is more in line with human eye reading habits, and even if the user only sees the corrected cropped result, they will feel that the image has become straight, improving visual comfort and thus enhancing the user's overall experience during information display.

[0048] S123. Perform target content recognition and cropping processing on the corrected image to obtain a second image containing the target content.

[0049] After obtaining the corrected image in S122, the corrected image undergoes target content recognition and cropping to obtain a second image containing the target content. For example, a lightweight layout analysis algorithm can be used to analyze the layout of the corrected image, locate the target content area, and then determine the cropping boundaries according to preset rules or user interaction, finally cropping out the second image. For instance, in a scenario where incorrect questions are entered, a lightweight layout analysis algorithm is used to project the corrected image horizontally, counting the number of black (or text) pixels in each row to form a projection histogram. The boundaries between questions are found based on the troughs (i.e., blank rows) in the projection histogram. For pages with multiple questions, the start and end rows of each question can be located. Therefore, the target content area can be located. Then, based on the user's operation context, such as a question area manually selected by the user or the system's default cropping of the first question area, or by analyzing the page structure, such as question number identifiers "1.", "2.", etc., the boundaries of the target question are determined. Simultaneously, combined with vertical projection, the coordinates of the top, bottom, left, and right boundaries of the question (usually the area between the left and right margins of the page) are determined. Based on the determined top, bottom, left, and right boundary coordinates, the corresponding rectangular region is cropped from the corrected image to generate the second image. The second image should contain at least one complete question (question stem, options, or graph, etc.) and should minimize interference from adjacent questions.

[0050] In one embodiment, in the application scenario of homework grading, after the teacher photographs the student's homework, they need to crop out the answer area or grading mark area for each question. The system can identify red grading marks, such as "√" or "×", and crop out a second image for each question one by one. In the application scenario of test paper entry, similar to the entry of incorrect questions, but the entire test paper needs to be split into individual questions for entry into the question bank. The system automatically identifies the starting position of each question (such as question number or score indicator) and crops out a second image for each question one by one. In the application scenario of document scanning, for documents with multi-column layout or containing titles, body text, and tables, the system can identify text blocks or paragraphs and crop out the paragraph or entire page content selected by the user as the second image. In the application scenario of document entry, depending on the document type (such as ID card), the system can crop out different functional areas such as the portrait area, name, or number area, and output them as the second image respectively. In product entry scenarios, the system crops the main product image from the corrected product image using edge detection or saliency detection, removes unnecessary packaging, and obtains a clean product image as the second image. In case and report entry scenarios, medical records contain areas such as patient information, examination results, and diagnostic opinions. The system can identify and crop key information areas as the second image, such as the numerical data area of ​​examination results.

[0051] In one embodiment, in the scenario of recording incorrect questions, the user photographs a workbook page containing two math problems, one algebra and one geometry, arranged vertically. After horizontal projection analysis, the corrected image detects a blank area between the two problems. The system automatically identifies the area containing the first algebra problem (located by the problem number "1."). Then, it is cropped according to the boundaries to obtain an image containing only the algebra problem (i.e., the second image). The entire process takes approximately 100 milliseconds.

[0052] As described above, through precise cropping of the target content, the second image seen by the user is an image focused on the target content (such as the question content), eliminating the need for manual adjustments and further reducing redundant information. This allows the user to immediately focus on the current target object (such as a wrong answer or document), providing cleaner input for subsequent second image processing. The second image is displayed immediately after cropping; the time from taking the photo to seeing a clear, upright, and focused second image is only a few hundred milliseconds, significantly improving the display speed of the input results.

[0053] As described above, the first image processing is performed by a local lightweight model, which can be implemented and run on terminal devices (such as smartphones) without relying on cloud computing power, supports offline use, and enables rapid input. The processing time of the first image is controlled within a few hundred milliseconds (e.g., 500 milliseconds), ensuring the immediate display of the second image and significantly shortening the user's perception time.

[0054] Figure 5 This is a flowchart of another information display method provided in the embodiments of this application, referred to... Figure 5 The specific methods for displaying this information include: S21. Upon receiving the first trigger operation, acquire the first image.

[0055] The specific implementation method of this step is the same as that of S11 mentioned above, and will not be repeated here.

[0056] S22. Perform first image processing on the first image to obtain the second image, and display the second image.

[0057] The specific implementation method of this step is the same as that of S12 mentioned above, and will not be repeated here.

[0058] S23. Perform image enhancement processing on the second image to obtain the third image.

[0059] After obtaining the second image and its corresponding base version marker in S22, the second image is displayed, and image enhancement processing is performed on the second image to obtain a third image and a first version marker. This image enhancement processing includes a series of processes such as shadow removal, contrast enhancement, brightness adjustment, noise reduction, sharpening, and white balance correction. The first version marker has a higher version than the base version marker; for example, the first version marker is denoted as level 1, and the base version marker as level 0. This step is one of the processing procedures in the aforementioned second image processing, and the resulting third image and first version marker constitute the first incremental data. For example, the terminal device executes the aforementioned S22 to obtain the second image and displays it on the corresponding interactive interface. Simultaneously, it can perform image enhancement processing on the second image in the background to obtain the first incremental data, which includes the third image and the first version marker level 1. Subsequent page refreshes can be based on this first incremental data. In one embodiment, the terminal device executes the aforementioned S22 to obtain a second image and displays the second image on the corresponding interactive interface. Simultaneously, the second image can be transmitted to the server. The server performs image enhancement processing on the second image to obtain first incremental data, which includes a third image and a first version marker level 1. After receiving the first incremental data, the server returns it to the terminal device, which can then refresh the page based on this first incremental data.

[0060] S24. Replace the currently displayed second image with the third image using a smooth image transition method.

[0061] After obtaining the third image and the first version marker in S23, the terminal device replaces the currently displayed second image with the third image using a smooth image transition method. This smooth image transition method includes fade-in / fade-out animations, cross-dissolve, or scaling transitions to avoid abrupt image jumps perceived by the user. For example, after the terminal device's backend or server performs image enhancement processing on the second image to obtain the first incremental data, the terminal device parses this first incremental data to determine its corresponding target version marker, which is now the first version marker level 1. This is compared with the current version marker of the currently displayed content. As mentioned earlier, the currently displayed image is the second image, therefore the current version marker of the currently displayed content is level 0. Since the target version marker level 1 based on the first incremental data is higher than the current version marker level 0 of the currently displayed content, a corresponding refresh mechanism is used to refresh the first incremental data to the current interface. For example, identifying the data type of the first incremental data, which is an image (i.e., the first incremental data includes a third image and a first version marker), a smooth image transition can be used to replace the second image with the third image to refresh the currently displayed content. Therefore, the currently displayed content changes to the third image, and the current version marker of the currently displayed content is updated to the first version marker, i.e., level 1. This process, based on refreshing the interface using a smooth image transition, means that the user is unaware of or only perceives that the image quality has naturally improved.

[0062] In one embodiment, in the application scenario of incorrect question entry, a user (such as a student) takes a picture of a geometry problem they got wrong using a terminal device (such as a smartphone). After the terminal device performs a first image processing (i.e., fast processing), the screen displays a cropped original image (i.e., the second image). This image may be dark due to insufficient lighting at the time of shooting, or may contain shadows from books or fingers, but the question content is clearly legible. After receiving the second image, the terminal device's backend or server sequentially performs a series of image enhancement processes, including automatic white balance, contrast enhancement, shadow removal, and sharpening. After processing, a third image (i.e., the enhanced image of the incorrect question) and a first version label (level 1) are generated. The automatic white balance processing corrects for yellowish or bluish color temperatures. The contrast enhancement processing stretches the histogram, making the text darker and the background whiter. The shadow removal processing uses a preset algorithm or deep learning model to eliminate large areas of shadow on the book page caused by the shooting angle. The sharpening processing slightly sharpens the edges of the text, improving the recognition rate of subsequent optical character recognition. After receiving the third image, the terminal device detects its target version marker. It then detects that this is an upgraded image, meaning the target version marker (currently level 1, the first version marker) is higher than the current version marker (level 0, the base version marker) of the currently displayed second image. The terminal device creates an alpha (transparency) animation in the image display area, causing the transparency of the second image to gradually decrease from 1 to 0, while the transparency of the third image gradually increases from 0 to 1. This animation lasts for 300 milliseconds. After the animation ends, the third image completely replaces the second image. For the user, the incorrect question image appears to smoothly transition from slightly dark and shadowy to bright and clear in approximately 0.3 seconds, as if the lighting has been naturally adjusted without flickering or abrupt changes.

[0063] As described above, by placing the complex and time-consuming image enhancement processing in the background of the terminal device or on the server side, the user interface is not blocked. Users can observe the second image on the interface while waiting for the result of the second image processing to be displayed, or they can immediately perform the next operation after the second image is displayed. The image enhancement processing and other second image processing performed in the background or on the server side are unaffected, thereby improving the efficiency of information entry. Moreover, the traditional method of directly cutting the image can make users notice the sudden change in the image, which may create a bad impression of system lag. However, this embodiment uses smooth image transitions such as fade-in and fade-out to refresh the interface, which can eliminate the visual abruptness. Users only perceive that the image quality has improved naturally, which enhances the user's visual experience. In addition, the perceived waiting time from taking a picture to seeing the complete result is only a few hundred milliseconds of the initial execution of the first image processing. The subsequent third image, which is based on the image enhancement processing, replaces the second image with a smooth image transition, without requiring the user to wait for the image enhancement processing time. The perceived waiting time is greatly shortened, thus significantly improving the speed of information display from a visual perspective, and thus significantly improving the user's visual experience and overall operating experience.

[0064] Figure 6 This is a flowchart of another information display method provided in the embodiments of this application, referred to as... Figure 6 The specific methods for displaying this information include: S31. Upon receiving the first trigger operation, acquire the first image.

[0065] The specific implementation method of this step is the same as that of S11 mentioned above, and will not be repeated here.

[0066] S32. Perform first image processing on the first image to obtain the second image, and display the second image.

[0067] The specific implementation method of this step is the same as that of S12 mentioned above, and will not be repeated here.

[0068] S33. Perform image enhancement processing on the second image to obtain the third image.

[0069] The specific implementation method of this step is the same as that of S23 mentioned above, and will not be repeated here.

[0070] S34. Replace the currently displayed second image with the third image using a smooth image transition method.

[0071] The specific implementation method of this step is the same as that of S24 mentioned above, and will not be repeated here.

[0072] S35. Perform optical character recognition processing on the third image to obtain the text content.

[0073] After obtaining the first incremental data in the aforementioned S34, that is, after obtaining the third picture and the corresponding first version mark, the terminal device replaces the currently displayed second picture with the third picture in an image smooth transition manner on the display interface (i.e., the step of S34 is executed). At this time, the terminal device background or the server performs optical character recognition processing on the third picture to obtain the text content and the corresponding second version mark. Among them, the version of the second version mark is higher than that of the first version mark. For example, the first version mark is denoted as level 1, and the second version mark is denoted as level 2. This step belongs to one of the processing processes in the aforementioned second image processing, and the corresponding text content and second version mark obtained are the second incremental data. Exemplarily, the terminal device executes the aforementioned S32 to obtain the second picture, and displays the second picture on the corresponding interaction interface. After that, the terminal device background or the server completes the picture beautification processing to obtain the first incremental data (i.e., the third picture and the first version mark). The terminal device replaces the second picture with the third picture (i.e., the picture after beautification) in an image smooth transition manner based on the first incremental data. At this time, the clear and bright third picture is displayed on the interaction interface of the terminal device, but the corresponding preset text area (such as the title text area) is still blank or displays the corresponding placeholder (such as "Recognizing the title..."). The terminal device background performs optical character recognition processing on the third picture to obtain the corresponding second incremental data, and the second incremental data includes the text content and the second version mark level 2. Alternatively, the server performs optical character recognition processing on the third picture to obtain the corresponding second incremental data, and the second incremental data includes the text content and the second version mark level 2; the server returns the second incremental data to the terminal device.

[0074] In one embodiment, in the wrong question input scenario, after completing the picture beautification processing to obtain the third picture, image preprocessing is performed on the third picture to convert the third picture into a grayscale image, and binaryzation processing is performed to remove noise points to obtain a to-be-recognized image. The text blocks in the to-be-recognized image are recognized, and regions such as the question stem, options, and graphic annotations are distinguished. The recognized text blocks are segmented into single characters, and the characters are recognized through a preset deep learning model to output a candidate text sequence. Then, a preset language model is used to correct the recognition result (i.e., the candidate text sequence). For example, "解万程" is corrected to "解方程". After the correction is completed, the final text content is obtained. It should be noted that the finally obtained recognition result includes information such as the complete question text, the coordinate positions of each text area, and the recognition confidence. In the embodiment, at least the complete question stem and option text are recognized as the text content to be displayed subsequently.

[0075] S36. Refresh the text content to the preset text area of the current display interface through a preset display component refresh mechanism for display.

[0076] After obtaining the second incremental data in step S35, the text content is refreshed to the preset text area of ​​the current display interface through a preset display component refresh mechanism. For example, after the terminal device or server performs optical character recognition (OCR) on the third image to obtain the second incremental data, the terminal device parses the second incremental data to determine its corresponding target version marker, which is now the second version marker level 2. This target version marker is compared with the current version marker of the currently displayed content. As mentioned earlier, the currently displayed image is the third image, therefore the current version marker of the currently displayed content is level 1. Since the target version marker level 2 based on the second incremental data is higher than the current version marker level 1 of the currently displayed content, the text content is refreshed to the preset text area of ​​the current display interface through the preset display component refresh mechanism. Then, the current version marker of the currently displayed content is updated to the second version marker, i.e., level 2.

[0077] In one embodiment, during the refresh process, the attributes of the text display component are pre-bound to a first field in a preset responsive data model in the interface layout file. After obtaining the text content, the text content can be written into the first field of the responsive data model. The responsive data model triggers an attribute change event for the first field to display the corresponding text content in a preset text area on the display interface. From the user's perspective, the user can see that the originally blank or "Identifying questions..." text area instantly displays complete text content.

[0078] Figure 7 This is a first schematic diagram of a display interface provided in an embodiment of this application. Figure 8 This is a second schematic diagram of a display interface provided in an embodiment of this application. After obtaining the third image through the aforementioned implementation method, refer to... Figure 7The third image 201 is displayed on the display interface 20, and "Recognizing the question..." is displayed in the corresponding preset text area 202. At this time, the terminal device's backend performs optical character recognition (OCR) processing on the third image 201 to obtain the corresponding second incremental data. This second incremental data includes text content and a second version marker level 2. Assume the recognized text content is "Question 1: 2x + 3 = 7, find the value of x". The terminal device parses this second incremental data to determine its corresponding target version marker, which is now the second version marker level 2. This is compared with the current version marker of the currently displayed content. As mentioned earlier, the currently displayed image is the third image 201, therefore the current version marker of the currently displayed content is level 1. Since the target version marker level 2 based on the second incremental data is higher than the current version marker level 1 of the currently displayed content, the text content is refreshed to the preset text area 202 of the current display interface 20 through a preset display component refresh mechanism. Afterwards, the current version marker of the currently displayed content is updated to the second version marker, i.e., level 2. (Refer to...) Figure 8 The refreshed display interface 20 shows the third image 201, and the preset text area 202 displays the text content "Question 1: 2x + 3 = 7, find the value of x".

[0079] As described above, by placing the complex and time-consuming image enhancement and optical character recognition (OCR) processing in the background or on the server side of the terminal device, the user interface is not blocked. Users can observe the second image on the interface while waiting for the result of the second image processing to be displayed, or they can immediately proceed to the next operation after the second image is displayed. The image enhancement and OCR processing performed in the background or on the server side are unaffected, thereby improving the efficiency of information entry. Moreover, the OCR processing takes only a few milliseconds, and the corresponding text content can be displayed on the interface within milliseconds. Since there is no animation delay, users can see the text content almost instantly after the task is completed in the background / on the server side. The perceived waiting time is only tens of milliseconds of network transmission, which significantly improves the speed of information display and thus shortens the user's waiting time visually. In addition, after displaying the third image after image enhancement, the corresponding text content is also displayed. Users can see the clear third image first, and then see the automatically appearing text content. The information is gradually enriched, achieving hierarchical display, which conforms to the cognitive rhythm. Compared with the traditional method of displaying all content at once, this progressive presentation allows users to focus more on the information at each step and increases the attention to each content in the information display result.

[0080] In one embodiment, when there is a large amount of identified text content, the height of the text area can be automatically expanded in conjunction with a scrolling view. If the second incremental data is structured data, such as JSON data containing information such as the question stem, options, and answers, it can be refreshed separately through multiple preset text areas, for example, displaying options A, B, C, and D in four different controls.

[0081] Figure 9 This is a flowchart of another information display method provided in the embodiments of this application, referred to... Figure 9 The specific methods for displaying this information include: S41. Upon receiving the first trigger operation, acquire the first image.

[0082] The specific implementation method of this step is the same as that of S11 mentioned above, and will not be repeated here.

[0083] S42. Perform a first image processing on the first image to obtain a second image, and display the second image.

[0084] The specific implementation method of this step is the same as that of S12 mentioned above, and will not be repeated here.

[0085] S43. Perform image enhancement processing on the second image to obtain the third image.

[0086] The specific implementation method of this step is the same as that of S23 mentioned above, and will not be repeated here.

[0087] S44. Replace the currently displayed second image with the third image using a smooth image transition method.

[0088] The specific implementation method of this step is the same as that of S24 mentioned above, and will not be repeated here.

[0089] S45. Classify and label the third image to obtain tag information.

[0090] After obtaining the first incremental data in S43, i.e., after obtaining the third image and its corresponding first version tag, the terminal device's display interface uses a smooth image transition method to replace the currently displayed second image with the third image (i.e., step S44 is executed). At this time, the terminal device's backend or server performs classification and tagging processing on the third image to obtain tag information and the corresponding third version tag. The third version tag has a higher version than the second version tag; for example, the second version tag is denoted as level 2, and the third version tag is denoted as level 3. The tag information includes category tags, such as subject tags, department tags, and document type tags, etc. For example, in the application scenario of incorrect question entry, the tag information can be subject tags, difficulty tags, knowledge point tags, and question type tags, etc. This step belongs to one of the processing procedures in the aforementioned second image processing, and the corresponding tag information and third version tag obtained constitute the third incremental data. For example, the terminal device executes the aforementioned S42 to obtain a second image and displays it on the corresponding interactive interface. Then, the terminal device's backend or server performs image enhancement processing to obtain first incremental data (i.e., a third image and a first version tag). Based on the first incremental data, the terminal device uses an image smoothing transition method to replace the second image with the third image (i.e., the enhanced image). The terminal device's backend performs optical character recognition (OCR) processing on the third image to obtain corresponding second incremental data, which includes text content and a second version tag level 2. Simultaneously, the terminal device's backend or server asynchronously performs classification and tagging processing on the third image to obtain third incremental data, which includes tag information and a third version tag. That is, the terminal device's backend or server performs OCR and classification / tag processing simultaneously and asynchronously. Since OCR processing takes approximately 1 second and classification / tag processing takes approximately 1.5 seconds, OCR processing completes before classification / tag processing, meaning the terminal device's backend or server obtains the corresponding second incremental data first. Therefore, the terminal device parses the second incremental data to determine its corresponding target version marker, which is now the second version marker level 2. This is compared with the current version marker of the currently displayed content. As mentioned earlier, the currently displayed image is the third image, so its current version marker is level 1. Since the target version marker level 2 based on the second incremental data is higher than the current version marker level 1 of the currently displayed content, the text content is refreshed to the preset text area of ​​the current display interface using a preset display component refresh mechanism. Then, the current version marker of the currently displayed content is updated to the second version marker, i.e., level 2. Afterward, the corresponding classification and tagging process is completed, resulting in the third incremental data, which includes tag information and the third version marker.

[0091] In one embodiment, in the application scenario of incorrect question input, after the image enhancement process is completed to obtain the third image, the terminal device backend or server performs optical character recognition (OCR) processing on the third image through the aforementioned S35, while simultaneously extracting features from the third image, such as layout, font, and formula features, and parsing keywords and question type identifiers in the text content (obtained through OCR). Based on the extracted features and the parsed keywords and question type identifiers, multi-dimensional matching is performed, matching against a preset subject database, difficulty database, knowledge point database, and question type database to determine the corresponding category. The matching results are integrated to generate structured tag information, such as subject tags, difficulty tags, knowledge point tags, and question type tags.

[0092] S46. The label information is refreshed to the preset label area of ​​the current display interface through the preset display component refresh mechanism.

[0093] After obtaining the third incremental data in step S45, the tag information is refreshed to the preset tag area of ​​the current display interface through a preset display component refresh mechanism. For example, after the terminal device or server performs classification and tagging processing on the third image to obtain the third incremental data, the terminal device parses the third incremental data to determine its corresponding target version tag. At this time, the target version tag is the third version tag level 3. This is compared with the current version tag of the currently displayed content. As mentioned earlier, the current version tag of the currently displayed content is level 2. Since the target version tag level 3 based on the third incremental data is higher than the current version tag level 2 of the currently displayed content, the tag information is refreshed to the preset tag area of ​​the current display interface through the preset display component refresh mechanism. Afterwards, the current version tag of the currently displayed content is updated to the third version tag, i.e., level 3.

[0094] In one embodiment, during the refresh process, the attributes of the label display component are pre-bound to a second field in a preset responsive data model in the interface layout file. After obtaining the label information, the label information can be written into the second field of the responsive data model. The responsive data model triggers an attribute change event for the second field to display the corresponding label information in the preset label area of ​​the display interface. For users with visual perception, they can see that the originally blank or text area displaying "Identifying Classification..." instantly displays relevant label information, such as labels like "Mathematics," "Intermediate," and "Word Problems."

[0095] Figure 10 This is a third schematic diagram of a display interface provided in an embodiment of this application, referring to... Figure 7-8 as well as Figure 10 After obtaining the third image through the aforementioned implementation method, refer to... Figure 7 The third image 201 is displayed on the display interface 20, and "Identifying the question..." is displayed in the corresponding preset text area 202, and "Identifying the category..." is displayed in the corresponding preset label area 203. At this time, the terminal device's backend or server performs optical character recognition processing on the third image 201 to obtain the corresponding second incremental data. This second incremental data includes text content and a second version marker level 2. Assume the identified text content is "Question 1: 2x + 3 = 7, find the value of x". The terminal device parses the second incremental data to determine its corresponding target version marker, which is now the second version marker level 2. This is compared with the current version marker of the currently displayed content. As mentioned above, the currently displayed image is the third image 201, therefore the current version marker of the currently displayed content is level 1. Since the target version marker level 2 based on the second incremental data is higher than the current version marker level 1 of the currently displayed content, the text content is refreshed to the preset text area 202 of the current display interface through a preset display component refresh mechanism. (Refer to...) Figure 8 The refreshed display shows a third image 201, and the text "Problem 1: 2x + 3 = 7, find the value of x" is displayed in the preset text area 202. The terminal device's backend or server performs classification and labeling on the third image 201 to obtain third incremental data. This third incremental data includes tag information and a third version marker level 3. Assume the obtained tag information is "Mathematics," "Intermediate," and "Word Problem." The terminal device parses this third incremental data to determine its corresponding target version marker, which is now a third version marker level 3. This is compared with the current version marker of the currently displayed content. As mentioned earlier, the current version marker of the currently displayed content is level 2. Since the target version marker level 3 based on the third incremental data is higher than the current version marker level 2 of the currently displayed content, the tag information is refreshed to the preset tag area 203 of the current display interface 20 through the preset display component refresh mechanism. Afterwards, the current version marker of the currently displayed content is updated to the third version marker, i.e., level 3. (Refer to...) Figure 10 The refreshed display interface 20 shows the third image 201, and the preset text area 202 shows the text content "Problem 1: 2x+3=7, find the value of x". At the same time, the preset label area 203 shows three label information: "Mathematics", "Intermediate" and "Word Problem".

[0096] As described above, by offloading the complex and time-consuming image enhancement, optical character recognition, and classification labeling processes to the background or server of the terminal device, the user interface is not blocked. Users can observe the second image on the interface while waiting for the results of the second image processing to be displayed, or they can immediately proceed to the next step after the second image is displayed. The image enhancement, optical character recognition, and classification labeling processes performed in the background or on the server remain unaffected, thereby improving the efficiency of information entry. Moreover, traditional information entry (such as entering incorrect questions) requires users to manually classify information, such as manually selecting subjects, difficulty levels, and question types, which can take tens of seconds or even a minute. This embodiment can automatically complete the classification and labeling. Users only need to take a photo, and the system can intelligently identify the corresponding tags and fill them into the interactive interface, greatly saving users' time and significantly improving the automation and intelligence of information display, enriching the content dimensions of information display, and thus enhancing the overall user experience.

[0097] In one embodiment, different label colors can be configured according to different category labels. For example, different subject or difficulty levels can be assigned different label colors, such as green for mathematics and blue for physics, thereby improving visual recognition. After the corresponding label information is displayed on the interactive interface, if the user believes that the system's automatic labeling is inaccurate, they can click on the label area to manually modify it. The modified label information will be saved and used for subsequent model optimization. When dealing with complex content, multi-level labels can be generated. For example, if a complex math problem is identified, multi-level labels such as "Mathematics - Algebra - Linear Equation in One Variable" can be generated.

[0098] In one embodiment, the processing model for classification and labeling is updated online using the user's correction data for automatic labeling as feedback data, so that the label information obtained from subsequent classification is more in line with the user's personal usage habits.

[0099] Figure 11 This is a flowchart of another information display method provided in the embodiments of this application, referred to as... Figure 11 The specific methods for displaying this information include: S51. Upon receiving the first trigger operation, acquire the first image.

[0100] The specific implementation method of this step is the same as that of S11 mentioned above, and will not be repeated here.

[0101] S52. Perform first image processing on the first image to obtain the second image, and display the second image.

[0102] The specific implementation method of this step is the same as that of S12 mentioned above, and will not be repeated here.

[0103] S53. Perform image enhancement processing on the second image to obtain the third image.

[0104] The specific implementation method of this step is the same as that of S23 mentioned above, and will not be repeated here.

[0105] S54. Replace the currently displayed second image with the third image using a smooth image transition method.

[0106] The specific implementation method of this step is the same as that of S24 mentioned above, and will not be repeated here.

[0107] S55. Perform optical character recognition processing on the third image to obtain the text content.

[0108] The specific implementation method of this step is the same as that of S35 mentioned above, and will not be repeated here.

[0109] S56. The text content is refreshed to the preset text area of ​​the current display interface through the preset display component refresh mechanism.

[0110] The specific implementation method of this step is the same as that of S36 mentioned above, and will not be repeated here.

[0111] S57. Based on the text content, perform similar content search processing in the preset database to obtain matching information.

[0112] After obtaining the text content in step S55, the terminal device's backend or server performs a similar content search in a preset database based on the text content to obtain matching information and a fourth version tag. The fourth version tag is higher than the third version tag; for example, the fourth version tag is denoted as level 4, and the third version tag as level 3. This step is one of the processing procedures in the aforementioned second image processing, and the obtained matching information and fourth version tag constitute the fourth incremental data. The matching information can be understood as content in the preset database that is similar to or identical to the text content. For example, while the third image undergoes optical character recognition processing in step S55, the terminal device's backend or server asynchronously performs classification and tagging processing on the third image (i.e., step S45). In other words, after the image enhancement processing of the second image in S53 to obtain the third image, the terminal device's backend or server will simultaneously and asynchronously perform optical character recognition processing (i.e., step S55) and classification tagging processing (i.e., step S45). Since the optical character recognition processing takes approximately 1 second and the classification tagging processing takes approximately 1.5 seconds, the optical character recognition processing will complete before the classification tagging processing. That is, the terminal device's backend or server will obtain the corresponding second incremental data first. Therefore, the terminal device parses the second incremental data to determine its corresponding target version marker, which is now the second version marker level 2. This is compared with the current version marker of the currently displayed content. As mentioned above, the currently displayed image is the third image, so the current version marker of the currently displayed content is level 1. Since the target version marker level 2 based on the second incremental data is higher than the current version marker level 1 of the currently displayed content, the text content is refreshed to the preset text area of ​​the current display interface through a preset display component refresh mechanism. Afterwards, the current version marker of the currently displayed content is updated to the second version marker, i.e., level 2. At this time, the corresponding classification and tagging process is still in progress. The terminal device's backend or server performs a similar content search in a preset database based on the aforementioned text content. This similar content search process takes approximately 1.84 seconds. Therefore, the classification and tagging process completes before the similar content search process. Thus, the terminal device's backend or server completes the classification and tagging process to obtain the third incremental data. The terminal device parses this third incremental data to determine its corresponding target version tag, which is now the third version tag level 3. This is compared with the current version tag of the currently displayed content. As mentioned earlier, the current version tag of the currently displayed content is level 2. Since the target version tag level 3 based on the third incremental data is higher than the current version tag level 2 of the currently displayed content, the tag information is refreshed to the preset tag area of ​​the current display interface through a preset display component refresh mechanism.Next, the current version tag of the currently displayed content is updated to the third version tag, i.e., level 3. Afterwards, the similar content search processing is completed, yielding the fourth incremental data, which includes matching information and the fourth version tag, level 4.

[0113] In one embodiment, in the application scenario of incorrect question input, after obtaining the text content through optical character recognition (OCR), similar content search processing is performed based on the text content, i.e., similar question retrieval processing, which can employ a deduplication retrieval strategy. For example, core keywords such as speed, distance, and time can be extracted from the text content and a Boolean search can be performed in a preset question bank to obtain the corresponding search results, i.e., similar question information. Alternatively, the text content (i.e., the question text) can be converted into an embedding vector through a preset conversion model, and then the top N questions with the highest pre-defined similarity can be retrieved from a preset vector database, using these questions as search results. Alternatively, based on the aforementioned tag information, question type or knowledge point matching can be performed to obtain questions with the same question type or the same knowledge point, using these questions as search results. The search results are sorted from high to low similarity, and duplicate or overly simple questions are filtered out. Finally, a preset number of questions are selected as the final matching information. For example, for each matched original question, its complete content, standard answer, solution steps, and knowledge point explanation are extracted and assembled into a structured matching information data package for subsequent display and refresh.

[0114] S58. The matching information is refreshed to the preset matching information area of ​​the current display interface through the preset display component refresh mechanism.

[0115] After obtaining the matching information (i.e., the fourth incremental data) in S57, the matching information is refreshed to the preset matching information area of ​​the current display interface through a preset display component refresh mechanism. For example, after the terminal device's backend or server performs a similar content search in a preset database based on the text content to obtain the fourth incremental data, the terminal device parses the fourth incremental data to determine its corresponding target version marker. At this time, the target version marker is the fourth version marker level 4. This is compared with the current version marker of the currently displayed content. As mentioned earlier, the current version marker of the currently displayed content is level 3. Since the target version marker level 4 based on the fourth incremental data is higher than the current version marker level 3 of the currently displayed content, the matching information is refreshed to the preset matching information area of ​​the current display interface through the preset display component refresh mechanism. Afterwards, the current version marker of the currently displayed content is updated to the fourth version marker, i.e., level 4.

[0116] During the refresh process, the properties of the matching display components are pre-bound to the third field of a preset responsive data model in the interface layout file. After obtaining the matching information, this information is written to the third field of the responsive data model. The responsive data model triggers a property change event for the third field to display the corresponding matching information in the preset matching information area of ​​the display interface. For users with visual perception, the previously blank or "Searching for similar questions..." matching information area suddenly displays a list of similar question entries, each containing a question summary, answer, and explanation hints. Users can obtain value-added learning content without any user interaction.

[0117] Figure 12 This is a fourth schematic diagram of a display interface provided in an embodiment of this application, referring to... Figure 7-8 , Figure 10 as well as Figure 12 After obtaining the third image through the aforementioned implementation method, refer to... Figure 7 The third image 201 is displayed on the display interface 20, and "Identifying the question..." is displayed in the corresponding preset text area 202, "Identifying the category..." is displayed in the corresponding preset tag area 203, and "Searching for similar questions..." is displayed in the corresponding preset matching information area 204. At this time, the terminal device backend or server performs optical character recognition processing on the third image 201 to obtain the corresponding second incremental data. The second incremental data includes the text content and the second version tag level 2. Assume that the identified text content is "Question 1: 2x + 3 = 7, find the value of x". The terminal device parses the second incremental data to determine its corresponding target version marker, which is now the second version marker level 2. This is compared with the current version marker of the currently displayed content. As mentioned earlier, the currently displayed image is the third image 201, therefore the current version marker of the currently displayed content is level 1. Since the target version marker level 2 based on the second incremental data is higher than the current version marker level 1 of the currently displayed content, the text content is refreshed to the preset text area 202 of the current display interface using a preset display component refresh mechanism. (Refer to...) Figure 8The refreshed display shows a third image 201, and the text "Problem 1: 2x + 3 = 7, find the value of x" is displayed in the preset text area 202. The terminal device's backend or server performs classification and labeling on the third image 201 to obtain third incremental data. This third incremental data includes tag information and a third version marker level 3. Assume the obtained tag information is "Mathematics," "Intermediate," and "Word Problem." The terminal device parses this third incremental data to determine its corresponding target version marker, which is now a third version marker level 3. This is compared with the current version marker of the currently displayed content. As mentioned earlier, the current version marker of the currently displayed content is level 2. Since the target version marker level 3 based on the third incremental data is higher than the current version marker level 2 of the currently displayed content, the tag information is refreshed to the preset tag area 203 of the current display interface through the preset display component refresh mechanism. Afterwards, the current version marker of the currently displayed content is updated to the third version marker, i.e., level 3. (Refer to...) Figure 10 The refreshed display shows a third image 201, and the text "Problem 1: 2x + 3 = 7, find the value of x" is displayed in the preset text area 202. Simultaneously, the preset tag area 203 displays three tags: "Mathematics," "Intermediate," and "Word Problem." The terminal device's backend or server performs a similar content search in a preset database based on the text content to obtain fourth incremental data. This fourth incremental data includes matching information and a fourth version marker (level 4). Assume the obtained matching information consists of multiple similar question entries, namely question entry 1, question entry 2, and question entry 3. The terminal device parses this fourth incremental data to determine its corresponding target version marker, which is now the fourth version marker (level 4). This is compared with the current version marker of the currently displayed content. As mentioned earlier, the current version marker of the currently displayed content is level 3. Since the target version marker (level 4) based on the fourth incremental data is higher than the current version marker (level 3) of the currently displayed content, the matching information is refreshed to the preset matching information area 204 of the current display interface 20 through a preset display component refresh mechanism. Then, update the current version marker of the currently displayed content to the fourth version marker, i.e., level 4. (See reference...) Figure 12 The refreshed display interface 20 shows the third image 201, the preset text area 202 shows the text content "Problem 1: 2x + 3 = 7, find the value of x", the preset tag area 203 shows the three tags "Mathematics", "Intermediate" and "Word Problems", and the preset matching information area 204 shows "Recommended similar questions: Question 1; Question 2; Question 3."

[0118] As described above, by offloading the complex and time-consuming image enhancement, optical character recognition, classification and tagging, and similar content search processes to the background or server of the terminal device, the user interface is not blocked. Users can observe the second image on the interface while waiting for the results of the second image processing to be displayed, or they can immediately proceed to the next operation after the second image is displayed. The image enhancement, optical character recognition, classification and tagging, and similar content search processes performed in the background or on the server remain unaffected, thereby improving the efficiency of information entry and enabling uninterrupted continuous entry throughout the entire process, thus enhancing the smoothness of entry. Moreover, the matching information is only refreshed to the preset matching information area in the display interface, without altering the display of the aforementioned third image, text information, and tag information. Visually, there is no flickering or jump, and the user's visual experience is consistent, avoiding the lag caused by global refreshes, thus significantly improving the user's visual experience in the overall information display process. In addition, the information display progresses from the image (i.e., the third image) to the text content to the tag information to the matching information, from basic to extended, which conforms to the user's viewing habits, avoids information overload at one time, and improves user comfort.

[0119] In one embodiment, each incremental data obtained through the aforementioned embodiments includes a corresponding target version tag. For example, the first incremental data includes a first version tag, the second incremental data includes a second version tag, the third incremental data includes a third version tag, and the fourth incremental data includes a fourth version tag. When refreshing the currently displayed content based on the incremental data, the current version tag of the currently displayed content can be obtained. If the target version tag is higher than the current version tag, the currently displayed content is refreshed based on the incremental data, and the current version tag is updated. As mentioned above, by binding a target version tag to each level of incremental data, the terminal device compares the current version tag and refreshes only when the target version tag is higher, ensuring that the image (i.e., the third image), text content, tag information, and matching information are presented progressively from low to high, ensuring the orderly display logic. Furthermore, background or server asynchronous tasks do not need to strictly synchronize the timing, and data can still be pushed normally when there are network fluctuations or delays. The terminal device automatically adapts to out-of-order scenarios through the version mechanism, and can still stably display the highest-level result in a weak network environment, improving system robustness.

[0120] In one embodiment, after displaying the second image in S12, the next information entry operation can be performed. For example, upon receiving a second trigger operation, a fourth image is obtained. The second trigger operation can be understood as a re-execution of the trigger information entry operation, such as clicking the "Enter" or "Upload" control; this click operation is the second trigger operation. The specific execution method for obtaining the fourth image is the same as the execution method for obtaining the first image in S11, and will not be repeated here. After obtaining the fourth image, the fourth image undergoes first image processing to obtain a fifth image, which is then displayed. The fifth image undergoes second image processing to obtain incremental data, and the currently displayed content is refreshed based on the incremental data. After obtaining the fourth image, the same steps as described above are performed to achieve the corresponding information entry and display for the fourth image. It should be noted that, based on the execution of the second trigger operation, the incremental data obtained from the second image processing performed on the second image will not be displayed on the interactive interface. However, the corresponding display content is refreshed in the background of the terminal device. Later, when the information (such as the title) needs to be viewed, the corresponding third image, text content, tag information, and matching information are displayed in the corresponding interactive interface. As mentioned above, compared to the traditional method that requires waiting for all processing to be completed before the next information (such as question information) can be entered, this embodiment obtains and displays the second image through rapid first image processing. Then, the corresponding background or server performs second image processing on the currently entered information. At the same time, the user can immediately initiate the next entry operation based on the interactive interface without waiting for the second image processing of the currently entered information to be completed. Therefore, when entering multiple pieces of information (such as N questions) consecutively, the total perceived waiting time is greatly shortened. The waiting time is completely decoupled from the background processing time, which greatly improves the efficiency of information entry.

[0121] In one embodiment, in the application scenario of incorrect question entry, in traditional solutions, users face a long period of unresponsive interface after each entry, unable to perform any operations, resulting in a sense of stagnation and anxiety, and forcibly interrupting the operation rhythm. In this embodiment, after the user clicks the entry button, a clear cropped image (i.e., the second image) is displayed in less than half a second. At this point, the interface provides clear visual feedback, and the user can immediately turn the page, take a picture of the next question, or perform other operations. The entire entry process remains smooth and continuous, conforming to natural human operating habits. Moreover, since users can start entering the next question before the background task of the previous question is completed, the background processing tasks of multiple questions can be executed in parallel on the server side. That is, each question has its own independent image enhancement processing, optical character recognition processing, classification and tagging processing, and similar content search processing. Compared with the traditional solution, which requires waiting for each question to be completed before starting the next question, the concurrent processing capability of this embodiment enables the system to serve more entry requests in the same amount of time, improving server resource utilization and overall throughput.

[0122] In one embodiment, the continuous data entry mechanism provided in this embodiment can be applied to various application scenarios such as incorrect question entry, homework correction, test paper entry, document scanning, product listing, and medical record entry.

[0123] Based on the above embodiments, Figure 13 This is a schematic diagram of the structure of an information display device provided in an embodiment of this application. (Reference) Figure 13 The information display device provided in this embodiment specifically includes: an image acquisition module 21, a first processing module 22, a display module 23, a second processing module 24, and a refresh module 25.

[0124] The image acquisition module 21 is used to acquire the first image upon receiving the first trigger operation. The first processing module 22 is used to perform a first image processing on the first image to obtain a second image; Display module 23 is used to display the second image; The second processing module 24 is used to perform second image processing on the second image to obtain incremental data, wherein the processing complexity of the second image processing is higher than that of the first image processing. The refresh module 25 is used to refresh the currently displayed content based on incremental data to replace the second image.

[0125] In one embodiment, the first processing module 22 includes: a target detection submodule, a correction submodule, and a cropping submodule; The target detection submodule is used to perform target object detection and cropping processing on the first image to obtain the corresponding target region image; The correction submodule is used to perform image correction processing on the target region image to obtain a corrected image; The cropping submodule is used to perform target content recognition and cropping processing on the corrected image to obtain a second image containing the target content.

[0126] In one embodiment, based on the foregoing implementation, the incremental data includes a target version tag; Accordingly, the refresh module 25 includes: a version tag acquisition submodule and a refresh submodule; The version tag acquisition submodule is used to obtain the current version tag of the currently displayed content; The refresh submodule is used to refresh the currently displayed content and update the current version mark when the target version mark is higher than the current version mark.

[0127] In one embodiment, the second processing module 24 includes: an image enhancement submodule; The image enhancement submodule is used to enhance the second image to obtain the third image; Correspondingly, the refresh module 25 is also used to replace the currently displayed second image with the third image using a smooth image transition method.

[0128] In one embodiment, the information display device further includes: a text recognition module and a text refresh module; The text recognition module is used to perform optical character recognition processing on the third image to obtain the text content. The text refresh module is used to refresh the text content to the preset text area of ​​the current display interface through a preset display component refresh mechanism.

[0129] In one embodiment, the information display device further includes: a classification and labeling module and a label refreshing module; The classification and labeling module is used to classify and label the third image to obtain label information. The label refresh module is used to refresh label information to the preset label area of ​​the current display interface through a preset display component refresh mechanism.

[0130] In one embodiment, the information display device further includes: a similar search module and a matching refresh module; The similarity search module is used to search for similar content in a preset database based on the text content to obtain matching information; The matching refresh module is used to refresh the matching information to the preset matching information area of ​​the current display interface through a preset display component refresh mechanism.

[0131] The information display device provided in this application embodiment can be used to execute the information display method provided in the above embodiment, and has corresponding functions and beneficial effects.

[0132] This application provides an information display device. Figure 14 This is a schematic diagram of the structure of an information display device provided in an embodiment of this application, with reference to... Figure 14 The information display device includes a processor 31, a memory 32, a communication module 33, an input device 34, and an output device 35. The information display device may have one or more processors, and one or more memories. The processor, memory, communication module, input device, and output device of the information display device can be connected via a bus or other means.

[0133] The memory 32, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as program instructions / modules corresponding to the information display method described in any embodiment of this application (e.g., image acquisition module, first processing module, display module, second processing module, and refresh module in an information display device). The memory may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created based on the use of the device. Furthermore, the memory may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0134] The communication module 33 is used for data transmission.

[0135] The processor 31 executes various functional applications and data processing of the device by running software programs, instructions and modules stored in the memory, thereby realizing the above-mentioned information display method.

[0136] Input device 34 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the device. Output device 35 may include display devices such as a display screen.

[0137] The information display device provided above can be used to execute the information display method provided in the above embodiments, and has corresponding functions and beneficial effects.

[0138] This application embodiment also provides a storage medium for storing computer-executable instructions. When executed by a computer processor, the computer-executable instructions are used to perform an information display method. The information display method includes: upon receiving a first trigger operation, acquiring a first image; performing a first image processing on the first image to obtain a second image, and displaying the second image; performing a second image processing on the second image to obtain incremental data, wherein the processing complexity of the second image processing is higher than that of the first image processing; and refreshing the currently displayed content according to the incremental data to replace the second image.

[0139] Storage medium – any type of memory device or storage device. The term “storage medium” is intended to include: mounting media, such as CD-ROM, floppy disk, or magnetic tape devices; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (e.g., hard disk or optical storage); registers or other similar types of memory elements, etc. Storage medium may also include other types of memory or combinations thereof. Furthermore, storage medium may reside in a first computer system in which the program is executed, or it may reside in a different second computer system connected to the first computer system via a network (such as the Internet). The second computer system can provide program instructions to the first computer for execution. The term “storage medium” can include two or more storage media residing in different locations (e.g., in different computer systems connected via a network). Storage medium may store program instructions (e.g., specifically implemented as a computer program) executable by one or more processors.

[0140] Of course, the computer-executable instructions provided in the embodiments of this application are not limited to the information display method described above, but can also perform related operations in the information display method provided in any embodiment of this application.

[0141] The information display device, storage medium, and information display equipment provided in the above embodiments can execute the information display method provided in any embodiment of this application. For technical details not described in detail in the above embodiments, please refer to the information display method provided in any embodiment of this application.

[0142] The above description is merely a preferred embodiment and the technical principles employed in this application. This application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions that can be made by those skilled in the art will not depart from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of this application, the scope of which is determined by the scope of the claims.

Claims

1. An information display method, characterized in that, include: Upon receiving the first trigger operation, acquire the first image; The first image is processed to obtain the second image, and the second image is then displayed. The second image is subjected to a second image processing to obtain incremental data. The processing complexity of the second image processing is higher than that of the first image processing. The currently displayed content is refreshed based on the incremental data to replace the second image.

2. The method according to claim 1, characterized in that, The step of performing a first image processing on the first image to obtain a second image includes: The first image is processed by target object detection and cropping to obtain the corresponding target region image; The target region image is subjected to image correction processing to obtain a corrected image; The corrected image is then subjected to target content recognition and cropping to obtain a second image containing the target content.

3. The method according to claim 1, characterized in that, The incremental data includes a target version tag; Accordingly, refreshing the currently displayed content based on the incremental data includes: Get the current version flag of the currently displayed content; If the target version marker is higher than the current version marker, refresh the currently displayed content based on the incremental data and update the current version marker.

4. The method according to claim 1, characterized in that, The step of performing a second image processing step on the second image to obtain incremental data includes: The second image is then enhanced to obtain the third image; Accordingly, refreshing the currently displayed content based on the incremental data includes: The third image is used to replace the currently displayed second image using a smooth image transition method.

5. The method according to claim 4, characterized in that, After performing image enhancement processing on the second image to obtain the third image, the process further includes: The text content is obtained by performing optical character recognition processing on the third image; Accordingly, after replacing the currently displayed second image with the third image using an image smoothing transition method, the method further includes: The text content is refreshed to the preset text area of ​​the current display interface through a preset display component refresh mechanism.

6. The method according to claim 4, characterized in that, After performing image enhancement processing on the second image to obtain the third image, the process further includes: The third image is classified and labeled to obtain tag information; Accordingly, after replacing the currently displayed second image with the third image using an image smoothing transition method, the method further includes: The label information is refreshed to the preset label area of ​​the current display interface through a preset display component refresh mechanism.

7. The method according to claim 5, characterized in that, After performing optical character recognition processing on the third image to obtain the text content, the method further includes: Based on the text content, a similar content search is performed in a preset database to obtain matching information; Accordingly, after the text content is refreshed to a preset text area of ​​the current display interface through a preset display component refresh mechanism, the method further includes: The matching information is refreshed to the preset matching information area of ​​the current display interface through a preset display component refresh mechanism.

8. An information display device, characterized in that, include: The image acquisition module is used to acquire the first image upon receiving the first trigger operation; The first processing module is used to perform a first image processing on the first image to obtain a second image; A display module is used to display the second image; The second processing module is used to perform a second image processing on the second image to obtain incremental data. The processing complexity of the second image processing is higher than that of the first image processing. The refresh module is used to refresh the currently displayed content based on the incremental data, so as to replace the second image.

9. An electronic device, characterized in that, include: Memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-7.

10. A storage medium for storing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a processor, are used to perform the method as described in any one of claims 1-7.

Images