Method for creating schedule information, electronic equipment and readable storage medium

Abstract

The invention discloses a method for creating schedule information, electronic equipment and a readable storage medium, and belongs to the technical field of terminals. Comprising the steps that a schedule extraction operation on a first picture is responded, a text recognition result, an edge recognition result and a picture category of the first picture are determined through a target recognition model, the picture category of the first picture indicates that the first picture is an order screenshot, and the edge recognition result comprises color block attribute information of the first picture. Interference information irrelevant to the schedule in the text recognition result and the edge recognition result of the first picture is filtered out. And creating schedule information of the first picture based on the filtered text recognition result of the first picture, the filtered edge recognition result and the picture category. And displaying the schedule information. According to the invention, the schedule information can be automatically created based on the first picture, a user does not need to manually record item by item in a calendar application, and the schedule information creation efficiency is improved.

Description

Technical Field

[0001] This application relates to the technical field of terminals, and particularly to a method for creating schedule information, an electronic device, and a readable storage medium. Background Art

[0002] With the rapid development of terminal technology, electronic devices can usually install various types of third-party social applications, such as ticket booking applications. Some orders or service notifications in third-party social applications usually involve schedule information. For example, the order interface of a ticket booking application involves travel schedule information. In some scenarios, users usually need to record the schedule information involved in third-party social applications through electronic devices.

[0003] In related technologies, generally, users need to manually record in a calendar application, resulting in low efficiency in creating schedule information. Summary of the Invention

[0004] This application provides a method for creating schedule information, an electronic device, and a readable storage medium, which can solve the problem of low efficiency caused by the need for users to manually create schedule information in related technologies. The technical solutions are as follows:

[0005] In a first aspect, a method for creating schedule information is provided. The method includes:

[0006] In response to a schedule extraction operation on a first picture, through a target recognition model, determine the text recognition result, edge recognition result, and picture category of the first picture. The picture category of the first picture indicates that the first picture is an order screenshot, where an order screenshot refers to a picture obtained by taking a screenshot of an order interface in an application. The edge recognition result includes the color block attribute information of the first picture. Filter out the interference information unrelated to the schedule in the text recognition result and edge recognition result of the first picture. Based on the filtered text recognition result of the first picture, the filtered edge recognition result, and the picture category, create the schedule information of the first picture. Display the schedule information. In this way, it is not necessary for users to manually create the schedule information in the order screenshot in the calendar application, improving the efficiency of creating schedule information.

[0007] As an example of the present application, the color block attribute information includes the color block coordinates, the text recognition result includes the text line coordinates and the text line recognition content. The specific implementation of filtering out the interference information unrelated to the schedule in the text recognition result and the edge recognition result of the first picture may include: according to the color block coordinates in the edge recognition result and the text line coordinates in the text recognition result of the first picture, matching the text line recognition content in each color block of the first picture from the text recognition result of the first picture. Based on the text line recognition content in each color block, determine the color blocks that do not include time and location, and filter out the recognition data corresponding to the determined color blocks from the text recognition result and the edge recognition result of the first picture. In this way, by filtering out the color blocks unrelated to the schedule, the efficiency and accuracy of subsequent schedule information extraction can be improved.

[0008] As an example of the present application, before matching the text line recognition content in each color block of the first picture from the text recognition result of the first picture according to the color block coordinates in the edge recognition result and the text line coordinates in the text recognition result of the first picture, filter out the recognition data corresponding to the skewed text lines in the text recognition result of the first picture according to the text line coordinates in the text recognition result of the first picture. According to the text line coordinates in the text recognition result of the first picture, filter out the recognition data corresponding to the text lines with a line height less than the target line height in the text recognition result and the edge recognition result of the first picture, and the target line height is the average line height of all text lines in the first picture. Filter out the recognition data corresponding to the middle timestamp from the text recognition result and the edge recognition result of the first picture, and the middle timestamp refers to the text line located in the middle position of the first picture and including only one line of timestamp. In this way, by filtering out possible interference information, the accuracy of subsequent schedule information extraction can be improved.

[0009] As an example of the present application, the target recognition model includes a first optical character recognition (OCR) recognition model, a second OCR recognition model, and multiple edge detection models. The first OCR recognition model can be used to determine the text recognition result of the picture, the second OCR recognition model can be used to determine the picture category of the picture, and different edge detection models can be used to perform edge recognition on pictures of different picture categories. In this case, in response to the schedule extraction operation on the first picture, the specific implementation of determining the text recognition result, the edge recognition result, and the picture category of the first picture through the target recognition model may include: in response to the schedule extraction operation on the first picture, input the first picture into the first OCR recognition model for processing, and output the text recognition result of the first picture. Input the first picture into the second OCR recognition model for processing, output the picture category of the first picture, determine the edge detection model corresponding to the picture category of the first picture from the multiple edge detection models, and input the first picture into the determined edge detection model for processing, and output the color block attribute information of the first picture.

[0010] In this way, after performing text recognition and edge recognition processing on the first picture through the above two branches respectively, a first recognition result and a second recognition result are obtained. Since the text recognition result in the first text recognition result can represent the text content in the first picture, and the edge recognition result in the second recognition result can represent the layout of the first picture, therefore, subsequent extraction of schedule information based on the two types of data, namely the first recognition result and the second recognition result, can improve the accuracy of information extraction.

[0011] As an example of the present application, the specific implementation of creating the schedule information of the first picture based on the text recognition result of the filtered first picture, the filtered edge recognition result, and the picture category may include: matching the text line recognition content corresponding to each remaining color block from the text recognition result of the filtered first picture based on the coordinates of each remaining color block in the filtered edge recognition result and the text line coordinates in the text recognition result of the first picture. Determining the scene corresponding to the first picture according to the text line recognition content corresponding to each remaining color block and the picture category of the first picture. Performing text splicing on the text line recognition content corresponding to each remaining color block based on the scene corresponding to the first picture to obtain spliced text. Constructing a prompt based on the spliced text and the picture category of the first picture, where the prompt includes scene description information, and the scene description information is used to describe the scene corresponding to the first picture. Inputting the prompt into a natural language recognition model for processing to extract the schedule information in the first picture and create the schedule information. In this way, by adding scene description information to the prompt, the subsequent NLP model can extract an accurate schedule information.

[0012] As an example of the present application, the specific implementation of performing text splicing on the text line recognition content corresponding to each remaining color block based on the scene corresponding to the first picture to obtain spliced text may include: in the case where the scene corresponding to the first picture is a ticket booking scene, for any one of the remaining color blocks in each remaining color block, if any one of the remaining color blocks includes multiple text lines, perform text splicing line by line. During the splicing process, if there is interactive button text in any one of the remaining color blocks, delete the interactive button text, where the interactive button text is the text line recognition content in the interactive button, the interactive button corresponds to a single text line and the length of the text line is less than the length threshold and the text line recognition content is a verb.

[0013] For the first picture in the ticket booking scene, performing splicing line by line during text splicing can make the finally obtained spliced text closer to natural language, thereby improving the accuracy of schedule information extraction by the NLP model. And during the text splicing process, deleting the interactive button text that may cause interference can further improve the accuracy of subsequent schedule information extraction.

[0014] As an example of the present application, after displaying the schedule information, in response to an editing operation on the schedule title, a target interface is displayed, and the target interface includes spliced text. In response to a selection operation on the content of the text line displayed in the target interface, the text selected by the selection operation is input into the title input box, and in response to the end-of-editing operation, the schedule title is modified to the content input in the title input box. In this way, by displaying the target interface, the user can quickly modify the schedule title by smearing, improving the user experience.

[0015] As an example of the present application, the types of pictures that the target recognition model can recognize also include instant messaging chat screenshots, instant messaging notification card screenshots, and other types of pictures. An instant messaging chat screenshot refers to a picture obtained by taking a screenshot of the chat interface in an instant messaging application, and an instant messaging notification card screenshot refers to a picture obtained by taking a screenshot of the service notification card in an instant messaging application. Other types of pictures include pictures other than instant messaging chat screenshots, instant messaging notification card screenshots, and order screenshots. That is, the method provided in the embodiments of the present application can not only automatically create schedule information for order screenshots, but also create schedule information based on other pictures related to the schedule, improving the universality of the application.

[0016] In a second aspect, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method described in the first aspect is implemented.

[0017] In a third aspect, a computer-readable storage medium is provided. Instructions are stored in the computer-readable storage medium. When it runs on a computer, the computer is caused to execute the method described in the first aspect above.

[0018] In a fourth aspect, a computer program product containing instructions is provided. When it runs on a computer, the computer is caused to execute the method described in the first aspect above.

[0019] The technical effects obtained in the second, third, and fourth aspects are similar to those obtained by the corresponding technical means in the first aspect, and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a schematic diagram of an application scenario shown according to an exemplary embodiment;

[0021] Figure 2 is a schematic diagram of an application scenario shown according to another exemplary embodiment;

[0022] Figure 3 is a schematic diagram of an application scenario shown according to another exemplary embodiment;

[0023] Figure 4 is a schematic diagram of an application scenario shown according to another exemplary embodiment;

[0024] Figure 5 is a schematic diagram of an application scenario shown according to another exemplary embodiment;

[0025] Figure 6 is a schematic diagram of an application scenario shown according to another exemplary embodiment;

[0026] Figure 7 is a schematic diagram of an application scenario shown according to another exemplary embodiment;

[0027] Figure 8 is a schematic diagram of a software system of an electronic device shown according to an exemplary embodiment;

[0028] Figure 9 is a schematic diagram of an implementation framework for creating schedule information shown according to an exemplary embodiment;

[0029] Figure 10 is a flowchart of a method for creating schedule information shown according to an exemplary embodiment;

[0030] Figure 11 is a schematic diagram of the processing of an instant messaging chat screenshot shown according to an exemplary embodiment;

[0031] Figure 12 is a schematic diagram of the processing of an order screenshot shown according to an exemplary embodiment;

[0032] Figure 13 is a flowchart of a method for creating schedule information shown according to another exemplary embodiment;

[0033] Figure 14 is a schematic diagram of the architecture of an electronic device shown according to an exemplary embodiment. Detailed implementation manners

[0034] To make the objectives, technical solutions, and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0035] It should be understood that the "multiple" mentioned in this application refers to two or more. In the description of this application, unless otherwise specified, " / " means "or". For example, A / B can mean A or B; the "and / or" in this text is just a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in order to clearly describe the technical solution of this application, words such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions and roles. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit being different.

[0036] The reference to "one embodiment" or "some embodiments" etc. described in the specification of this application means that in one or more embodiments of this application, the specific features, structures or characteristics described in combination with this embodiment are included. Thus, the statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments" etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "include", "comprise", "have" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0037] Before introducing the method for creating schedule information provided in the embodiments of this application, the terms or nouns involved in the embodiments of this application will be briefly described first.

[0038] One-stop office software: It can provide a variety of office functions, such as document processing, spreadsheets, presentation production, project management, calendars and emails, etc. Users can complete a variety of office tasks in the same software, improving the work efficiency of users. One-stop office software usually adopts a unified user interface design, making the switching and use between each function module more convenient and consistent.

[0039] KV form: It means presenting information in the format of a table (it can have a table or not). For example, taking the presented information including two columns of information as an example, the left side is the theme, and the right side is the specific content corresponding to the theme.

[0040] Global collection: It means that users can trigger the electronic device to collect information by swiping three fingers up and down on the screen.

[0041] Magic Text: It is a function for quickly extracting text from pictures. Usually, users can turn on or off the Magic Text switch through the path of "Settings > Smart Assistant > Magic Text" to enable or disable this function.

[0042] Prompt: In the embodiments of this application, it refers to the model input data formed by adding a piece of text or instruction to the input text information when using a machine learning model, that is, it includes user prompt and system prompt. The user prompt is the input text information, and the system prompt is a piece of text or instruction added. In this way, it can guide the machine learning model to generate more accurate and targeted outputs. The system Prompt can be a question, a description, a formatted input, or even some keywords. By reasonably designing the prompt, the machine learning model can be guided to understand the intention of the question, so as to generate a relatively accurate answer. In short, Prompt is a technology widely used in machine learning models, which can help users solve data bias, improve the controllability of machine learning models and the problem modeling ability.

[0043] Vertical domain: It refers to providing specific services for a defined group, including industries such as entertainment, medical care, environmental protection, education, and sports.

[0044] Currently, various third-party social applications can be installed in electronic devices. Exemplarily, they include but are not limited to instant messaging (IM) applications, ticket booking applications, hotel reservation applications, etc. For example, instant messaging applications can be WeChat, QQ, DingTalk, Feishu, etc., and ticket booking applications and hotel reservation applications can be Ctrip, Qunar, Tongcheng, Fliggy, etc. Users can chat, book tickets, and reserve hotels through third-party social applications, and can also follow the official accounts (or notification accounts) of various industries through third-party social applications to understand the fields and events to be concerned about through the service notification cards issued by the official accounts (or notification accounts). In some scenarios, some messages in third-party social applications may involve schedule information. For example, the order interface of a ticket booking application involves the relevant schedule of an event. Users usually have the need to record schedule information. In this case, generally, users need to manually record the schedule information item by item in the calendar application. However, manually creating schedule information is rather cumbersome and the creation efficiency of schedule information is low. Therefore, the embodiments of this application provide a method for creating schedule information, enabling the electronic device to automatically create schedule information and improving the creation efficiency of schedule information.

[0045] As an example of this application, referring to Table 1, the electronic device can create the schedules involved in the following scenarios into schedule information:

[0046] Table 1

[0047]

[0048] The above graphics and text include pictures and text. That is, the method provided by the embodiments of the present application can not only automatically extract schedule information from text, but also automatically extract schedule information from pictures. Specifically, the scope supported by the method provided by the embodiments of the present application is shown in Table 2:

[0049] Table 2

[0050]

[0051] As can be seen from Table 2, the electronic device can extract schedule information from pictures. The pictures can be screenshot pictures or pictures taken by a camera. The form of the screenshot picture can include, but is not limited to, full-screen screenshot or area screenshot. The windows involved in the screenshot picture can be full-screen, split-screen or floating window. The sources of the screenshot pictures usually come from mobile phones, tablets, PCs, etc.; the form of the pictures taken by the camera can be, but is not limited to, printed text, handwritten text and artistic words. In addition, the electronic device can also extract schedule information from text. The form of the text includes, but is not limited to, Text and Webview. The format of the text can be plain text, formatted text or text mixed with pictures. The length of the text can be single-paragraph, multi-paragraph, short text or long text, etc. The embodiments of the present application will be mainly described by taking the input as a picture as an example.

[0052] For ease of understanding, the application scenarios provided by the embodiments of the present application will be introduced next.

[0053] In one example, schedule information can be extracted from an order screenshot. The order screenshot can be a screenshot of an order such as a hotel, train ticket, or airplane ticket. Exemplarily, taking the train ticket order screenshot as an example, when the user wants to create relevant schedule information, the user can trigger the mobile phone to take a screenshot of the application interface where the train ticket order is located. For example, when the mobile phone displays the application interface where the train ticket order is located, the user can double-click on the screenshot trigger area of the mobile phone screen to trigger the screenshot operation, so that the mobile phone takes a screenshot of the application interface where the train ticket order is located. After that, referring to Figure 1 Figure (a) therein, the mobile phone displays the screenshot editing interface U1 and displays the train ticket order screenshot obtained after the screenshot in the screenshot editing interface U1, that is, displays the picture p1. The screenshot editing interface U1 includes a "Share" control. When the user wants to create the schedule information in the picture p1, the user can click the "Share" control. Referring to Figure 1 Figure (b) therein, in response to the user's trigger operation on the "Share" control, the mobile phone displays the sharing floating window 10. The sharing floating window 10 includes a calendar icon 11, and the user can click the calendar icon 11. In response to the user's trigger operation on the calendar icon 11, the mobile phone starts to process the picture p1 to extract and create relevant schedule information. Exemplarily, referring toFigure 1 In figure (c) thereof, during this process, the mobile phone can display a prompt message of "offline parsing schedule information" so that the user can know that the schedule is being extracted from picture p1. Refer to Figure 1 Figure (d) thereof. After the schedule information is successfully created, the mobile phone displays a schedule display interface U2. The schedule display interface U2 includes a schedule display window 12, and the created schedule information 13 is displayed in the schedule display window 12. The schedule information 13 includes content such as a schedule title, train number, departure time and arrival time, departure place and destination, etc. In this way, the mobile phone achieves the purpose of automatically creating and displaying schedule information, which can avoid the need for the user to manually record and improve the efficiency of creating schedule information.

[0054] As an example of the present application, the schedule display window 12 further includes a plurality of editing controls, and the user can also edit the schedule information 13 created by the mobile phone based on the plurality of editing controls according to needs. For example, refer to Figure 2 Figure (a) thereof. When the user wants to modify the schedule title, they can click on the title editing control 14 in the schedule display window 12. In response to the user's click operation on the title editing control 14, the mobile phone displays a target interface U3 (which can be called a scribbling interface) as shown in Figure 2 Figure (b) thereof. Information related to the schedule in picture p1 is displayed in the target interface U3. In this way, the user can scribble on the information displayed in the target interface U3 to modify or fill in the schedule title. Correspondingly, the mobile phone inputs the content scribbled by the user in the title input box 15 of the target interface U3. For example, refer to Figure 2 Figure (b) thereof. When the user wants to modify the schedule title to "Xiaobo Order Details", the user can scribble the content of "Xiaobo", "Order", and "Details" in the target interface U3 in sequence. Correspondingly, the mobile phone inputs "Xiaobo", "Order", and "Details" in the title input box 15 in sequence. Refer to Figure 2 Figure (c) thereof. After the user finishes scribbling, they can trigger the "input" control (or "√" control) of the target interface U3. In response to the user's triggering operation on the "input" control (or "√" control), refer to Figure 2 Figure (d) thereof. The mobile phone resumes displaying the schedule display window 12. At this time, the user can see from the schedule display window 12 that the schedule title has been modified to the content modified by the user through scribbling operation, that is, the schedule title has been changed from "Ticket Purchase Success Notification" to "Xiaobo Order Details". In this way, by displaying information related to the schedule and that can be scribbled in the target interface U3, the user can quickly modify the schedule title by scribbling, improving the user experience.

[0055] In addition, refer to Figure 2In figure (a), the schedule display window 12 further includes a time editing control. When the user wants to edit the time in the schedule information 13, the user can also modify it based on the time editing control. For example, the user can click on the displayed time for editing. Additionally, after the user swipes down the schedule display window 12, the schedule display window 12 can also provide other editing controls, such as editing controls for the number of repetitions, reminder time, important reminder, etc. In this way, the user can edit the schedule information based on other editing controls, and the embodiments of the present application do not limit this.

[0056] As an example of the present application, after the user clicks on the "√" control in the schedule display window 12, in response to this trigger operation, the mobile phone displays the schedule information 13 in the schedule details area of the calendar application, so that the user can view the schedule information 13 from the schedule details area of the calendar application. As an optional example, after the user clicks on the "√" control in the schedule display window 12, the mobile phone can also display the schedule information 13 in the form of a card at positions such as the desktop, the negative first screen, or the notification center, for the convenience of the user to quickly view later. The embodiments of the present application do not limit this.

[0057] It should be noted that the above is only an example of the user triggering the mobile phone to create schedule information through the sharing entry (i.e., the sharing control). In another example, see Figure 1 In figure (a), the mobile phone provides a Magic text control 00 in the screenshot editing interface U1. When the user needs the mobile phone to create schedule information based on the picture p1, the user can also click on the Magic text control 00, thereby triggering the mobile phone to create and display the schedule information with one click.

[0058] In another example, the user can also trigger the mobile phone to create and display schedule information through the Any Door entry. Exemplarily, see Figure 3 In figure (a), after the mobile phone takes a screenshot of the application interface where the order screenshot is located, the obtained picture p1 is automatically saved to the gallery. Thus, when the user wants the mobile phone to automatically create the schedule information in the picture p1, the user can open the screenshot picture interface U4 in the gallery, and the picture p1 is displayed in the screenshot picture interface U4. See Figure 3 In figure (b), the user can trigger the mobile phone to select the picture p1, and then, the user can drag the picture p1 to the right side of the mobile phone screen. When the user drags to a certain position, in response to the user's drag operation, the mobile phone displays the application programs that can receive and process the picture p1, such as Figure 3 As shown in figure (b), the calendar, WeChat, and QQ are displayed. Thus, the user can continue to drag the picture p1 onto the calendar application and then release it. In response to the user's release operation, the mobile phone starts to process the picture p1 to extract and create the relevant schedule information. See Figure 3In figure (c), after the mobile phone creates schedule information, the schedule information is displayed in the schedule display window 12.

[0059] In another example, the user can only select the content related to the schedule in picture p1, and then trigger the mobile phone to extract schedule information from the selected part. For example, see Figure 4 In figure (a), the user can select a part of the content in picture p1. For example, in the screenshot editing interface U1, a control for triggering the selection operation can be provided. After triggering this control, the user can select on picture p1. In response to the user's selection operation, the mobile phone selects the part selected by the user. As an example, the mobile phone can take a screenshot of the area selected by the user, as shown in Figure 4 figure (b). After that, the user can trigger the mobile phone to extract schedule information from the selected area through an interaction entry such as the "Share" control, and create and display schedule information related to the content in this area. In this way, by supporting the user to select on picture p1, the data processing amount of the mobile phone can be reduced, thereby improving the creation efficiency of schedule information.

[0060] It should be noted that the interaction entries used by the user to trigger the mobile phone to extract schedule information from picture p1 in the above application scenarios are only exemplary. In some embodiments, the mobile phone can also be triggered to extract schedule information from picture p1 through other interaction entries. For example, it can also be triggered through interaction entries such as global collection. The embodiments of the present application do not limit this.

[0061] The above application scenarios are only exemplary. In addition, the mobile phone can also extract schedule information from some types of pictures in other scenarios. For example, in another example, the mobile phone can also extract schedule information from an instant messaging chat screenshot, which is a picture obtained by taking a screenshot of the interface of the chat window of an instant messaging application. See Figure 5 In figure (a), the figure shows a schematic diagram of an instant messaging chat screenshot (i.e., picture p2) shown according to an exemplary embodiment, and multiple messages in it involve schedule information. When the user wants the mobile phone to create and display the schedule information in picture p2, the mobile phone can be triggered according to the operation process described above. For example, the user can trigger the mobile phone to extract schedule information from picture p2 through the Magic text entry. Correspondingly, the mobile phone extracts schedule information based on picture p2, and then creates or displays relevant schedule information. For example, the displayed schedule information is as shown in Figure 5 50 in figure (b). The schedule information 50 includes chat topics, time, location, etc.

[0062] In another example, see Figure 6, there is a forwarded picture p3 in a certain chat window (which can be obtained by screenshot or by taking a photo), and there is schedule information in the picture p3. When the user wants to extract the schedule information in the picture p3, the user can drag the picture p3 to the right side of the mobile phone screen. Refer to Figure 6 Figure (b) in Figure 6 . When the user drags to a certain position, in response to the user's drag operation, the mobile phone displays application programs that can receive and process the picture p3, such as Figure 6 Figure (b) in

[0063] shows, displaying the calendar, WeChat, and QQ. After the user drags the picture p3 onto the calendar application and releases it, in response to the user's release operation, the mobile phone starts to automatically process the picture p3 to extract and create relevant schedule information. Refer to Figure 6 Figure (c) in

[0064] . After the mobile phone creates the schedule information, it displays the schedule information in the schedule display window 12. Figure 6 Refer to

[0065] Figure (c) in Figure 7 . In the case where the mobile phone creates multiple schedule information, multiple schedule information can be displayed in the schedule display window 12. When not all schedule information can be fully displayed in the schedule display window 12, the user can trigger the mobile phone to display the hidden schedule information by swiping the schedule display window 12 left and right. Figure 7

[0066] In addition, the mobile phone not only supports the user to drag the picture in the chat window to the AnyDoor entrance, but also supports the user to drag the text to the AnyDoor entrance. For example, a certain chat window includes chat text, and the chat text includes schedule information. When the user needs the mobile phone to create the schedule information in the chat text, the user can select the chat text and then drag the chat text to the calendar application in the manner Figure 6 shown. Accordingly, the mobile phone extracts the schedule information in the chat text, creates and displays relevant schedule information.

[0065] In another example, the mobile phone can also extract schedule information from a screenshot of an instant messaging notification card. For example, refer to Figure 7 Figure (a) in Figure 7 . The figure shows a schematic diagram of a screenshot of an instant messaging notification card (i.e., picture p4) shown according to an exemplary embodiment. The picture p4 is obtained by taking a screenshot of a notification card published in the service notification in WeChat, and it includes schedule information. When the user wants the mobile phone to create and display the schedule information in the picture p4, the user can trigger the mobile phone to extract the schedule information according to the operation process described above. For example, the user can trigger the mobile phone to extract the schedule information from the picture p4 through the sharing entrance. Accordingly, the mobile phone extracts the schedule information based on the picture p4, and then creates or displays relevant schedule information. For example, refer to Figure 7 Figure (b) in . The mobile phone creates and displays schedule information 70, and the schedule information 70 includes the theme, ticket number, departure time and date, location, etc.

[0066] It should be noted that the above application scenarios are all exemplary and do not limit the application scenarios of the method provided by the embodiments of the present application. In another embodiment, the mobile phone can also extract schedule information from other types of pictures, and other types of pictures include pictures other than instant messaging chat screenshots, instant messaging notification card screenshots, and order screenshots. Of course, other types of pictures can be pictures obtained by taking screenshots or pictures taken by a camera.

[0067] In addition, it should be noted that the above is an example with the electronic device being a mobile phone. The electronic device involved in the embodiments of the present application can also be an action camera (GoPro), a digital camera, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, a vehicle-mounted device, an ultra-mobile personal computer (UMPC), a netbook, etc. The embodiments of the present application do not limit this.

[0068] Figure 8 It is a block diagram of a software system of an electronic device provided by an embodiment of the present application. Refer to Figure 8 , the layered architecture divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom are the application layer, the application framework layer, the Android runtime and the system layer, and the kernel layer.

[0069] The application layer may include a series of application packages. Refer to Figure 8 , the application packages may include a calendar and other applications. For example, other applications may include instant messaging, ticket booking, camera, gallery, call, map, navigation, Bluetooth, music, video, short message and other applications.

[0070] In addition, as an example of the present application, the application layer further includes a schedule management service and a model management service. The schedule management service can be used to provide services for the calendar, or be called by the calendar. For example, the schedule management service can create schedule information for the calendar and store data such as schedule information. The schedule management service may include a schedule creation service (which can be called: intelligent parsing and processing service) and a schedule database (such as: Calednar Provider schedule database). The schedule management service can create schedule information through the schedule creation service and store the created schedule information through the schedule database. The model management service (which can be called: MagicLive large model service) can be used to provide various models for the schedule management service to call when needed.

[0071] In one example, the model management service may provide a natural language processing (NLP) model, an object recognition model, and a personal behavior feature model. Among them, the NLP model can be used to identify prompts to determine schedule information. The NLP model can run through a natural language unit (NLU). In some examples, the NLP model can not only identify prompts but also extract keywords in a piece of text, such as keywords like time and location. The object recognition model can be used to perform text recognition and edge recognition on pictures, and can also be used to determine the picture category of the pictures. In one example, the object recognition model includes a first optical character recognition (OCR) model, a second OCR model, and an edge detection model. The first OCR model can be used for text recognition, the second OCR model can be used to determine the picture category, and the edge detection model can be used to perform edge recognition on pictures. The number of edge detection models can be multiple, and different edge detection models can be used to perform edge recognition on pictures of different picture categories; the personal behavior feature model can be used to determine a user profile based on the user's historical behavior data.

[0072] The application framework layer provides application programming interfaces (APIs) and programming frameworks for the applications in the application layer. The application framework layer includes some predefined functions. As Figure 8 shown, the application framework layer may include a window manager, a content provider, a view system, a telephone manager, a resource manager, a notification manager, etc.

[0073] Android Runtime includes a core library and a virtual machine. Android runtime is responsible for the scheduling and management of the Android system. The core library contains two parts: one part is the functional functions that need to be called by the Java language, and the other part is the core library of Android. The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as the management of object life cycles, stack management, thread management, security and exception management, and garbage collection.

[0074] The system library may include multiple functional modules, such as: a surface manager, Media Libraries, a 3D graphics processing library (such as: OpenGL ES), a 2D graphics engine (such as: SGL), etc.

[0075] The kernel layer is the layer between hardware and software. The kernel layer includes at least a display driver, a camera driver, an audio driver, and a sensor driver.

[0076] The electronic device can implement the method for creating schedule information provided in the embodiments of the present application through the interaction of the above-mentioned multiple modules. Exemplarily, refer to Figure 9 , Figure 9 is a schematic diagram of a method implementation framework shown according to an exemplary embodiment. In implementation, the electronic device can pass the picture to be processed to the calendar application through interaction entrances such as sharing, Any Door, global collection, or Magic Text. Among them, the picture can be a screenshot of the order of train tickets, air tickets, or a screenshot of the order of hotels, restaurants, entertainment, or a screenshot of the order of sports, health, or a screenshot of WeChat mini-programs, web, etc., or a screenshot of the service notification card of the IM notification number, or a screenshot of the IM chat conversation, etc. By way of example and not limitation, the screenshot of the order of train tickets and air tickets can be from applications such as 12306, China Railway, or Ctrip, and the screenshot of the order of hotels, restaurants, entertainment can be from applications such as Ctrip, Qunar, Tongcheng, Meituan, Fliggy, Dianping, Damai, etc., and the screenshot of the order of sports, health can be from applications such as Keep, Lianduoduo, registration platforms, etc., and the screenshot of WeChat mini-programs, web, etc. can include content such as performances, flash sales, marathons, etc., and the screenshot of the service notification card of the IM notification number can include notification cards issued by third parties such as hospitals, scenic spot tickets, educational institutions, insurance companies, etc. through the IM notification number, and the screenshot of the IM chat conversation can include content such as work arrangements, invitations, educational tasks, etc.

[0077] After receiving the picture, the calendar application requests the target recognition model to perform text recognition and edge recognition on the picture. Through text recognition, the text information in the picture can be extracted, and through edge recognition, the chat elements or color blocks in the picture can be recognized. For example, refer to Figure 9 , passing the picture p2 to the target recognition model through the interaction entrance. After text recognition by the target recognition model, the text recognition result is as shown in Figure 9 90 in, and in addition, after edge recognition by the target recognition model, the coordinates and categories of each chat element in the picture p2 can be recognized.

[0078] After that, the electronic device can perform filtering processing on the text recognition result and the edge recognition result based on the filtering rules to remove the interference information unrelated to the schedule in the text recognition result and the edge recognition result. As an example of the application, the preset filtering rules can include at least one of the following rules: 1. Discard dense text blocks. 2. Discard small characters. 3. Discard skewed lines. 4. Discard the floating text on the attached drawing. 5. Discard the color blocks unrelated to the schedule. Exemplarily, after performing filtering processing on the text recognition result based on the filtering rules, the content shown in 91 in Figure 9 can be removed. Additionally, after performing filtering processing on the edge recognition result based on the filtering rules, the coordinates and categories of the chat elements unrelated to the schedule can be removed.

[0079] If the text recognition result is not filtered but directly input into the NLP model for schedule information extraction, in this case, referring to Table 3, the recognition success rate of the NLP model is relatively low, usually only reaching about 30%. The reason is the problems of interference information, format line breaks, and loss of layout information. Therefore, in the embodiment of the present application, filtering the text recognition result before extracting schedule information through the NLP model can improve the accuracy of the subsequent NLP model in extracting schedule information.

[0080] Table 3

[0081]

[0082] After the filtering processing, the electronic device constructs a prompt (i.e., prompt) based on the remaining text recognition result and edge recognition result after filtering. Then, the prompt is input into the NLP model to extract schedule information through the NLP model. The NLP model outputs schedule information based on the input prompt, for example Figure 9 the content processed by the NLP model shown in 92 in. After that, by performing post-processing on the schedule fields of the schedule information, the schedule information to be displayed can be obtained. Exemplarily, the processed schedule information is as shown in Figure 9 93 in. After that, the electronic device can display the schedule information.

[0083] In one example, the post-processing of the schedule fields can include at least one of the following: 1. Process according to the reminder time rule. 2. Process according to the start time rule of tomorrow. 3. Process according to the details rule. 4. Process based on the title merging pre-filling rule.

[0084] The reminder time rule includes setting the reminder time of the schedule information earlier than the preset duration of the time extracted by the NLP model. The preset duration can be set according to requirements. For example, if the preset duration is 30 minutes and the time extracted by the NLP model is 8:30, the reminder time of the schedule information can be set to 8:00. In addition, the reminder time rule also includes repeated reminders. For example, if the schedule information extracted by the NLP model is to grab numbers on Monday, Tuesday, and Wednesday, the electronic device sets the reminder time for Monday, the reminder time for Tuesday, and the reminder time for Wednesday for this schedule information, rather than just setting one reminder time.

[0085] The start time rule for the next day means that if the time extracted by the NLP model crosses days, months, or years, the time after crossing days, months, or years is supplemented. For example, if the date extracted by the NLP model is December 5th, the start time is 23:00, and the end time is 00:30, the electronic device can supplement the end time as 00:30 on December 6th in the schedule information.

[0086] The details rule means adjusting the layout and font size of the created schedule information according to the size of the screen of the electronic device, so that it can be correctly and clearly displayed in the schedule details area of the calendar application.

[0087] The title merging pre-filling rule includes, in the case where there are multiple different titles corresponding to the same time, selecting the title with the longest length as the title of the schedule information from these multiple titles. In addition, the title merging pre-filling rule also includes using the specified schedule title corresponding to the scenario of the picture as the schedule title of the schedule information. For example, if the scenario of the picture is to make an appointment to pick up a number for medical treatment, and the schedule title extracted by the NLP model is "get a number" or "pick up a number", the schedule title can be standardized to "register for medical treatment", where the specified schedule titles corresponding to different scenarios can be preset according to requirements.

[0088] It should be noted that the above post-processing of the schedule fields is only exemplary. In another example, the post-processing of the schedule fields may also include, but is not limited to, at least one of time similarity processing, discarding empty results, risk control, and cleaning non-natural language titles. Time similarity processing means that if the time in the output schedule information is earlier than the current system time of the electronic device, the time closest to the time in the schedule information is determined according to the current system time, and the determined time is determined as the time in the schedule information. For example, if the time in the schedule information is Tuesday and the current system time is Wednesday, it is recorded as Tuesday of the next week in the schedule information. Discarding empty results means discarding the returned empty fields. Risk control means controlling sensitive words. Cleaning non-natural language titles means that if the schedule title does not include a verb, a verb can be added to the schedule title or the schedule title can be default set according to the scenario corresponding to the picture.

[0089] Next, in combination with Figure 10 a detailed introduction will be given to the method for creating schedule information provided by the embodiments of the present application. Refer to Figure 10 and the method may include the following implementation steps:

[0090] S1001: The calendar application receives the picture L to be processed.

[0091] The picture L may be a picture in bitmap format.

[0092] The calendar application receives the picture L passed by the interaction entry. As described above, the interaction entry may be a sharing entry, an arbitrary door entry, etc. Exemplarily, refer to Figure 1 Figure (a) in. When the user submits the picture L (such as p1) to the calendar application through the sharing control, the interaction entry is the sharing entry.

[0093] S1002: The calendar application sends a schedule creation instruction to the schedule management service, and the picture L is carried in the schedule creation instruction.

[0094] The schedule creation instruction is used to indicate the creation and display of relevant schedule information based on the picture L.

[0095] S1003: The schedule management service sends the picture L to the first OCR model in the model management service.

[0096] In implementation, the schedule management service invokes the first OCR model in the model management service and sends the picture L to the first OCR model for text recognition processing.

[0097] S1004: The first OCR model determines the first recognition result of the picture L.

[0098] As an example of the present application, the first recognition result includes the text recognition result of the picture L, and the text recognition result includes the text block coordinates, text line coordinates, and text line recognition content.

[0099] As an example, the first recognition result further includes target indication information, which can be used to indicate whether the picture L input into the first OCR model is a screenshot picture or a taken picture. Exemplarily, the target indication information can be a first identifier, a second identifier, or a third identifier. The first identifier is used to indicate that the picture L input into the first OCR model is a screenshot picture. The second identifier is used to indicate that the picture L input into the first OCR model is a taken picture and is a picture taken of a document. The third identifier is used to indicate that the picture L input into the first OCR model is other taken pictures, such as pictures taken of advertisements, road signs, magazines, etc. The first identifier, the second identifier, and the third identifier can be set according to requirements. For example, the first identifier is F1, the second identifier is F2, and the third identifier is F3.

[0100] That is, after the first OCR model receives the picture L, it recognizes the picture L and outputs the first recognition result of the picture L.

[0101] S1005: The first OCR model sends the first recognition result to the schedule management service.

[0102] As an example, after receiving the first recognition result, the schedule management service can cache the first recognition result.

[0103] S1006: The schedule management service sends the picture L to the second OCR model in the model management service.

[0104] In one example, after receiving the schedule creation instruction sent by the calendar application, in addition to sending the picture L to the first OCR model for text recognition processing, the schedule management service can also call the second OCR model in the model management service and send the picture L to the second OCR model to determine the picture category through the second OCR model. That is, the operation of S1006 and the operation of S1003 can be executed in parallel.

[0105] Since the method provided in the embodiments of the present application can extract schedule information from pictures of different picture categories, and the information layout of pictures of different picture categories is different, the electronic device processes pictures of different picture categories in different ways. Therefore, in implementation, after receiving the picture L to be processed, the schedule management service not only performs text recognition through the first OCR model, but also inputs the picture L into the second OCR model to determine the picture category of the picture L.

[0106] As an example of this application, the picture categories include instant messaging chat screenshots, instant messaging notification card screenshots, order screenshots, and other category pictures. An instant messaging chat screenshot refers to a picture obtained by taking a screenshot of the chat interface in an instant messaging application; an instant messaging notification card screenshot refers to a picture obtained by taking a screenshot of the service notification card in an instant messaging application; an order screenshot refers to a picture obtained by taking a screenshot of the order interface in an application program. For example, the order screenshot can be a train ticket order screenshot, an air ticket order screenshot, a hotel order screenshot, etc.; other category pictures include other pictures other than instant messaging chat screenshots, instant messaging notification card screenshots, and order screenshots. Other category pictures can be screenshot pictures or photographed pictures.

[0107] S1007: The second OCR model determines the picture category of picture L.

[0108] After receiving picture L, the second OCR model identifies picture L and outputs the picture category of picture L. Exemplarily, the picture category of picture L is an order screenshot.

[0109] S1008: The second OCR model sends the picture category of picture L to the schedule management service.

[0110] S1009: The schedule management service sends picture L to the edge detection model corresponding to the picture category of picture L.

[0111] As an example of this application, there are multiple edge detection models provided in the model management service. The multiple edge detection models can be pre-trained. Different edge detection models can perform edge recognition on pictures of different picture categories to determine the attribute information of the objects in the pictures. For example, the object is a chat element or a color block, and the attribute information includes the coordinates, category, number, etc. of the object. In some embodiments, the attribute information of the chat element can be referred to as the graphic element attribute information, and the attribute information of the color block can be referred to as the color block attribute information.

[0112] Exemplarily, the multiple edge detection models include a first edge detection model and a second edge detection model. The first edge detection model can be used to perform edge recognition on instant messaging chat screenshots to determine the coordinates and category of the chat elements in the instant messaging chat screenshots; the second edge detection model can perform edge recognition on other pictures other than instant messaging chat screenshots. For example, the second edge detection model can be used to perform edge recognition on instant messaging notification card screenshots, order screenshots, or other category pictures to determine the coordinates of the color blocks in the other pictures, and further can also determine the category of the color blocks.

[0113] Different edge detection models can be obtained by iteratively training an initial training model with picture training samples corresponding to the picture category of the corresponding picture. The picture training samples can be obtained in advance through edge annotation according to requirements. The initial training model can be set according to requirements. By way of example and not limitation, the initial training model can be a recurrent neural network (RNN), etc.

[0114] After the schedule management service receives the picture category of picture L, it determines the edge detection model corresponding to the picture category of picture L from multiple edge detection models. Exemplarily, when picture L is Figure 1 p1 in the (a) figure in Figure 5 , that is, in the case of an order screenshot, the edge detection model corresponding to the picture category of picture L is determined from multiple edge detection models as the second edge detection model; when picture L is

[0115] p2 in the (a) figure in

[0116] (which is an instant messaging chat screenshot), the edge detection model corresponding to the picture category of picture L is determined from multiple edge detection models as the first edge detection model. Subsequently, the schedule management service invokes the determined edge detection model and sends picture L to the edge detection model to request edge recognition of picture L.

[0117] In one example, when the picture category of picture L is an instant messaging chat screenshot, the edge detection model corresponding to the picture category of picture L is the first edge detection model. After inputting picture L into the first edge detection model for processing, the output edge recognition result includes graphic and text element attribute information. Exemplarily, chat elements include avatars, titles, nicknames, chat content, chat timestamps, usernames, specified identifiers, etc. The specified identifier includes the "+" identifier. For example, referring to Figure 11 , after inputting picture L into the first edge detection model, the first edge detection model can determine that the chat elements in picture L include Figure 11 multiple items identified by the dashed boxes in

[0118] In another example, when Picture L is not a screenshot of an instant messaging chat, such as a screenshot of an instant messaging notification card, an order screenshot, or a picture of other categories, the edge detection model corresponding to the picture category of Picture L is the second edge detection model. After inputting Picture L into the second edge detection model, the second edge detection model performs color block segmentation, and the output edge recognition result includes color block attribute information. For example, see Figure 12 , when Picture L is an order screenshot, after inputting Picture L into the second edge detection model, the second edge detection model can determine that the color blocks in Picture L include Figure 12 multiple items identified by the dashed box in

[0119] S1011: The edge detection model sends the edge recognition result to the schedule management service.

[0120] As an example, after receiving the edge recognition result, the schedule management service can cache the edge recognition result.

[0121] It is worth mentioning that after performing text recognition and edge recognition processing on Picture L through the above two branches respectively, a first recognition result and a second recognition result can be obtained. The first recognition result includes the text recognition result and the target indication information, and the second recognition result includes the edge recognition result and the picture category. Since the text recognition result can represent the text content in Picture L and the edge recognition result can represent the information layout of Picture L, therefore, subsequent schedule information extraction based on the two types of data of the first recognition result and the second recognition result can improve the accuracy of information extraction. The specific implementation can refer to the following steps.

[0122] S1012: When the picture category of Picture L is an order screenshot, the schedule management service filters the recognition data corresponding to the skewed lines, small characters, and middle timestamps from the text recognition result and the edge recognition result of Picture L.

[0123] Different picture categories correspond to different filtering rules. In implementation, the schedule management service determines the corresponding filtering rules according to the picture category of Picture L, and then filters the text recognition result and the edge recognition result of Picture L according to the determined filtering rules to filter out the interference information irrelevant to the schedule.

[0124] As an example of this application, when picture L is a screenshot of an order, the schedule management service can first filter out the recognition data corresponding to skewed lines, small characters, and the middle timestamp from the text recognition result and the edge recognition result of picture L. Among them, a skewed line refers to a text line with an inclination angle greater than a preset angle, and the preset angle can be set according to requirements. For example, the preset angle is 10 degrees or 15 degrees; small characters (usually carried in the attached drawing) refer to text lines with a line height less than the target line height, and the target line height can refer to the average line height of all text lines in picture L, and small characters are generally less than 10 dp; the middle timestamp is a timestamp used to indicate the order generation time, usually located in the middle of picture L.

[0125] As an example, in the implementation of filtering out the recognition data corresponding to skewed lines, the inclination angle of each text line can be calculated according to the text line coordinates in the text recognition result, so as to determine which text lines are skewed lines, and then delete the recognition data corresponding to the skewed lines. For example, delete the text line coordinates and text line recognition content of the skewed lines.

[0126] As an example, in the implementation of filtering out the recognition data corresponding to small characters, the line height of each text line in picture L can be determined according to the text line coordinates in the text recognition result, and the recognition data corresponding to the text lines with a line height less than the target line height can be filtered out, so as to filter out the recognition data corresponding to small characters. For example, filter out the text line coordinates and text line recognition content of the text lines with a line height less than the target line height.

[0127] As an example, in the implementation of filtering out the recognition data corresponding to the middle timestamp, the color blocks with the category of timestamp can be determined from the edge recognition result according to the categories of each color block in the edge recognition result, and at least one candidate color block can be obtained. According to the coordinates of each candidate color block in the at least one candidate color block and the text line coordinates in the text recognition result of picture L, the text line recognition content corresponding to each candidate color block is matched from the text recognition result of picture L. For any one candidate color block, when the text line recognition content corresponding to the any one candidate color block is a timestamp, the position of the any one candidate color block in picture L is determined according to the coordinates of the any one candidate color block. If it is determined to filter out the recognition data corresponding to the any one candidate color block according to the position of the any one candidate color block, the recognition data corresponding to the any one candidate color block is filtered out from the text recognition result and the edge recognition result of picture L.

[0128] Specifically, since the edge recognition result of the order screenshot includes the category of color blocks, at least one candidate color block can be obtained by filtering out the color blocks with the category of timestamp according to the category of color blocks in the edge recognition result. Since the edge recognition result does not include the text line recognition content, that is, it is impossible to know the text content corresponding to each color block. In some possible cases, the edge detection model may misjudge the category of color blocks that are not timestamps. Therefore, in order to avoid filtering out color blocks that are not timestamps as much as possible, after determining at least one candidate color block, the text line recognition content corresponding to each candidate color block can be matched from the text recognition result of picture L according to the coordinates of each candidate color block and the text line coordinates of picture L. For example, for any candidate color block, according to the coordinates of this candidate color block and the text line coordinates of picture L, the text line recognition content of at least one text line located within this candidate color block is determined from the text recognition result of picture L, so as to match the text line recognition content corresponding to this candidate color block. Subsequently, the schedule management service can determine whether the matched text line recognition content is a timestamp through the NLP model. For example, the schedule management service can send the matched text line recognition content to the NLP model through getEntitiy() to request the NLP model to identify whether this text line recognition content is a timestamp. If it is determined through the NLP model that this text line recognition content is a timestamp, it can be further determined that this candidate color block may be a middle timestamp. Otherwise, if it is determined through the NLP model that this text line recognition content is not a timestamp, it can be determined that this candidate color block is not a middle timestamp.

[0129] Furthermore, since the position of the timestamp in the order screenshot is generally fixed, for example, located in the middle of picture L, that is, most of them are middle timestamps, and the middle timestamp only includes one line of text line recognition content, that is, only includes the timestamp. Therefore, in the case where it is determined through the NLP model that the text line recognition content within a certain candidate color block is a timestamp, if it is determined according to the coordinates of this candidate color block that this candidate color block is located in the middle of picture L, and the area corresponding to this candidate color block includes a single text line, it is determined that this candidate color block is a middle timestamp, that is, the corresponding recognition data is determined to be filtered out.

[0130] In the case where it is determined through the above process that a certain candidate color block is a middle timestamp, the schedule management service deletes the recognition data corresponding to this candidate color block from the text recognition result and the edge recognition result of picture L. For example, it deletes the text line coordinates and text line recognition content corresponding to this candidate color block from the text recognition result of picture L, and deletes the coordinates and category of this candidate color block from the edge recognition result of picture L.

[0131] It should be noted that, in the embodiments of the present application, the time stamp in the order screenshot is taken as an example of the central time stamp for illustration. In some embodiments, the time stamp in the order screenshot may also be located on the right or left side, that is, it is not the central time stamp. In this case, for any candidate color block, after determining that the text line recognition content in the any candidate color block is a time stamp, if it is determined according to the coordinates of the any candidate color block that the any candidate color block is located on the right side (or left side) of the picture L, and the any candidate color block includes a single text line, then the recognition data corresponding to the any candidate color block is determined to be filtered out.

[0132] It is worth mentioning that first, determine the color blocks whose category is a time stamp according to the edge recognition result, then match the corresponding text line recognition content from the text recognition result, determine whether it is a time stamp through the NLP model according to the matched text line recognition content, and then determine whether it is a central time stamp (or right time stamp, or left time stamp) according to the position of the color block, which can improve the accuracy of time stamp recognition, thereby improving the accuracy of filtering, and further improving the accuracy of schedule information creation.

[0133] It should be noted that the above filtering rules are only exemplary. When the order screenshots are from different application programs, the information layout is usually different, and the included information may also be different, so that the interference information in the order screenshots of different application programs may be different. For example, referring to Table 4, the interference information that may exist in the hotel order from the Meituan application is the small characters and skewed lines in the bottom picture, while the interference information that may exist in the hotel order from the Qunar application is the small characters or skewed lines in the picture in the lower right corner. In order to effectively filter out the interference information in the order screenshots from different application programs, in addition to filtering small characters, skewed lines, and central time stamps, other interference information irrelevant to the schedule can also be filtered, so as to ensure that no matter which order screenshot is processed, the interference information can be effectively removed. Exemplarily, the filtering rules corresponding to the order screenshots may also include filtering out specified identifiers, keyboard text, etc., and the embodiments of the present application do not limit this.

[0134] Table 4

[0135]

[0136]

[0137] It should be noted that S1012 is an optional operation.

[0138] S1013: The schedule management service filters out the recognition data corresponding to the color blocks that do not include schedule-related content in the text recognition result and the edge recognition result of the picture L.

[0139] In one example, the schedule management service may match the text line recognition content in each color block from the filtered text recognition result according to the coordinates of each color block in the filtered edge recognition result and the text line coordinates in the filtered text recognition result. For any one of the color blocks, if it is determined from the text line recognition content in any one of the color blocks that the any one color block does not include schedule-related content, such as time and location, then the recognition data corresponding to the any one color block is filtered out from the filtered edge recognition result and the filtered text recognition result. For example, the color block attribute information of the any one color block is filtered out from the filtered edge recognition result, and the text line coordinates and text line recognition content included in the any one color block are filtered out from the filtered text recognition result.

[0140] In one example, the schedule management service may determine whether the color block includes time and location through an NLP model. For example, the text line recognition content in the color block can be sent to the NLP model one by one to request the NLP model to determine whether it includes time and location.

[0141] As an example of the present application, before the schedule management service performs the operation of S1013, it may also determine whether the number of color blocks included in the picture L exceeds a specified number according to the color block attribute information in the edge recognition result. If the number of color blocks included in the picture L does not exceed the specified number, it indicates that there are no a large number of color blocks in the picture L. In this case, the electronic device can usually perform recognition processing on the picture L. At this time, the schedule management service performs the operation of S1014. Otherwise, if the number of color blocks included in the picture L exceeds the specified number, it indicates that there are a large number of color blocks in the picture L. In this case, it may not be possible to accurately extract schedule information. Therefore, filtering processing may not be performed. For example, a prompt message may be displayed in the calendar application to prompt the user that there are too many color blocks, so as to guide the user to re-crop the picture L according to the requirements. Among them, the specified number can be set according to the requirements. For example, the specified number can be 10, and the embodiments of the present application do not limit this.

[0142] As an example of the present application, before filtering, the schedule management service may also determine whether there are dense text blocks according to the text block coordinates in the text recognition result of the picture L. If there are dense text blocks, such as the number of text blocks is greater than a preset number threshold, a prompt message may be displayed in the calendar application. The prompt message is used to prompt the user that there are dense text blocks, so that the user can re-crop the picture L according to the requirements. If there are no dense text blocks, the schedule management service performs the filtering operation.

[0143] Alternatively, in another example, during the filtering process, if the schedule management service determines that there are dense text blocks based on the text block coordinates in the text recognition result of Picture L, then the text line coordinates and text line recognition content in these text blocks are deleted.

[0144] It is worth mentioning that if no filtering is performed, it is likely to cause the subsequent NLP model to be unable to accurately extract schedule information. For example, for Figure 12 the picture in, the start time may be extracted as "2023.12.02", and the schedule theme may be taken as "ordering food", etc. In the embodiments of the present application, after determining the first recognition result and the second recognition result, filtering out the interference information in Picture L can improve the accuracy of subsequent schedule information extraction. Additionally, removing color blocks unrelated to the schedule can improve the efficiency of schedule information extraction.

[0145] It should be further noted that the above is described by taking Picture L as an order screenshot as an example. In another example, if Picture L is an instant messaging notification card screenshot, filtering can also be performed in a similar manner. In another example, if Picture L is an instant messaging chat screenshot, the schedule management service filters out the recognition data corresponding to interference information such as small characters, skewed lines, and chat timestamps from the edge recognition result and text recognition result of Picture L. The specific implementation can refer to S1012.

[0146] S1014: The schedule management service determines the scene corresponding to Picture L according to the text line recognition content of each remaining color block and the picture category of Picture L.

[0147] The scene corresponding to Picture L refers to the scene involved in the text content in Picture L. For example, it may be a ticket booking scene, an IM chat scene, a hotel reservation scene, a medical appointment scene, etc.

[0148] As an example, in the case where it is determined that Picture L is an order screenshot according to the picture category of Picture L, it is possible to further determine what kind of order under what scene according to the text line recognition content of each filtered color block. For example, it is determined whether it is a ticket booking scene or a hotel reservation scene. The ticket booking scene includes booking train tickets or airplane tickets.

[0149] In addition, when the picture category of Picture L is other pictures, such as an instant messaging notification card screenshot or an instant messaging chat screenshot, the schedule management service can also combine the matched text line recognition content to determine the scene corresponding to Picture L, such as determining an IM chat scene, etc.

[0150] S1015: In the case where the scene corresponding to Picture L is a ticket booking scene, for any one of the remaining color blocks in each remaining color block, if the any one color block includes multiple text lines, the future management service performs text splicing line by line.

[0151] In implementation, the schedule management service processes the text line recognition content of each filtered color block, such as line breaks, carriage returns, adding spaces, and splicing, according to the scene corresponding to Picture L. If the scene corresponding to Picture L is a ticket booking scene, that is, Picture L is a screenshot of a train ticket order or an airplane ticket order, refer to Table 4. The start time, end time, origin, and destination in the train ticket order screenshot or airplane ticket order screenshot are usually spliced by columns. For example, a train ticket output from the 12306 ticket booking application is "Beijing South 06:00 C2551 Binhai 06:56", and a train ticket output from the national railway ticket booking application is "C2551 06:00 Beijing South The whole journey is 56 minutes 06:56 Binhai". However, this is not close to natural language text and can easily cause the subsequent NLP model to fail to accurately recognize. Therefore, the schedule management service splices the text by rows. For example, it converts the train ticket information from the 12306 ticket booking application from columns to rows and splices it into "Beijing South C2551 Binhai 06:00 06:56", and converts the train ticket information from the railway ticket booking application from columns to rows and splices it into "C2551 06:00 06:56 Beijing South The whole journey is 56 minutes Binhai", making the spliced text closer to natural language text.

[0152] S1016: During the splicing process, if there is interactive button text in any remaining color block, the schedule management service deletes the interactive button text.

[0153] The interactive button text is the text line recognition content in the interactive button. The interactive button corresponds to a single text line, the length of the text line is less than the length threshold, and the text line recognition content is a verb. Exemplarily, refer to Figure 12 The "Order Food", "Compose Accident Insurance", and "QR Code Ticket Checking" in the third color block from top to bottom in

[0154] Of course, it should be noted that this application embodiment takes filtering the interactive button text during the splicing process as an example for illustration. In another example, it can also be filtered at other times. For example, it can also be executed after filtering the color blocks irrelevant to the schedule, or it can also be executed before filtering the color blocks irrelevant to the schedule. This application embodiment does not limit this.

[0155] In this way, after splicing the text line recognition content in each color block, the complete spliced text can be spliced according to the order of the remaining color blocks in Picture L. Exemplarily, refer to Figure 12 , the spliced text is as Figure 12 shown in 1201 in

[0156] It should be noted that S1014 to S1016 are optional operations. In another example, the schedule management service can also perform text splicing on the text line recognition content of each filtered color block according to the picture category of picture L.

[0157] In addition, it should be noted that the above is described by taking the scenario corresponding to picture L as the ticket booking scenario as an example. In another example, the scenario corresponding to picture L may also be other scenarios, such as a hotel booking scenario, an IM chat scenario, etc. Correspondingly, the schedule management service can also perform text splicing according to the preset splicing rules according to the scenario. For example, for the IM chat scenario, the user may send a sentence in multiple messages. After determining the scenario of picture L, the messages sent by the user multiple times can be spliced into one sentence according to the IM chat scenario.

[0158] S1017: The schedule management service constructs a prompt according to the scenario corresponding to picture L and the spliced text.

[0159] Pictures with different picture categories correspond to different prompt construction templates. When constructing a prompt, the schedule management service can use the prompt construction template corresponding to the picture category of picture L to construct the prompt. As an example of the present application, the constructed prompt includes scenario description information, and the scenario description information is used to indicate the scenario corresponding to picture L, that is, when constructing the prompt, the schedule management service adds the scenario description information of picture L.

[0160] It is worth noting that if the scenario description information is not added when constructing the prompt, then when the NLP model performs recognition later, the NLP model is likely to extract each time and the verbs related before and after that time into a schedule information, so it is easy to extract multiple schedule information, resulting in inaccurate extraction of schedule information. Therefore, in order to enable the NLP model to accurately identify the schedule information, the schedule management service adds the scenario description information to the constructed prompt, so that the NLP model can extract an accurate schedule information.

[0161] In one example, a prompt can be constructed based on the typesetting order of each remaining color block in picture L according to the scenario corresponding to picture L and the spliced text. In another example, taking the color block as the granularity, a prompt corresponding to each remaining color block can be constructed based on the spliced text of each remaining color block and the scenario corresponding to picture L, that is, one prompt is constructed for one color block.

[0162] Exemplarily, taking picture L as an order screenshot, the prompt constructed by the schedule management service can be:

[0163] <|Human|>The following content may be a travel order, which contains <'title','start time', 'end time','repeat cycle', 'location', 'url link', 'convener', 'participants'> fields. The fields existing for each event are output in one line in JSON format without additional response.\n\nBeijing South Station 07:21 December 5, 2023 Wuwei Station 11:41 December 5, 2023 <|Moss|>.

[0164] S1018: The schedule management service sends a prompt to the NLP model.

[0165] After the schedule management service constructs the prompt, it calls the NLP model and sends the constructed prompt to the NLP model for recognition to extract schedule information. Exemplarily, the schedule management service can request the NLP model to extract schedule information through extractinformation().

[0166] As mentioned above, the NLP model not only has the ability to recognize schedule information but also has the ability to extract keywords. Since the recognition of the prompt by the NLP model may have a certain time delay, if the recognition duration is long, it will affect the user experience. Therefore, in some examples, the schedule management service can also send the concatenated text of each remaining color block to the NLP model one by one at the granularity of the color block to request the NLP model to extract the keywords in the concatenated text of each remaining color block. In this way, when the recognition duration is long, the schedule information can be constructed using the extracted keywords. As an example, the schedule management service can instruct the NLP model to extract keywords through getEntity(), for example, it can specify the NLP model to extract keywords such as time and location, that is, specify the module as time and location.

[0167] It should be noted that the embodiments of this application are described by taking the NLP model deployed in an electronic device as an example. In another example, the NLP model can also be deployed in the cloud, and the cloud can provide an interface for the electronic device to call the NLP model. In this way, when the NLP model is needed, the schedule management service can call the NLP model through the provided interface. The embodiments of this application do not limit this.

[0168] S1019: The NLP model determines the schedule information based on the prompt.

[0169] In one example, the schedule information output by the NLP model is: {"data": "[{'title': 'Go to Wuwei','start time': 'December 5, 2023 07:21', 'end time': 'December 5, 2023 11:41', 'location': 'Beijing South - Wuwei'}]"}.

[0170] In one example, when the schedule management service also sends the concatenated text of each color block to the NLP model, the schedule management service can also extract keywords in each color block.

[0171] S1020: The NLP model sends schedule information to the schedule management service.

[0172] Exemplarily, the NLP model may send schedule information to the schedule management service in JSON format.

[0173] Furthermore, if the schedule management service extracts the keyword, the extracted keyword in each color block is also sent to the schedule management service.

[0174] S1021: The schedule management service performs schedule field post-processing on the schedule information.

[0175] For the post-processing of the schedule field of the schedule information, please refer to the above text.

[0176] In one example, if the NLP model returns schedule information within a specified time, the schedule management service performs schedule field post-processing based on the schedule information fed back by the NLP model to create the schedule information. Since the schedule information is identified based on prompts, the accuracy of schedule information creation can be improved. If the NLP model does not return schedule information within the specified time, it means that the schedule information identification has timed out. In this case, the schedule management service can construct schedule information based on the keywords extracted by the NLP model to avoid the problem of schedule information creation jams as much as possible. The specified time is set according to demand, for example, the specified time can be 5s.

[0177] It should be noted that when creating schedule information based on keywords, the schedule fields can also be post-processed and then created according to a preset template, such as including schedule title, train number, departure and arrival time, origin and destination, etc.

[0178] In one example, before creating the schedule information, the schedule management service can also call the personal behavior feature model to request to query the user's historical behavior data, such as historical location. Correspondingly, the personal behavior feature model returns the number of historical behaviors. In this way, the schedule management service can predict where the user may go based on the historical behavior data, and then combine the schedule information fed back by the NLP model to create the final schedule information, such as adding the predicted address information to the schedule information.

[0179] S1022: The schedule management service displays the processed schedule information in the calendar application.

[0180] For example, when the image L is a screenshot of an order, the schedule information displayed by the electronic device is as follows:Figure 1 as shown at 13 in figure (d) of

[0181] In one example, before presenting the schedule information, a request confirmation notice may be presented first. After receiving the confirmation display instruction triggered by the user based on the request confirmation notice, the schedule management service then presents the schedule information in the calendar application.

[0182] As an example of this application, the electronic device also supports the user to edit the displayed schedule information. Exemplarily, it supports the user to modify the schedule title of the schedule information. For example, refer to Figure 2 the embodiments shown. In this process, when the user triggers the electronic device to display the target interface, text that can be smeared needs to be displayed in the target interface. For this purpose, the schedule management service can also request the NLP model to perform word combination processing on the spliced text. For example, combine the two words "I" and "we" into "we" so that the text can be displayed in the target interface according to the combined words, thus facilitating the user to smear. In implementation, the schedule management service can call the NLP model through getWordSegment() to request the NLP model to perform word combination processing. In addition, the schedule management service can also call the NLP model through getWordSegment() to request the NLP model to determine the theme of the spliced text. For example, specify the NLP model to extract the theme entity (such as meeting, dinner). In this way, the schedule management service can display the theme in the target interface.

[0183] In the embodiment of this application, after receiving the schedule extraction operation on picture L, in response to the schedule extraction operation, the first recognition result and the second recognition result are determined through the target recognition model. The first recognition result includes the text recognition result of picture L, and the second recognition result includes the edge recognition result and the picture category of picture L. The picture category of picture L is an order screenshot, and the edge recognition result includes the color block attribute information of picture L. Filter out the interference information irrelevant to the schedule in the text recognition result and the edge recognition result according to the picture category of picture L, and then based on the filtered first recognition result and second recognition result, create and display the schedule information of picture L. In this way, it is not necessary for the user to manually create the schedule information in the order screenshot in the calendar application, improving the efficiency of creating schedule information.

[0184] Next, in combination with Figure 13 a general introduction to the implementation process of the method for creating schedule information provided by the embodiment of this application is given. Refer to Figure 13 , taking the electronic device as the execution subject as an example, this method mainly may include the following parts or all of the content:

[0185] S1301: Obtain the picture L to be processed.

[0186] For example, picture L can be obtained by the electronic device through screenshot, or can also be forwarded by other electronic devices. For the specific implementation, please refer to S1001.

[0187] S1302: Input picture L into the first OCR model for recognition to obtain the first recognition result.

[0188] For the specific implementation, please refer to S1002 - S1005.

[0189] S1303: Input picture L into the second OCR model for recognition to obtain the picture category of picture L.

[0190] For the specific implementation, please refer to S1006 - S1008.

[0191] S1304: In the case where the picture category is an order screenshot, an instant messaging notification card screenshot, or a screenshot of other categories of pictures, input picture L into the second edge detection module to obtain color block attribute information.

[0192] S1305: In the case where the picture category is an instant messaging chat screenshot, input picture L into the first edge detection module to obtain graphic and text element attribute information.

[0193] For the specific implementation of S1304 and S1305, please refer to S1009 - S1011.

[0194] S1306: In the case where picture L is an order screenshot, an instant messaging notification card screenshot, or a screenshot of other scenarios, filter out the recognition data corresponding to small characters, skewed lines, and middle timestamps in the text recognition result and edge recognition result of picture L respectively.

[0195] S1307: Filter out the recognition data corresponding to color blocks irrelevant to the schedule from the filtered text recognition result and edge recognition result.

[0196] S1308: In the case where picture L is an instant messaging chat screenshot, filter out the recognition data corresponding to small characters, skewed lines, and chat timestamps in the text recognition result and edge recognition result of picture L respectively.

[0197] For the specific implementation of S1306 to S1308, please refer to S1012 - S1013.

[0198] S1309: In the case where picture L is a taken picture, output the text recognition result.

[0199] As an example rather than a limitation, in the case where picture L is a taken picture, since the taken picture may be skewed and include a background, etc., filtering processing may not be performed, and the text recognition result is used for text splicing later.

[0200] In another example, when Picture L is a captured picture, the text recognition result and edge recognition result of Picture L can also be filtered according to the filtering rules corresponding to the order screenshot. The embodiments of the present application do not limit this.

[0201] S1310: Determine the scene corresponding to Picture L based on the text line content of each remaining object after filtering and the picture category of Picture L.

[0202] For its specific implementation, reference can be made to S1014.

[0203] S1311: Perform text splicing on the text line recognition content of each object based on the scene corresponding to Picture L.

[0204] For its specific implementation, reference can be made to S1015 - S1016.

[0205] S1312: Construct a prompt based on the scene corresponding to Picture L and the spliced text.

[0206] For its specific implementation, reference can be made to S1017.

[0207] S1313: Recognize the prompt through the NLP model to obtain schedule information.

[0208] For its specific implementation, reference can be made to S1018 - S1020.

[0209] S1314: Perform post - processing on the schedule fields of the schedule information.

[0210] The purpose is to create schedule information. For example, for a hotel order screenshot, during the process of creating schedule information, the check - in time and check - out time are used as the schedule time, the schedule title is the name of the hotel where you check in, and the location is the name of the hotel. Another example is for a train ticket order screenshot or an airplane ticket order screenshot, the departure time and arrival time are used as the schedule time, the train number is used as the schedule title, and the departure place and destination are used as the location. For its specific implementation, reference can be made to S1021.

[0211] S1315: Display the schedule information in the calendar application.

[0212] After that, during the process of displaying the schedule information, when an edit instruction for the schedule information is received, the calendar application displays a smearing interface, and the content related to the schedule information is displayed in the smearing interface, such as the spliced text. In this way, the user can modify the schedule information through the smearing operation. For the specific implementation, reference can be made to Figure 2 the operation process.

[0213] In an embodiment of the present application, after receiving an operation for extracting the schedule of picture L, in response to the schedule extraction operation, a first recognition result and a second recognition result are determined through a target recognition model. The first recognition result includes the text recognition result of picture L, and the second recognition result includes the edge recognition result and the picture category of picture L. The picture category of picture L is an order screenshot, and the edge recognition result includes the color block attribute information of picture L. Interference information irrelevant to the schedule in the text recognition result and the edge recognition result is filtered according to the picture category of picture L, and then based on the filtered first recognition result and second recognition result, the schedule information of picture L is created and displayed. In this way, it is not necessary for the user to manually create the schedule information in the order screenshot in the calendar application, improving the efficiency of creating schedule information.

[0214] Figure 14 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. Refer to Figure 14 , the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone interface 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. Among them, the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0215] It can be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than those illustrated, or combine certain components, or split certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0216] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0217] Among them, the controller may be the nerve center and command center of the electronic device 100. The controller may generate operation control signals according to the instruction operation code and timing signal to complete the control of fetching and executing instructions.

[0218] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory may save the instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can directly call it from this memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0219] It can be understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is only illustrative and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0220] The charging management module 140 is used to receive a charging input from a charger. Among them, the charger may be a wireless charger or a wired charger. The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives the inputs from the battery 142 and / or the charging management module 140 to supply power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160, etc.

[0221] The wireless communication function of the electronic device 100 can be implemented by the antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modulation and demodulation processor, baseband processor, etc. The antenna 1 and the antenna 2 are used for transmitting and receiving electromagnetic wave signals. In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technologies.

[0222] The electronic device 100 implements the display function through the GPU, the display screen 194, the application processor, etc. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change the display information.

[0223] The display screen 194 is used for displaying images, videos, etc. The display screen 194 includes a display panel. The display panel can adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is an integer greater than 1.

[0224] The electronic device 100 can implement the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194, the application processor, etc.

[0225] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement the data storage function. For example, files such as music and videos are saved in the external memory card.

[0226] The internal memory 121 can be used to store computer-executable program code, and the computer-executable program code includes instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.). The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.). In addition, the internal memory 121 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0227] The electronic device 100 can implement audio functions, such as music playback, recording, etc., through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0228] The touch sensor 180K, also known as the "touch panel". The touch sensor 180K can be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also known as the "touch control screen". The touch sensor 180K is used to detect touch operations acting thereon or nearby. The touch sensor 180K can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In some other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, at a different position from that of the display screen 194.

[0229] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that includes one or more integrated available media. The available medium may be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a Digital Versatile Disc (DVD)), or a semiconductor medium (such as a Solid State Disk (SSD)), etc.

[0230] The above are the optional embodiments provided by the present application, which are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the technical scope disclosed in the present application shall be included in the protection scope of the present application.

Claims

1. A method for creating schedule information, characterized in that, The method includes: In response to a schedule extraction operation on a first picture, through a target recognition model, determining the text recognition result, edge recognition result, and picture category of the first picture, where the picture category of the first picture indicates that the first picture is an order screenshot, and the order screenshot refers to a picture obtained by taking a screenshot of an order interface in an application, and the edge recognition result includes the color block attribute information of the first picture; Filtering out interference information unrelated to the schedule in the text recognition result of the first picture and the edge recognition result; Based on the filtered text recognition result of the first picture, the filtered edge recognition result, and the picture category, creating the schedule information of the first picture; Displaying the schedule information.

2. The method according to claim 1, wherein The color block attribute information includes color block coordinates, and the text recognition result includes text line coordinates and text line recognition content; The filtering out interference information unrelated to the schedule in the text recognition result of the first picture and the edge recognition result includes: According to the color block coordinates in the edge recognition result and the text line coordinates in the text recognition result of the first picture, matching the text line recognition content within each color block of the first picture from the text recognition result of the first picture; Based on the text line recognition content within each color block, determining color blocks that do not include time and location; Filtering out the recognition data corresponding to the determined color blocks from the text recognition result of the first picture and the edge recognition result.

3. The method according to claim 2, characterized in that, Before the step of matching the text line recognition content within each color block of the first picture from the text recognition result of the first picture according to the color block coordinates in the edge recognition result and the text line coordinates in the text recognition result of the first picture, it further includes: According to each text line coordinate in the text recognition result of the first picture, filtering out the recognition data corresponding to skewed text lines in the text recognition result of the first picture; According to each text line coordinate in the text recognition result of the first picture, filtering out the recognition data corresponding to text lines with a line height less than a target line height in the text recognition result of the first picture and the edge recognition result, where the target line height is the average line height of all text lines in the first picture; Filtering out the recognition data corresponding to the middle timestamp from the text recognition result of the first picture and the edge recognition result, where the middle timestamp refers to a text line located in the middle position of the first picture and including only one line of timestamp.

4. The method according to any one of claims 1 to 3, characterized in that, The target recognition model includes a first optical character recognition (OCR) recognition model, a second OCR recognition model, and multiple edge detection models. The first OCR recognition model can be used to determine the text recognition result of a picture, the second OCR recognition model can be used to determine the picture category of a picture, and different edge detection models can be used to perform edge recognition on pictures of different picture categories; The step of, in response to a schedule extraction operation on a first picture, through a target recognition model, determining the text recognition result, edge recognition result, and picture category of the first picture includes: In response to a schedule extraction operation on the first picture, input the first picture into the first OCR recognition model for processing, and output the text recognition result of the first picture; Input the first picture into the second OCR recognition model for processing, and output the picture category of the first picture; Determine an edge detection model corresponding to the picture category of the first picture from the multiple edge detection models; Input the first picture into the determined edge detection model for processing, and output the color block attribute information of the first picture.

5. The method according to any one of claims 1-4, characterized in that, Creating the schedule information of the first picture based on the filtered text recognition result of the first picture, the filtered edge recognition result, and the picture category, includes: Based on the coordinates of each remaining color block in the filtered edge recognition result and the text line coordinates in the text recognition result of the first picture, match the text line recognition content corresponding to each remaining color block from the filtered text recognition result of the first picture; Determine the scene corresponding to the first picture according to the text line recognition content corresponding to each remaining color block and the picture category of the first picture; Based on the scene corresponding to the first picture, perform text splicing on the text line recognition content corresponding to each remaining color block to obtain spliced text; Construct a prompt based on the spliced text and the picture category of the first picture, where the prompt includes scene description information for describing the scene corresponding to the first picture; Input the prompt into a natural language recognition model for processing to extract the schedule information in the first picture; Create the schedule information.

6. The method according to claim 5, characterized in that, Performing text splicing on the text line recognition content corresponding to each remaining color block based on the scene corresponding to the first picture to obtain spliced text, includes: In the case where the scene corresponding to the first picture is a ticket booking scene, for any one of the remaining color blocks, if the any one of the remaining color blocks includes multiple text lines, perform text splicing line by line; During the splicing process, if there is interactive button text in any one of the remaining color blocks, delete the interactive button text, where the interactive button text is the text line recognition content in the interactive button, the interactive button corresponds to a single text line and the length of the text line is less than the length threshold and the text line recognition content is a verb.

7. The method according to claim 5 or 6, characterized in that, After displaying the schedule information, further includes: In response to an editing operation on the schedule title, display a target interface, where the target interface includes the spliced text; In response to a selection operation on the text line content displayed in the target interface, input the text selected by the selection operation into a title input box; In response to an editing end operation, modify the schedule title to the content input in the title input box.

8. The method according to any one of claims 1-7, characterized in that, The picture categories that the target recognition model can recognize also include instant messaging chat screenshots, instant messaging notification card screenshots, and other category pictures. The instant messaging chat screenshots refer to the pictures obtained by taking screenshots of the chat interfaces in instant messaging applications. The instant messaging notification card screenshots refer to the pictures obtained by taking screenshots of the service notification cards in instant messaging applications. The other category pictures include other pictures except the instant messaging chat screenshots, the instant messaging notification card screenshots, and the order screenshots.

9. An electronic device, characterized in that, The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method described in any one of claims 1-8 is implemented.

10. A computer-readable storage medium, characterized in that, Instructions are stored in the computer-readable storage medium. When it runs on a computer, the computer is made to execute the method described in any one of claims 1-8.

Images