Schedule creation method and device, equipment and medium

Abstract

The invention discloses a schedule creation method and device, equipment and a medium, and belongs to the technical field of computers. The method comprises the following steps: receiving a first input aiming at a first image, wherein the first image comprises schedule information; in response to the first input, segmenting the first image to obtain a plurality of sub-images; performing schedule information extraction on the plurality of sub-images to obtain first schedule information; a schedule is created based on the first schedule information.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and specifically relates to a schedule creation method, device, equipment and medium. Background Art

[0002] With the development of electronic devices, users are gradually accustomed to creating schedules in electronic devices and using electronic devices to remind schedules.

[0003] Currently, in order to create a schedule more conveniently, text recognition is usually performed on an image containing schedule information to extract the text information, and the schedule is created based on the extracted text information.

[0004] However, this method is only applicable to images with specific formats, such as train ticket images, air ticket images, screenshots of order pages of ticket booking software, etc. For images that do not have specific formats, it is impossible to accurately create the corresponding schedule, and this method has a smaller scope of application. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a schedule creation method, device, equipment and medium, which can accurately extract schedule information from images that do not have a specific format and accurately create a corresponding schedule.

[0006] In a first aspect, an embodiment of the present application provides a schedule creation method, the method comprising:

[0007] receiving a first input for a first image, the first image including schedule information;

[0008] In response to the first input, the first image is segmented to obtain a plurality of sub-images;

[0009] Extracting schedule information from each of the plurality of sub-images to obtain first schedule information;

[0010] A schedule is created based on the first schedule information.

[0011] In a second aspect, an embodiment of the present application provides a schedule creation device, the device comprising:

[0012] A receiving module, configured to receive a first input for a first image, wherein the first image includes schedule information;

[0013] A segmentation module, configured to segment the first image in response to the first input to obtain a plurality of sub-images;

[0014] An extraction module, used to extract schedule information from the multiple sub-images respectively to obtain first schedule information;

[0015] The creation module is used to create a schedule based on the first schedule information.

[0016] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0017] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0018] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0019] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0020] In the embodiment of the present application, the first image containing the schedule information can be segmented to obtain multiple sub-images, and then the schedule information can be extracted from the multiple sub-images to obtain the first schedule information, and then the schedule can be created based on the first schedule information. In this way, by segmenting the entire large image into multiple small sub-images and extracting the schedule information from the multiple small sub-images respectively, the extraction of the schedule information can be made more detailed, so that accurate schedule information can be extracted from images that do not have a specific format, and then the corresponding schedule can be accurately created, which has a wide range of applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a flowchart of a schedule creation method according to an exemplary embodiment;

[0022] Figure 2 This is one of the scenario schematic diagrams of a schedule creation method according to an exemplary embodiment;

[0023] Figure 3 is a second schematic diagram of a scenario of a schedule creation method according to an exemplary embodiment;

[0024] Figure 4 is a schematic diagram of a first image according to an exemplary embodiment;

[0025] Figure 5 is one of schematic diagrams showing a sub-image according to an exemplary embodiment;

[0026] Figure 6is a second schematic diagram of a sub-image according to an exemplary embodiment;

[0027] Figure 7 is a third schematic diagram of a scenario of a schedule creation method according to an exemplary embodiment;

[0028] Figure 8 This is a fourth schematic diagram of a scenario of a schedule creation method according to an exemplary embodiment;

[0029] Fig. 9 is a structural block diagram of a schedule creation device according to an exemplary embodiment;

[0030] Fig.10 is a structural block diagram of an electronic device according to an exemplary embodiment;

[0031] Fig.11 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0033] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0034] As the background technology shows, with the development of electronic devices, users are gradually accustomed to creating schedules in electronic devices and using electronic devices to remind schedules. Currently, in order to create schedules more conveniently, text recognition is usually performed on images containing schedule information to extract text information, and schedules are created based on the extracted text information.

[0035] However, in the process of extracting schedule information through images, a series of complex problems are often encountered, which significantly affect the extraction effect and user experience. First, the large size of the image file may lead to slower processing speed, increasing the system's computing burden and time cost; second, the text content in the image may be arranged in a disorderly manner, and different fonts, colors and backgrounds make text recognition more difficult; in addition, blurry images or other quality problems (such as dark light or reflection) will further increase the difficulty of identifying text and key information. These problems together lead to a significant decrease in the accuracy and efficiency of the schedule information extraction process. For example, an image of an event announcement taken by a user may contain important information such as the start time and location, which may be hidden in an inconspicuous position or cannot be accurately identified due to blur. In this case, it is not only difficult for users to quickly obtain the required schedule information, but they may also miss important event arrangements, bringing many inconveniences and troubles to their lives and work.

[0036] The main method of extracting schedule information through images is to obtain images that meet the preset format requirements (such as images of tickets such as train tickets and airline tickets), or screenshots of specific pages in a specified application (such as screenshots of the order page of a ticket booking software); after obtaining these specific scene images that meet the requirements, perform image text recognition and feature extraction, extract text information from the image, and then use preset rules or classifiers for matching processing to generate corresponding schedule information, and display the results to the user for confirmation and modification. After the user confirms, the corresponding schedule can be created.

[0037] However, the limitation of this method is that it can only process images with relatively fixed formats and regular information, such as tickets for booking tickets or hotel reservations. If the schedule information is presented in a more complex or irregular form, such as notices written on a blackboard, handwritten memos, or other unstructured scene images, this method cannot effectively process it and cannot extract schedule information from it and generate the corresponding schedule. Therefore, the current technical solutions are difficult to cope with the information extraction needs in diverse and complex scenarios, resulting in inefficiency and cumbersome operations for users when creating schedules.

[0038] The embodiments of the present application provide a schedule creation method, device, equipment and medium, which can segment a first image containing schedule information to obtain multiple sub-images, then extract schedule information from the multiple sub-images respectively to obtain first schedule information, and then create a schedule based on the first schedule information. In this way, by segmenting a plurality of small sub-images from the entire large image and extracting schedule information from the multiple small sub-images respectively, the extraction of schedule information can be made more detailed, so that accurate schedule information can be extracted from images that do not have a specific format, and then the corresponding schedule can be accurately created, which has a wide range of applications.

[0039] The schedule creation method, device, equipment and medium provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.

[0040] The schedule creation method provided in the embodiment of the present application can be applied to the application scenario of creating a schedule based on pictures.

[0041] The schedule creation method provided in the embodiment of the present application can be executed by an electronic device. It should be noted that the above execution entity does not constitute a limitation on the present application.

[0042] Combine the following Figure 1 The schedule creation method provided in the embodiment of the present application is described in detail.

[0043] Figure 1 The figure is a flowchart of a method for creating a schedule according to an exemplary embodiment.

[0044] like Figure 1 As shown, the schedule creation method may include the following steps:

[0045] Step 110: Receive a first input for a first image.

[0046] Here, the first image may include schedule information. The first input may be an input for triggering creation of a schedule based on the first image.

[0047] For example, Figure 2 As shown, after the user takes a photo or downloads the first image, the first image can be saved in the album. After opening the first image in the album, the electronic device can display the first image 210 and the "schedule identification" control 220, and the first input can be an input of clicking the "schedule identification" control 220.

[0048] For example, Figure 3 As shown, the first input can also send the first image 210 to the smart assistant of the electronic device and send a message instruction 310 "Please help me extract schedule information" to the smart assistant.

[0049] Step 120 , in response to the first input, segment the first image to obtain a plurality of sub-images.

[0050] Here, the sub-image may include part of the schedule information, or may include other characters in addition to the schedule information.

[0051] In the related art, schedule information can only be extracted from images with fixed formats and structure (such as train ticket images, airline ticket images, etc.), and it is unable to cope with complex and unstructured scenes. In the embodiment of the present application, by dividing the entire large image into multiple small images, the processing capability of diversified images such as handwritten blackboards and bulletin boards can be enhanced, so that schedule information can be extracted from diversified images, effectively expanding the scope of application of schedule creation.

[0052] In some implementations, the first image may include multiple characters, and step 120 may include:

[0053] Based on the correlation between the multiple characters and the positions corresponding to the multiple characters, the multiple characters are divided to obtain multiple character strings;

[0054] Image regions corresponding to a plurality of character strings are segmented from the first image to obtain a plurality of sub-images.

[0055] Here, the sub-image can contain a string.

[0056] Among the multiple characters, characters whose positions are connected and whose correlation is higher than a threshold value can be divided into a character string. The image area where each character string is located can be used as an image area corresponding to a sub-image, and the image area corresponding to each sub-image can be cut out from the first image to obtain multiple sub-images. For example, a character string can be a word, a phrase or a sentence.

[0057] Specifically, a deep learning model (Character Region Awareness for Text Detection, CRAFT) can be used to detect the character region in the first image. The CRAFT model can accurately locate each character in the image by learning the connection information between the character region and the characters. The CRAFT model can not only detect the position of a single character, but also identify the relationship between characters, and can more accurately identify the character region in a complex background.

[0058] The character region can refer to the boundary or position of each character; the connection information between characters can describe the relationship between characters, indicate whether multiple characters belong to the same string, and help the CRAFT model determine whether characters should be connected together. Assuming there is a word "HELLO", the CRAFT model will first detect the position of each character "H", "E", "L", "L", "O", and then determine that these characters are continuous and belong to the same word based on the connection information between the characters.

[0059] The CRAFT model can output the bounding box coordinates (x min ,y min ,xmax ,y max ), according to the bounding box coordinates output by the CRAFT model, multiple sub-images can be cut out from the first image. Each sub-image can contain an image area corresponding to a bounding box, thereby ensuring that the main body of the string is accurately separated, which is convenient for subsequent schedule information extraction.

[0060] Among them, x min y may be the horizontal coordinate of the left edge of the bounding box, indicating the leftmost position of the bounding box in the first image; min x may be the ordinate of the upper edge of the bounding box, indicating the uppermost position of the bounding box in the first image; max y may be the horizontal coordinate of the right edge of the bounding box, indicating the rightmost position of the bounding box in the first image; max It can be the ordinate of the lower edge of the bounding box, indicating the bottom position of the bounding box in the first image; (x min ,y min ,x max ,y max ) can define a rectangular bounding box to enclose the image area where the string is located.

[0061] For example, Figure 4 As shown, the bounding boxes determined from the first image may include bounding box 410, bounding box 420, and bounding box 430. Bounding box 410 is the bounding box of the string "XX Concert", bounding box 420 is the bounding box of the string "10.02", and bounding box 430 is the bounding box of the string "XX Camp".

[0062] In this way, by dividing multiple characters based on the correlation between multiple characters and the positions corresponding to the multiple characters, it is possible to ensure that the character string body is accurately separated, avoiding characters that should belong to one character string from being segmented into different sub-images, resulting in inaccurate subsequent schedule information extraction.

[0063] In some implementations, after responding to the first input and before segmenting the first image to obtain a plurality of sub-images, the method may further include:

[0064] A preprocessing operation is performed on the first image.

[0065] Here, the preprocessing operation may include at least one of noise removal, image enhancement, and geometric correction.

[0066] Specifically, noise can be removed by Gaussian filtering; clarity can be improved by adjusting at least one of brightness, contrast and sharpness to achieve image enhancement; and set correction can be achieved by edge detection and perspective transformation to correct rotated or tilted images.

[0067] Wherein, Gaussian filtering can be expressed as the following formula (1):

[0068]

[0069] is the Gaussian function, I(x,y) is the pixel value of the first image before Gaussian filtering, I filtered (x,y) is the pixel value of the first image after Gaussian filtering. σ is the standard deviation in the Gaussian function, which controls the "width" or "diffusion" of the Gaussian distribution. The larger the σ value, the stronger the smoothing effect of the filter and the greater the blur; the smaller the σ value, the smaller the smoothing range of the filter and the weaker the blur. σ can directly affect the weight distribution of the filter kernel. i is the horizontal offset of the pixel in the first image, and j is the vertical offset of the pixel in the first image. i and j can be used to describe the relative position of the filter kernel. i and j can represent the horizontal and vertical distances of the filter kernel relative to the current processing pixel. In other words, when applying Gaussian filtering, the neighborhood of the current pixel is defined by the relative offsets i and j. The Gaussian function calculates the weight of each neighborhood pixel based on these distances. K can determine the size of the filter, which is usually a positive integer representing the radius of the filter kernel. K can determine the size of the area covered by the filter kernel. In general, the size of the filter kernel is a (2k+1)×(2k+1) matrix. For example, k=1 corresponds to a 3×3 filter window, and k=2 corresponds to a 5×5 filter window. As k increases, the filter kernel becomes larger and the range of influence increases. This formula represents a two-dimensional convolution operation, which is a weighted sum of the surrounding pixels according to the weight of the Gaussian kernel. These variables work together to determine how the Gaussian filter smooths each pixel in the first image.

[0070] The main purpose of edge detection is to enhance the structural and contour information in the image and help locate the boundaries of the character area more accurately. Specifically, for images with low contrast between the character area and the background, or complex backgrounds (such as natural scenes, cluttered images), edge detection can help identify and highlight the boundaries of the characters, making subsequent character recognition more accurate, thereby accurately locating the character area; for images with complex backgrounds, when there are many interfering elements in the image background, edge detection can also help separate the character area and reduce the interference of unnecessary background information; edge detection can also assist in perspective transformation.

[0071] In this way, by performing preprocessing operations such as noise removal, image enhancement, and geometric correction, the image quality of the first image can be significantly improved, thereby enhancing the visibility of the character area in the first image and improving the accuracy of character recognition.

[0072] Step 130 , extracting schedule information from each of the plurality of sub-images to obtain first schedule information.

[0073] Here, the first schedule information may be schedule information included in the first image.

[0074] Each sub-image may contain at least part of the schedule information, and may also contain other characters in addition to the schedule information. Therefore, by extracting the schedule information from the multiple sub-images respectively, the schedule information in the first image, ie, the first schedule information, can be obtained.

[0075] Exemplarily, the first schedule information may include at least one of date information, time information, location information, theme information, etc. The theme information may be an activity name.

[0076] In some implementations, before step 130, the method may further include:

[0077] Perform clarity restoration on multiple sub-images separately.

[0078] Here, for each of the multiple sub-images, clarity restoration can be performed using an enhanced super-resolution generative adversarial network (Real-ESRGAN) model.

[0079] Specifically, each sub-image can be input into the Real-ESRGAN model, which can improve the resolution of each sub-image and enhance the details and texture of the sub-image through generative adversarial network technology. The Real-ESRGAN model can effectively generate high-resolution and clear sub-images based on low-resolution sub-images, eliminate blur and noise, and improve the edges and details of the character area.

[0080] For example, the sub-image before definition restoration can be Figure 5 As shown, the sub-image after definition restoration can be Figure 6 shown.

[0081] In this way, the clarity of the sub-image can be improved by performing clarity restoration on the sub-image, which facilitates the subsequent extraction of schedule information and improves the accuracy of schedule information extraction.

[0082] In some implementations, step 130 may include:

[0083] Filtering a target sub-image containing schedule information from a plurality of sub-images;

[0084] Schedule information is extracted from the target sub-image to obtain first schedule information.

[0085] Here, the target sub-image is a sub-image including schedule information.

[0086] Some of the multiple sub-images may not contain schedule information. Therefore, the target sub-images containing schedule information may be first screened from the multiple sub-images, sub-images not containing schedule information may be filtered out, and then schedule extraction may be performed on the target sub-images containing schedule information.

[0087] In this way, by first screening out the target sub-images containing schedule information and then performing schedule extraction on the target sub-images containing schedule information, schedule information extraction on sub-images not containing schedule information can be avoided, thereby improving the efficiency of schedule information extraction.

[0088] In some implementations, screening a target sub-image containing schedule information from a plurality of sub-images may include:

[0089] Performing character recognition on the multiple sub-images respectively to obtain target characters corresponding to the multiple sub-images respectively;

[0090] Extracting schedule information from target characters corresponding to the plurality of sub-images respectively, to obtain second schedule information corresponding to the plurality of sub-images respectively;

[0091] From the plurality of sub-images, a target sub-image whose second schedule information meets a preset condition is selected.

[0092] Here, the second schedule information corresponding to each sub-image may be the schedule information contained in the sub-image. The target sub-image may include one or more sub-images.

[0093] Specifically, for each sub-image, optical character recognition (OCR) can be performed on the sub-image to obtain the characters contained in the sub-image, that is, the target characters. Then, the target characters can be analyzed using natural language processing (NLP) technology to identify key schedule information, that is, the second schedule information, such as at least one of date information, time information, location information, and subject information. By analyzing the target characters and identifying the schedule information through NLP, the schedule information can be structured for further processing.

[0094] Then, a target sub-image whose second schedule information meets a preset condition may be screened from the plurality of sub-images.

[0095] The preset condition may include: the second schedule information contains a preset schedule keyword that meets a preset format, or the relevance between the second schedule information and the schedule information is greater than a relevance threshold.

[0096] The preset schedule keywords may be pre-set keywords, which may be words or phrases. The preset schedule keywords may include at least one of a date keyword, a time keyword, a location keyword, and a theme keyword. For example, the preset schedule keywords may include "2024", "2 p.m.", etc.

[0097] The preset format may be a pre-set format, and different preset schedule keywords may be pre-set with different formats. For example, the preset formats of date keywords may include "YYYY-MM-DD" and "MM / DD / YYYY", and the preset formats of time keywords may be "HH" and "AM / PM".

[0098] Specifically, it may be checked whether the extracted second schedule information contains preset schedule-related keywords that meet a preset format.

[0099] The relevance threshold can be set according to actual needs and is not limited here.

[0100] Specifically, a fine-tuned Bidirectional Encoder Representations from Transformers (Bert) model can be used to perform correlation analysis on the second schedule information to evaluate whether the second schedule information in each sub-image meets the requirements of schedule information, that is, to evaluate whether the second schedule information in each sub-image contains at least one of time, place, and event, so as to evaluate the correlation between the second schedule information and schedule information.

[0101] Exemplarily, for each sub-image, if the second schedule information contained in the sub-image contains preset schedule keywords that meet the preset format, or the correlation between the second schedule information contained in the sub-image and the schedule information is greater than the correlation threshold, the sub-image can be determined as a target sub-image. If the second schedule information contained in the sub-image does not contain preset schedule keywords that meet the preset format, and the correlation between the second schedule information contained in the sub-image and the schedule information is less than or equal to the correlation threshold, the sub-image can be determined as a non-target sub-image. Non-target sub-images can be marked, and these non-target sub-images can be discarded in subsequent processing, and target sub-images can be retained.

[0102] In this way, by screening the target sub-image containing schedule information from multiple sub-images through the above process, the accuracy of screening can be improved.

[0103] In some implementations, extracting schedule information from the target sub-image to obtain the first schedule information may include:

[0104] The target sub-image is input into the multimodal large model, and the schedule information of the target sub-image is extracted by the multimodal large model to obtain the first schedule information.

[0105] Here, the selected target sub-image can be input into the multimodal large model for schedule information extraction. The multimodal large model can identify and extract schedule information, such as at least one of date information, time information, location information, and subject information, by comprehensively analyzing the visual features in the target sub-image and the text data extracted by OCR. The multimodal large model can use its multimodal processing capabilities to fuse image and text data, perform in-depth analysis, and finally output structured schedule information to ensure the accuracy and completeness of the extraction results.

[0106] The related technologies are limited to text matching, and the scope of information extraction is limited. However, in the embodiments of the present application, a multimodal large model can be used to perform in-depth analysis on the target sub-image, and a variety of information such as image content, text, and background can be used to achieve more comprehensive and accurate schedule information extraction, breaking through the limitations of single text recognition.

[0107] Exemplarily, the multimodal large model can be Generative Pre-trained Transformer-4Omni (GPT-4O).

[0108] In this way, by introducing a large multimodal model and conducting in-depth analysis of the screened target sub-images, more comprehensive and accurate schedule information extraction can be achieved, breaking through the limitations of single text recognition and improving the accuracy of schedule information extraction.

[0109] Step 140: Create a schedule based on the first schedule information.

[0110] Here, after a schedule is created based on the first schedule information, the schedule may be displayed, and a schedule reminder may be issued based on the schedule.

[0111] In some implementations, step 140 may include:

[0112] Display the schedule information modification interface;

[0113] receiving a second input to the schedule information modification interface;

[0114] In response to the second input, modify the first schedule information to obtain modified first schedule information;

[0115] A schedule is created based on the modified first schedule information.

[0116] Here, the schedule information modification interface may include the first schedule information.

[0117] The second input may be an input for modifying the first schedule information in the schedule modification interface.

[0118] For example, if the first schedule information displayed in the schedule information modification interface includes "Time: October 2nd Location: YY Camp Theme: XX Concert", the user can use the second input to change "Time: October 2nd Location: YY Camp Theme: XX Concert" to "Time: October 2nd Location: XX Camp Theme: XX Concert", and then create a schedule based on the modified first schedule information.

[0119] In this way, by displaying the schedule information modification interface after obtaining the first schedule information, the user can quickly modify the schedule information that is identified as inaccurate.

[0120] In some implementations, displaying a schedule information modification interface may include:

[0121] Display schedule information interface;

[0122] receiving a third input to the first control;

[0123] In response to the third input, a schedule information modification interface is displayed.

[0124] Here, the schedule information interface may include first schedule information and a first control.

[0125] The first control may be a control for triggering the electronic device to display a schedule information modification interface. Exemplarily, the first control may be a "modify" control.

[0126] The third input may be used to trigger the electronic device to display the schedule information modification interface. Exemplarily, the third input may be an input of clicking the first control.

[0127] For example, Figure 7 As shown, after obtaining the first schedule information, the electronic device may display a schedule information interface 710, which may include the first schedule information 711 and a "modify" control 712. If the user wants to modify the first schedule information, the user may click the "modify" control 712 to enter the schedule information modification interface.

[0128] In this way, by displaying the schedule information interface, the user can select whether to enter the schedule information modification interface according to whether the first schedule information needs to be modified, so as to avoid directly displaying the schedule information modification interface when no modification is required, which causes inconvenience to the user.

[0129] In some implementations, step 140 may include:

[0130] Display schedule information interface;

[0131] receiving a fourth input to the second control;

[0132] In response to the fourth input, a schedule is created based on the first schedule information.

[0133] Here, the schedule information interface may include first schedule information and a second control.

[0134] The second control may be a control for triggering the electronic device to create a schedule based on the first schedule information. Exemplarily, the second control may be a "correct" control.

[0135] The fourth input may be used to trigger the electronic device to create a schedule based on the first schedule information. Exemplarily, the fourth input may be an input of clicking the second control.

[0136] For example, Figure 7 As shown in FIG. 7 , after obtaining the first schedule information, the electronic device may display a schedule information interface 710, which may include the first schedule information 711 and a “correct” control 713. If the user determines that the first schedule information is correct and wants to create a schedule based on the first schedule information, the user may click the “confirm” control 713, and the electronic device may create a schedule based on the first schedule information 711, and then Figure 8 As shown, the electronic device may also display a schedule interface 810 , and the schedule interface 810 may include first schedule information 811 .

[0137] It should be noted that the input in the embodiments of the present application can be a click input, a voice command input, or a specific gesture input, which can be determined according to actual use requirements, and the embodiments of the present application do not limit this. Among them, the specific gesture can be any one of a single-click gesture, a sliding gesture, a dragging gesture, a pressure recognition gesture, a long press gesture, an area change gesture, a double press gesture, and a double-click gesture; the click input can be a single-click input, a double-click input, or any number of click inputs, etc., and can also be a long press input or a short press input.

[0138] In the embodiments of the present application, the image complexity problem existing in the related art can be effectively overcome, and the extraction effect and automation level of schedule information can be significantly improved. The related art usually directly relies on OCR for recognition, which is easily affected by factors such as image blur and messy text layout. The embodiments of the present application can reduce recognition errors and improve the accuracy of character recognition by introducing a multi-stage strategy in the image processing process: image segmentation, clarity restoration, sub-image screening, etc., thereby improving the accuracy and efficiency of extracting schedule information from complex images.

[0139] In the embodiments of the present application, the accuracy and efficiency of schedule information extraction are significantly improved by intelligently identifying and processing schedule information in complex images. Through intelligent image segmentation and clarity restoration technology, complex images are decomposed into multiple sub-images for processing, thereby improving the impact of problems such as image blur and cluttered text on the recognition effect, ensuring efficient and accurate extraction of schedule information. By utilizing the deep analysis function of the multimodal large model, schedule information is automatically screened and identified, which greatly simplifies the user's operating process and reduces the need for manual intervention. The embodiments of the present application improve the level of automation in schedule information extraction and enhance the intelligence of the user experience, meeting the needs of users to efficiently create schedules in complex scenarios.

[0140] The schedule creation method provided in the embodiment of the present application can be executed by a schedule creation device. In the embodiment of the present application, the schedule creation device provided in the embodiment of the present application is described by taking the schedule creation method executed by the schedule creation device as an example.

[0141] Based on the same inventive concept, the present application also provides a schedule creation device. Fig. 9 The schedule creation device provided in the embodiment of the present application is described in detail.

[0142] Fig. 9 The figure is a structural block diagram of a schedule creation device according to an exemplary embodiment.

[0143] like Fig. 9 As shown, the schedule creation device 900 can be applied to an electronic device, and the schedule creation device 400 can include:

[0144] A receiving module 901 is configured to receive a first input for a first image, where the first image includes schedule information;

[0145] A segmentation module 902, configured to segment the first image to obtain a plurality of sub-images in response to a first input;

[0146] An extraction module 903 is used to extract schedule information from the multiple sub-images respectively to obtain first schedule information;

[0147] The creation module 904 is used to create a schedule based on the first schedule information.

[0148] The schedule creation device 900 is described in detail below, as follows:

[0149] In one embodiment, the first image includes a plurality of characters, and the segmentation module 902 may include:

[0150] A division submodule, for dividing the multiple characters based on the correlation between the multiple characters and the positions corresponding to the multiple characters, to obtain multiple character strings;

[0151] The sub-slicing module is used to slice the image regions corresponding to the multiple character strings from the first image to obtain multiple sub-images.

[0152] In one embodiment, the extraction module 903 may include:

[0153] A screening submodule, used for screening a target sub-image containing schedule information from a plurality of sub-images;

[0154] The extraction submodule is used to extract schedule information from the target sub-image to obtain first schedule information.

[0155] In one embodiment, the screening submodule may include:

[0156] A recognition unit, used for performing character recognition on the multiple sub-images respectively to obtain target characters corresponding to the multiple sub-images respectively;

[0157] An extraction unit, used to extract schedule information from target characters corresponding to the plurality of sub-images respectively, to obtain second schedule information corresponding to the plurality of sub-images respectively;

[0158] The screening unit is used to screen the target sub-image whose second schedule information meets the preset conditions from the multiple sub-images, wherein the preset conditions include: the second schedule information contains preset schedule-type keywords that meet the preset format, or the correlation between the second schedule information and the schedule-type information is greater than the correlation threshold.

[0159] In one embodiment, the creation module 904 may include:

[0160] A display submodule, used for displaying a schedule information modification interface, wherein the schedule information modification interface includes the first schedule information;

[0161] A receiving submodule, used for receiving a second input to the schedule information modification interface;

[0162] A modification submodule, configured to modify the first schedule information in response to the second input to obtain modified first schedule information;

[0163] The creation submodule is used to create a schedule based on the modified first schedule information.

[0164] In one embodiment, the display submodule may include:

[0165] A first display unit, used to display a schedule information interface, wherein the schedule information interface includes first schedule information and a first control;

[0166] A receiving unit, configured to receive a third input to the first control;

[0167] The second display unit is used to display a schedule information modification interface in response to a third input.

[0168] Thus, the first image containing the schedule information can be segmented to obtain a plurality of sub-images, and then the schedule information can be extracted from each of the plurality of sub-images to obtain the first schedule information, and then a schedule can be created based on the first schedule information. In this way, by segmenting a plurality of small sub-images from the entire large image and extracting schedule information from each of the plurality of small sub-images, the schedule information can be extracted more meticulously, so that accurate schedule information can be extracted from images that do not have a specific format, and then the corresponding schedule can be accurately created, and the application range is relatively wide.

[0169] The schedule creation device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or it can be other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0170] The schedule creation device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0171] The schedule creation device provided in the embodiment of the present application can achieve Figure 1-8 The various processes implemented in the method embodiment achieve the same technical effect and will not be described again here to avoid repetition.

[0172] In some embodiments, Fig.10As shown, an embodiment of the present application further provides an electronic device 1000, including a processor 1001 and a memory 1002, wherein the memory 1002 stores a program or instruction that can be executed on the processor 1001, and when the program or instruction is executed by the processor 1001, the various steps of the above-mentioned schedule creation method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0173] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0174] Fig.11 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0175] The electronic device 1100 includes but is not limited to: a radio frequency unit 1101, a network module 1102, an audio output unit 1103, an input unit 1104, a sensor 1105, a display unit 1106, a user input unit 1107, an interface unit 1108, a memory 1109, and a processor 1110 and other components.

[0176] Those skilled in the art will appreciate that the electronic device 1100 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 1110 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig.11 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0177] The user input unit 1107 is used to receive a first input for a first image, where the first image includes schedule information;

[0178] The processor 1110 is configured to segment the first image into multiple sub-images in response to the first input;

[0179] Extracting schedule information from each of the plurality of sub-images to obtain first schedule information;

[0180] A schedule is created based on the first schedule information.

[0181] Thus, the first image containing the schedule information can be segmented to obtain a plurality of sub-images, and then the schedule information can be extracted from each of the plurality of sub-images to obtain the first schedule information, and then a schedule can be created based on the first schedule information. In this way, by segmenting a plurality of small sub-images from the entire large image and extracting schedule information from each of the plurality of small sub-images, the schedule information can be extracted more meticulously, so that accurate schedule information can be extracted from images that do not have a specific format, and then the corresponding schedule can be accurately created, and the application range is relatively wide.

[0182] In some embodiments, the processor 1110 is further configured to divide the multiple characters based on the correlation between the multiple characters and the positions corresponding to the multiple characters to obtain multiple character strings;

[0183] Image regions corresponding to a plurality of character strings are segmented from the first image to obtain a plurality of sub-images.

[0184] In this way, by dividing multiple characters based on the correlation between multiple characters and the positions corresponding to the multiple characters, it is possible to ensure that the character string body is accurately separated, avoiding characters that should belong to one character string from being segmented into different sub-images, resulting in inaccurate subsequent schedule information extraction.

[0185] In some embodiments, the processor 1110 is further configured to filter a target sub-image containing schedule information from the plurality of sub-images;

[0186] Schedule information is extracted from the target sub-image to obtain first schedule information.

[0187] In this way, by first screening out the target sub-images containing schedule information and then performing schedule extraction on the target sub-images containing schedule information, schedule information extraction on sub-images not containing schedule information can be avoided, thereby improving the efficiency of schedule information extraction.

[0188] In some embodiments, the processor 1110 is further configured to perform character recognition on the multiple sub-images respectively to obtain target characters corresponding to the multiple sub-images respectively;

[0189] Extracting schedule information from target characters corresponding to the plurality of sub-images respectively, to obtain second schedule information corresponding to the plurality of sub-images respectively;

[0190] From the plurality of sub-images, a target sub-image whose second schedule information meets a preset condition is selected, wherein the preset condition includes: the second schedule information contains a preset schedule-type keyword meeting a preset format, or the relevance between the second schedule information and schedule-type information is greater than a relevance threshold.

[0191] In this way, by screening the target sub-image containing schedule information from multiple sub-images through the above process, the accuracy of screening can be improved.

[0192] In some embodiments, the display unit 1106 is used to display a schedule information modification interface, where the schedule information modification interface includes the first schedule information;

[0193] The user input unit 1107 is further used to receive a second input to the schedule information modification interface;

[0194] The processor 1110 is further configured to modify the first schedule information in response to the second input to obtain modified first schedule information;

[0195] A schedule is created based on the modified first schedule information.

[0196] In this way, by displaying the schedule information modification interface after obtaining the first schedule information, the user can quickly modify the schedule information that is identified as inaccurate.

[0197] In some embodiments, the display unit 1106 is further used to display a schedule information interface, where the schedule information interface includes first schedule information and a first control;

[0198] The user input unit 1107 is further used to receive a third input to the first control;

[0199] The display unit 1106 is further configured to display a schedule information modification interface in response to the third input.

[0200] In this way, by displaying the schedule information interface, the user can select whether to enter the schedule information modification interface according to whether the first schedule information needs to be modified, so as to avoid directly displaying the schedule information modification interface when no modification is required, which causes inconvenience to the user.

[0201] It should be understood that in the embodiment of the present application, the input unit 1104 may include a graphics processor (Graphics Processing Unit, GPU) 11041 and a microphone 11042, and the graphics processor 11041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 1106 may include a display panel 11061, and the display panel 11061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 1107 includes a touch panel 11071 and at least one of other input devices 11072. The touch panel 11071 is also called a touch screen. The touch panel 11071 may include two parts: a touch detection device and a touch controller. Other input devices 11072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0202] The memory 1109 can be used to store software programs and various data. The memory 1109 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 1109 may include a volatile memory or a non-volatile memory, or the memory 1109 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 1109 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0203] The processor 1110 may include one or more processing units; in some embodiments, the processor 1110 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 1110.

[0204] The embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned schedule creation method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0205] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory, a random access memory, a magnetic disk or an optical disk.

[0206] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned schedule creation method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0207] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0208] An embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the various processes of the above-mentioned schedule creation method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0209] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0210] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0211] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A schedule creation method, characterized in that: The method comprises: receiving a first input for a first image, wherein the first image includes schedule information; In response to the first input, segment the first image to obtain a plurality of sub-images; Extracting schedule information from each of the plurality of sub-images to obtain first schedule information; A schedule is created based on the first schedule information.

2. The method according to claim 1, characterized in that The first image contains a plurality of characters, and the first image is segmented to obtain a plurality of sub-images, including: Based on the correlation between the multiple characters and the positions corresponding to the multiple characters, the multiple characters are divided to obtain multiple character strings; Image regions corresponding to the plurality of character strings are segmented from the first image to obtain the plurality of sub-images.

3. The method according to claim 1, characterized in that The extracting schedule information from the plurality of sub-images respectively to obtain first schedule information includes: Filtering a target sub-image containing schedule information from the plurality of sub-images; Schedule information is extracted from the target sub-image to obtain the first schedule information.

4. The method according to claim 3, characterized in that The step of selecting a target sub-image containing schedule information from the plurality of sub-images includes: Performing character recognition on the multiple sub-images respectively to obtain target characters corresponding to the multiple sub-images respectively; Extracting schedule information from the target characters corresponding to the multiple sub-images respectively, to obtain second schedule information corresponding to the multiple sub-images respectively; From the multiple sub-images, a target sub-image whose second schedule information meets a preset condition is selected, wherein the preset condition includes: the second schedule information contains a preset schedule-type keyword that meets a preset format, or the relevance between the second schedule information and schedule-type information is greater than a relevance threshold.

5. The method according to claim 1, characterized in that The creating a schedule based on the first schedule information includes: Displaying a schedule information modification interface, wherein the schedule information modification interface includes the first schedule information; receiving a second input to the schedule information modification interface; In response to the second input, modify the first schedule information to obtain modified first schedule information; A schedule is created based on the modified first schedule information.

6. The method according to claim 5, characterized in that The display schedule information modification interface includes: Displaying a schedule information interface, wherein the schedule information interface includes the first schedule information and a first control; receiving a third input to the first control; In response to the third input, the schedule information modification interface is displayed.

7. A schedule creation device, characterized in that: The device comprises: A receiving module, configured to receive a first input for a first image, wherein the first image includes schedule information; A segmentation module, configured to segment the first image in response to the first input to obtain a plurality of sub-images; An extraction module, configured to extract schedule information from each of the plurality of sub-images to obtain first schedule information; A creation module is used to create a schedule based on the first schedule information.

8. The device according to claim 7, characterized in that The first image contains a plurality of characters, and the segmentation module includes: A division submodule, used for dividing the multiple characters based on the correlation between the multiple characters and the positions corresponding to the multiple characters, so as to obtain multiple character strings; The sub-slicing module is used to slice the image regions corresponding to the multiple character strings from the first image to obtain the multiple sub-images.

9. The device according to claim 7, characterized in that The extraction module comprises: A screening submodule, used for screening a target sub-image containing schedule information from the plurality of sub-images; The extraction submodule is used to extract schedule information from the target sub-image to obtain the first schedule information.

10. The device according to claim 9, characterized in that The screening submodule comprises: A recognition unit, used for performing character recognition on the plurality of sub-images respectively to obtain target characters corresponding to the plurality of sub-images respectively; A first extraction unit is used to extract schedule information from the target characters corresponding to the multiple sub-images respectively, to obtain second schedule information corresponding to the multiple sub-images respectively; A screening unit is used to screen, from the multiple sub-images, a target sub-image whose second schedule information meets a preset condition, wherein the preset condition includes: the second schedule information contains a preset schedule-type keyword that meets a preset format, or the correlation between the second schedule information and schedule-type information is greater than a correlation threshold.

11. The device according to claim 7, characterized in that The creation module includes: A display submodule, used for displaying a schedule information modification interface, wherein the schedule information modification interface includes the first schedule information; A receiving submodule, configured to receive a second input to the schedule information modification interface; a modification submodule, configured to modify the first schedule information in response to the second input to obtain modified first schedule information; The creation submodule is used to create a schedule based on the modified first schedule information.

12. The device according to claim 11, characterized in that The display submodule comprises: A first display unit, configured to display a schedule information interface, wherein the schedule information interface includes the first schedule information and a first control; A receiving unit, configured to receive a third input to the first control; The second display unit is used to display the schedule information modification interface in response to the third input.

13. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the schedule creation method according to any one of claims 1 to 6 are implemented.

14. A readable storage medium, characterized in that: The readable storage medium stores a program or an instruction, and when the program or the instruction is executed by the processor, the steps of the schedule creation method according to any one of claims 1 to 6 are implemented.

Images