A data extraction-based schedule reminding method and device

Abstract

The application discloses a kind of based on data extraction's schedule reminding method and device, the method includes: obtaining multiple types of sample data, for each type, utilize the type sample data to carry out model training, obtain the schedule setting model corresponding to this type;The first output result of multiple types of schedule setting model is fused training through pre-set fusion model, and the fusion schedule setting model is obtained;From user terminal, obtain multiple types of to be analyzed data, utilize corresponding schedule setting model to process to-be-analyzed data, and obtain second output result;Utilize the fusion schedule setting model to process multiple types of second output result, and output recommended schedule.The application supports multiple types of schedule setting input, and is trained to different types of sample data, so that the prediction result of each type of schedule setting model is more accurate;The output result of multiple types of schedule setting model is input into fusion schedule setting model, and the accuracy of recommended result is increased.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data analysis, and in particular to a schedule reminding method and device based on data extraction. BACKGROUND

[0002] In daily life and production, users generate a large number of tasks every day, and as the number of tasks increases, it is particularly important to sort and timely process the tasks; in order to avoid missing or missing important events, users can set a schedule reminder to remind themselves to complete important tasks through software.

[0003] The existing schedule reminder software mainly contains the function of single setting by the user, and the schedule system performs single reminding or simple periodic reminding for the task; such setting is mostly text or voice input, and the processing method of voice input is generally voice to text, which is still essentially text input, the setting method is single, the reminding scene needs to be set manually, and the corresponding setting criteria are matched, resulting in low intelligent degree of schedule setting and few covered scenes. SUMMARY

[0004] In view of the above problems, the present application is proposed in order to provide a schedule reminding method and device based on data extraction which overcomes the above problems or at least partially solves the above problems.

[0005] According to one aspect of the present application, a schedule reminding method based on data extraction is provided, comprising:

[0006] Obtaining a plurality of types of sample data, for each type, using the sample data of the type to perform model training to obtain a schedule setting model corresponding to the type;

[0007] The first output result of the schedule setting model corresponding to the plurality of types is fused and trained through a preset fusion model to obtain a fused schedule setting model;

[0008] Obtaining a plurality of types of to-be-analyzed data from a user end, using the schedule setting model corresponding to the plurality of types to process the to-be-analyzed data of the corresponding type to obtain a second output result of the plurality of types;

[0009] Using the fused schedule setting model to process the second output result of the plurality of types, and outputting a recommended schedule.

[0010] According to another aspect of the present application, a schedule reminding device based on data extraction is provided, comprising:

[0011] A sub-model training module is configured to obtain a plurality of types of sample data, for each type, using the sample data of the type to perform model training to obtain a schedule setting model corresponding to the type;

[0012] The fusion model training module is configured to perform fusion training on first output results of the schedule setting models corresponding to the multiple types by using a preset fusion model to obtain a fusion schedule setting model;

[0013] The sub-model analysis module is configured to obtain multiple types of to-be-analyzed data from a user end, process the to-be-analyzed data of a corresponding type by using the schedule setting model corresponding to the type, and obtain second output results of the multiple types.

[0014] The fusion recommendation module is configured to process the second output results of the multiple types by using the fusion schedule setting model and output a recommended schedule.

[0015] According to another aspect of the present application, a computing device is provided, which comprises a processor, a memory, a communication interface and a communication bus, the processor, the memory and the communication interface being capable of communicating with each other through the communication bus.

[0016] The memory is configured to store at least one executable instruction, and the executable instruction is configured to enable the processor to perform operations corresponding to the above-mentioned data extraction-based schedule reminding method.

[0017] According to still another aspect of the present application, a computer storage medium is provided, and the storage medium stores at least one executable instruction, and the executable instruction is configured to enable a processor to perform operations corresponding to the above-mentioned data extraction-based schedule reminding method.

[0018] According to the data extraction-based schedule reminding method and device, sample data of multiple types are obtained, for each type, a schedule setting model corresponding to the type is obtained by using sample data of the type, first output results of the schedule setting models corresponding to the multiple types are fused by using a preset fusion model to obtain a fusion schedule setting model, multiple types of to-be-analyzed data are obtained from a user end, the to-be-analyzed data of a corresponding type is processed by using the schedule setting model corresponding to the type to obtain second output results of the multiple types, and the second output results of the multiple types are processed by using the fusion schedule setting model to output a recommended schedule. The method supports multiple types of schedule setting input, and different algorithms are used for training according to different types of data input mode, so that the prediction result of each type of schedule setting model is more accurate, and the fusion schedule setting model is input with the processing result of the sample data of the multiple types, so that the accuracy of the recommended result of the fusion schedule setting model is increased.

[0019] The above description is only a summary of the technical scheme of the present application, in order to more clearly understand the technical means of the present application, the specific embodiments of the present application can be implemented according to the content of the description, and in order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application are described below. Attached Figure Description

[0020] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0021] Figure 1 A flowchart of a schedule reminder method based on data extraction provided by an embodiment of the present invention is shown;

[0022] Figure 2 The following is a schedule diagram for February 2, 2021, provided by an embodiment of the present invention;

[0023] Figure 3 This diagram illustrates the training process of the text scheduling model provided in an embodiment of the present invention.

[0024] Figure 4 This diagram illustrates the training process of the voice scheduling model provided in an embodiment of the present invention.

[0025] Figure 5 This diagram illustrates the training process of the image scheduling model provided in an embodiment of the present invention.

[0026] Figure 6 The flowchart shows the recommended schedule generation process when there is a user-inputted text setting statement and it is a one-time setting;

[0027] Figure 7 The flowchart shows the recommended schedule generation process when there is a user-inputted text setting statement and it is a periodic setting;

[0028] Figure 8 The flowchart for generating a recommended schedule is shown when there is no text setting statement entered by the user.

[0029] Figure 9 A schematic diagram of a schedule reminder device based on data extraction provided in an embodiment of the present invention is shown;

[0030] Figure 10 A schematic diagram of the structure of a computing device provided in an embodiment of the present invention is shown. Detailed Implementation

[0031] Exemplary embodiments of the present application will be described herein below with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it is understood that the present application can be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the application to those skilled in the art.

[0032] The present application is composed of a learning layer and an application layer, wherein the learning layer is used for model training; and the application layer outputs schedule setting based on the application of the model.

[0033] Figure 1 A flowchart of an embodiment of a schedule reminding method based on data extraction of the present application is shown in FIG. 1, which comprises the following steps: Figure 1

[0034] Step S110: Obtain sample data of multiple types, for each type, use the sample data of the type to perform model training, and obtain a schedule setting model corresponding to the type.

[0035] Specifically, a schedule setting statement generally includes three input forms: text information such as "remind me at 6:00 am on Tomb Sweeping Day"; voice information including voice information input by the user himself / herself and voice information of an authorized social software automatically obtained; and image information which can be obtained from a relevant album of the user and used to determine the user's daily life routine and habits. Therefore, in the present application, the sample data of multiple types can include input text type, voice type and / or image type, and each type of schedule setting model is obtained by training and learning the sample data of multiple types.

[0036] In an optional embodiment, step S110 further comprises: obtaining a text setting statement and input time information corresponding to the text setting statement; pre-processing and analyzing the text setting statement to determine a periodic attribute label; performing matrix conversion on the input time information; obtaining sample data of the input text type based on the pre-processed text input statement, the periodic attribute label and the matrix-converted input time information; inputting the sample data of the input text type into a decision tree training model for training, and obtaining a text schedule setting model as the schedule setting model corresponding to the input text type.

[0037] In order to make the model prediction more accurate, in the present embodiment, different pre-processing methods and training models are adopted for the sample data of three types.

[0038] ​In this step, time can be expressed in multiple ways. The schedule unit is a time expression method specifically designed for schedule reminders, and generally has schedule features such as year, month, and day. In this embodiment, the description of the schedule unit corresponds to year, month, week, day, hour, and minute (represented as A, B, C, D, E, F), and is divided into lunar calendar (represented as A1, B1, C1, D1) and solar calendar (represented as A2, B2, C1, D2), solar term G (the solar term is converted into a number according to the order of the solar terms of the year, with 0 representing no solar term and numbering in sequence if there is a solar term, for example, New Year's Day is represented as 1, and so on), and working day H (working day is represented as 1, and non-working day is represented as 0). For example, the information contained in the schedule unit for February 12, 2021 is: A1=2021, B1=2, D1=12, A2=2021, B2=1, D2=1, C="Friday", G="4" (according to the above solar term setting rules, 1 is New Year's Day, 2 is Laba Festival, 3 is Lunar New Year's Eve, and 4 is Spring Festival), H=0.

[0039] After obtaining the text setting statement, it is necessary to determine the corresponding input time information. This involves writing the system time at the input moment of the text setting statement into the schedule unit. The system time at the input moment refers to the system time at the exact instant the user inputs the text setting statement. The correct setting time corresponding to the text setting statement, which the user desires, is also written into the schedule unit as a pre-set schedule. The dataset for input text types originates from user input collection. A large number of text setting statements and their corresponding input time information are collected, and their time and event information are labeled according to the time unit of their pre-set schedule, thus obtaining the sample dataset. Collecting the system time at the input moment is to learn the semantic logic of relative settings. For example, statements like "Remind me in two hours" need to be compared with the system time at the setting moment to obtain the pre-set schedule.

[0040] It should be noted that the default rules for scheduling in this embodiment also include the following four points:

[0041] 1. Unless otherwise specified by the user, the Gregorian calendar will be used by default;

[0042] 2. Default is 24-hour clock;

[0043] 3. If the input schedule unit features are incomplete, the missing time features will be filled in by default with the time features corresponding to the system time at the input time. For example, if the schedule is set to "Remind me at 10 o'clock" at 09:00 on September 16, 2021, the default schedule will be 10:00 on September 16, 2021.

[0044] 4. If the output schedule unit has expired, remind the user. For example, if the schedule is set for Qingming Festival in September, remind the user "Qingming Festival has passed, do you want to set it for the next Qingming Festival?"

[0045] Before training the model, the processing criteria of the time information in the schedule unit need to be determined to make it more suitable for artificial intelligence model processing and can carry more time information, so after obtaining the schedule unit data, the input time information can be matrix converted. Meanwhile, considering that the above-mentioned conventional schedule unit representation form is not enough to show the connection between different schedule units, such as March 28 and April 1 in the same year in different months, the human brain can judge that there is a four-day time difference between the two. Such an association, however, is difficult for a computer to extract from the literal. Therefore, the embodiment is directed to the input time information of the schedule unit and the common schedule setting requirements, and the schedule unit is subjected to a specific matrix processing to improve the correlation between the time information of each schedule unit.

[0046] Specifically, the row of the matrix is the feature of the schedule, which is set as: year, month, week, day, hour, minute, solar term, and working day; the column of the matrix is the comparison index, which is set as: year, month, week, day, hour, minute, solar term, and working day. It should be noted that for the same time, different numbers will be obtained due to different comparison indexes, such as September 16, which is the 16th day of the month, the 4th day of the week, and another number for the year.

[0047] It should be noted that the row feature element n is only compared with the previous n-1 column elements, and other comparisons are meaningless and filled with 0. For example, the feature “day” of January 16, 2021, is the 16th day of the year, the 16th day of the month, and the comparison for the day is meaningless. Among the row elements, the year, solar term, and working day three features have no meaning in comparison with other dimensions, and are set as absolute values, such as the column vector of the feature “year” of “2021” filled with 2021.

[0048] Figure 2 A schedule chart of February 2, 2021 provided by the embodiment of the application is shown as follows: Figure 2 As shown in the schedule chart, the schedule unit of February 2, 2021 at 12:30 is converted into a matrix as follows:

[0049] The first column is the absolute value column of the feature “year”, and the column vector (2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021) is obtained.

[0050] The second column is the feature “month”, the first row of the column vector is the second month of the year, and the number 2 is obtained, the last 7 rows are meaningless and are supplemented with 0, and the column vector (2, 0, 0, 0, 0, 0, 0, 0) is obtained.

[0051] The third column, the feature "week", the first row of the column vector is the 6th week for the year; the second row of the column vector is the 1st week for the month, and the following 6 rows are meaningless and supplemented as 0, obtaining the column vector (6, 1, 0, 0, 0, 0, 0, 0).

[0052] The fourth column, the feature "day", the first row of the column vector is the 33rd day for the year, obtaining the number 33; the day is the 2nd day for the month, obtaining the number 2; the day is the 2nd day for the week, obtaining the number 2, and the following 5 rows are meaningless and supplemented as 0, obtaining the column vector (33, 2, 2, 0, 0, 0, 0, 0).

[0053] The fifth column, the feature "hour", the first row of the column vector is the 804th hour for the year, obtaining the number 804; the hour is the 60th hour for the month, obtaining the number 60; the hour is the 36th hour for the week, obtaining the number 36; the hour is the 12th hour for the day, obtaining the number 12, and the following 4 rows are meaningless and supplemented as 0, obtaining the column vector (804, 60, 36, 12, 0, 0, 0, 0).

[0054] The sixth column, the feature "minute", the first row of the column vector is the 48270th minute for the year, obtaining the number 48270; the minute is the 2190th minute for the month, obtaining the number 2190; the minute is the 2190th minute for the week, obtaining the number 2190; the minute is the 750th minute for the day, obtaining the number 750; the minute is the 30th minute for the hour, obtaining the number 30; and the following 3 rows are meaningless and supplemented as 0, obtaining the column vector (48270, 2190, 2190, 750, 30, 0, 0, 0).

[0055] The seventh column, the feature "Solar Term" is an absolute value column, and the day has no solar term, obtaining the column vector (0, 0, 0, 0, 0, 0, 0, 0).

[0056] The eighth column, the feature "Workday" is an absolute value column, and the day is a workday, obtaining the column vector (1, 1, 1, 1, 1, 1, 1, 1).

[0057] In summary, the schedule unit conversion matrix is as follows:

[0058]

[0059] In the embodiment, the time information of the schedule unit is processed by using the matrix, so that the accuracy of schedule setting is improved, and the range of setting statements that can be understood by the computer is widened to be suitable for more schedule scenarios.

[0060] Figure 3 A text schedule setting model training process schematic diagram provided by an embodiment of the present application is shown in FIG. 1. Figure 3As shown, in the user input text type, the text setting sentence needs to be preprocessed and analyzed first to determine the periodic attribute label, that is, to determine whether the text setting sentence is a single setting or a periodic setting, and further, the data sample set is subjected to text preprocessing; wherein the text preprocessing mainly includes text cleaning, stop word removal and word segmentation, etc. Specifically, word segmentation tools can be used for word segmentation, and stop word tables can be imported at the same time to remove stop words.

[0061] Further, the data in the data sample set is subjected to text representation, that is, vectorization processing, specifically, the TF-IDF algorithm can be used to weight the features of the segmented words to obtain a VSM, so as to perform text vectorization processing and obtain the processed sample set.

[0062] Since the schedule setting is divided into single setting and periodic setting, the two setting logics are different and should be classified and processed, so first, the sample data set is subjected to single and periodic periodic attribute labeling, and then an adaboost model is used to simply train the two types of data to obtain a single periodic classification training model, and then a single setting input data set and a periodic setting input data set are obtained.

[0063] Since each sample data contains a text setting sentence and input time information, in this embodiment, the periodic setting sentence can be subjected to data set expansion; for example, expansion of 30 times, then 30 preset schedules are marked, and the corresponding input time information is updated synchronously, and the preset schedule time of the previous one is used as the input time system time of the next one.

[0064] The sample data of the input text type is input into a CART tree model for training, and the CART tree training rule method is summarization, which can induce classification rules from a large number of unordered schedule unit data, and is therefore suitable for the schedule setting scene, and then the output result is subjected to model evaluation to finally obtain the final schedule setting model of the input text type.

[0065] Among them, the effect of the model is evaluated by using the precision (Precision), recall (Recall) and F1 measurement value as evaluation indexes, and Table 1 shows the evaluation index base table, as shown in Table 1, wherein TP represents the number of correct examples, that is, the number of positive examples; FN is the number of examples that are incorrectly classified as negative examples, that is, the true positive examples, but the model prediction result is negative; FP is the number of examples that are incorrectly classified as positive examples, that is, the true negative examples, but the model prediction result is positive; TN represents the number of correctly classified negative examples.

[0066] Model prediction positive example Model prediction negative example True positive example TP FN True negative example FP TN

[0067] The evaluation index basis table in Table 1 is shown, wherein the precision (Precision) calculation formula is as follows formula (1) :

[0068]

[0069] Wherein, Precision is the ratio of the number of correct judgments in the samples judged as positive examples by the model.

[0070] The recall rate (Recall) calculation formula is as follows formula (2) :

[0071]

[0072] Wherein, in the entire positive class sample, Recall represents the ratio of the samples predicted correctly.

[0073] In the process of evaluating the model, only using precision or recall to evaluate the model cannot comprehensively evaluate the advantages and disadvantages of the model, therefore, the precision and recall can be combined to obtain F1 score as the actual score criterion of the model, in the multi-classification problem, F1 is the harmonic mean of precision and recall, and the F1 measurement value calculation formula is as follows formula (3) :

[0074]

[0075] The output results of precision (Precision), recall (Recall) and F1 measurement value all reach the preset value, and the model evaluation passes (Y), and the final text schedule setting model is obtained, if the model evaluation does not pass (N), the text schedule setting model is returned to the previous level for continuous training.

[0076] In an optional implementation, the step S110 further includes: reading the voice simulation signal data by using a voice grabbing tool package; converting the voice simulation signal data into a voice spectrum image; marking the voice spectrum image in multiple dimensions to obtain sample data of the voice type; inputting the sample data of the voice type into a long short-term memory network model for training to obtain a voice schedule setting model as the schedule setting model corresponding to the voice type.

[0077] Figure 4 The voice schedule setting model training process schematic diagram provided by the embodiment of the application is shown, as shown in Figure 4As shown, the relevant voice information of the client is extracted for the sample data of the voice type, and since the voice information contains the literal meaning and the emotion of the person, in order to more accurately read the intention of the user, the voice data authorized by the user can be read through a voice grabbing tool to obtain voice analog signal data; in this embodiment, the voice analog signal data can be read through a Charles voice grabbing tool package.

[0078] The voice analog signal data is subjected to Fourier transform to be converted into a voice spectrum, and then the voice spectrum is sampled to be converted into a voice spectrum image, that is, the digital signal corresponding to the voice analog signal obtained after sampling is quantized into a vector. The voice spectrum image and the meaning of the actual voice expression are compared and marked through multiple dimensions, wherein the multiple dimensions at least include: a time dimension, an event dimension and a tone dimension; wherein the marking method of the tone dimension is a classification marking, and common tones include a statement tone, a questioning tone, a counter-question tone and the like. The voice spectrum image after marking is obtained. Due to the length of the voice information, a 0 operation can be performed on the shorter voice vector to ensure the consistency of the feature dimension, and thus the sample data of the voice type is obtained. The sample data of the voice type is input into a Long Short-Term Memory (LSTM) model to be trained to obtain a voice schedule setting model, and the same model evaluation method as the input text type data is used to evaluate the final voice schedule setting model to analyze the time, event and other information contained in the voice.

[0079] In an optional implementation, the step S110 further includes: acquiring image data, pre-processing the image data through a first neural network model to obtain effective image data; marking the effective image data to obtain time information and event information, and constructing sample data of an image type by using the time information and the event information; inputting the sample data of the image type into a second neural network model to be trained to obtain an image schedule setting model as a schedule setting model corresponding to the image type.

[0080] The schedule setting model of the image type is mainly used to acquire the actual daily life rule of the user, thereby assisting the schedule setting model to make the prediction result more accurate. In actual application, the sample data of the image type is derived from the authorized album of the user and software images and the like, and such images have the fluctuation in a small time range and the stability in a long time sequence; for example, the morning work image and the evening exercise image both contain time and event information.

[0081] Figure 5 An image schedule setting model training process schematic diagram provided by the embodiment of the application is shown in FIG. 4. Figure 5As shown, the sample data set used by the schedule setting model corresponding to the training image type is the ImageNet data set, which is a large visual database for visual object recognition software research. The ImageNet data set is updated annually and has a large amount of data. The categories of images mainly include life, work, people, scenery, and others. In this step, life and work are defined as valid information, and others are invalid information. At the same time, in order to remove irrelevant image data, a first neural network model (for example Figure 5 the first GoogleNet neural network on the left side in the figure) can be used to preprocess the image data to obtain valid image data. Specifically, the image data is input into a filter through image classification processing by the first neural network model. The filter first removes irrelevant images and only retains valid images to form a valid image data set.

[0082] Each valid image data in the valid image data set is labeled according to time information and event information. Specifically, the time information label is divided into 24 hours, and the time from 6:00 to 24:00 is accurate to half an hour. This period of time is the time of frequent human activities, and the number of samples is 500 per half hour. The time from 24:00 at night to 6:00 in the morning is a low-frequency activity time, and the time is accurate to one hour, and the number of samples is 1000. The total number of samples is 6000. Then, the sample set is input into a second neural network model (i.e. Figure 5 the second GoogleNet neural network in the figure) to obtain an image schedule setting model through repeated training. After model evaluation, the final image schedule setting model is obtained, thereby obtaining a more objective actual time standard that conforms to the user's habits to assist in determining the schedule setting. The first neural network model and the second neural network model can be a GoogleNet convolutional network model.

[0083] Step S120: The first output results of the schedule setting models corresponding to multiple types are fused through a preset fusion model to obtain a fused schedule setting model.

[0084] Since the results predicted by the schedule setting models of multiple types can be different, a fusion needs to be performed, that is, the first output results are normalized into valid information. Specifically, different information is fused through a fusion algorithm, and then the contents are filtered according to the business rules and filtering rules (i.e., the default rules for schedule setting) required by the client. Through model evaluation, continuous correction is performed to obtain a final fused schedule setting model.

[0085] In an optional implementation, the first output result includes time information; and step S120 further includes: classifying the first output results of the schedule setting models corresponding to the multiple types according to a time dimension to obtain first output results corresponding to same time information; inputting the first output results corresponding to the same time information into a preset fusion model, obtaining corresponding log training results in the preset fusion model according to weights corresponding to the multiple types, adjusting model parameters of the preset fusion model according to a loss function and the log training results until a preset training end condition is met, and obtaining a fusion schedule setting model.

[0086] Specifically, since the time information and the event information in the first output results obtained by the schedule setting models corresponding to the multiple types can be inconsistent, the time information and the event information need to be marked again, the first output results of the same time information are marked as a class, and the first output results of the multiple types of marked schedule setting models are input into the preset fusion model as an input layer of the preset fusion model to obtain the event information corresponding to the time information through training.

[0087] In this step, the preset fusion model can be a fusion Wide&Deep model, which is divided into a Wide model and a Deep model. The Wide model is a linear model, and its parameters need to be set in advance. In this application, the preset parameters include weights corresponding to the multiple types; and the three first output results of the schedule setting models of the three types are input into the Wide model as input sources. Since the input text is input by a user, the user has a higher schedule setting intention, and the image and voice inputs are auxiliary and correction functions. Therefore, the weights of the three types of input text, voice and image can be set as (0.8, 0.1, 0.1).

[0088] The Wide model has a strong "memory ability", and the Deep model is a feedforward neural network that increases the generalization ability of the model to obtain more stable and smooth recommended information. In the preset fusion model, the outputs of the Wide and Deep parts are combined by weighting, and the parameters are continuously corrected by a loss function to obtain the final fusion schedule setting model.

[0089] Step S130: obtaining multiple types of to-be-analyzed data from a user end, processing the to-be-analyzed data of the corresponding types by using the schedule setting models corresponding to the multiple types to obtain second output results of the multiple types.

[0090] In this step, multiple types of to-be-analyzed data are obtained from the user end, and the sources of the to-be-analyzed data include user input text data (i.e., text setting sentences), voice data and image data obtained by the system; the to-be-analyzed data of the corresponding type is processed by using the corresponding schedule setting model of the multiple types, and second output results of the multiple types are obtained.

[0091] In an optional implementation, step S130 further includes: obtaining voice simulation signal data generated within a preset time period from the user end as voice type to-be-analyzed data, and obtaining image data generated within the preset time period from the user end as image type to-be-analyzed data; determining whether there is a user input text setting sentence within the current time range; if yes, obtaining the text setting sentence from the user end as input text type to-be-analyzed data.

[0092] Step S140: processing the second output results of the multiple types by using the fusion schedule setting model, and outputting a recommended schedule.

[0093] In an optional implementation, step S140 further includes: determining whether the second output results of the multiple types include the second output result of the input text type; if the second output result of the input text type is included, filtering the second output results of the multiple types according to time information in the second output result of the input text type, inputting the filtered second output results into the fusion schedule setting model for processing, and outputting a recommended schedule; if the second output result of the input text type is not included, filtering the second output results of the multiple types corresponding to the same time information, inputting the second output results corresponding to the same time information into the fusion schedule setting model for processing, and outputting a recommended schedule.

[0094] If there is a user input text setting sentence within the current time range, it is necessary to determine the periodicity attribute of the input text setting sentence, i.e., whether the sentence is a periodic setting or a one-time setting, Figure 6 A recommended schedule generation flowchart when there is a user input text setting sentence and the setting is one-time is shown as follows: Figure 6 As shown in the figure, after the user inputs a text setting sentence, it is determined by the adaboost model whether the setting is one-time or periodic, if the setting is one-time, the sentence is input into the final text schedule setting model, and voice data and image data of the current time range are obtained and input into the corresponding final voice schedule setting model and final image schedule setting model, the second output results of the voice schedule setting model and the image schedule setting model that are inconsistent with the time information in the second output result of the text schedule setting model are filtered out, and then input into the fusion schedule setting model, and the final recommended schedule is obtained through fusion determination.

[0095] Figure 7 The recommended schedule generation flow chart when there is a user input text setting sentence and for a period is shown as Figure 7 As shown, if the user input text setting sentence is judged to be a period setting by the adaboost model, a period button prompt can be popped up on the user terminal, and after the user confirms, the setting sentence is input to the final text schedule setting model, and the voice data and image data of the current time range are input to the corresponding final voice schedule setting model and final image schedule setting model, and the final recommended schedule is obtained through the final fusion schedule setting model.

[0096] It should be noted that for the period setting, after the recommended schedule reminding unit executes, the time information of the recommended schedule is assigned to the next input time system time, and the input of the text setting sentence and the schedule setting of the next time are started, and the cycle is executed until the user closes the period button. For example, if "remind me once every three days" is input at 9:00 on September 8, "9:00 on September 8" is the first input time system time, and the output setting is "9:00 on September 11". After the schedule reminder is completed on September 11, "9:00 on September 11" is automatically assigned to the next input time system setting time, and the input sentence "remind me once every three days" is executed again to obtain the next schedule output setting "9:00 on September 14". This cycle continues until the user closes the period button. In this way, without performing a large number of repetitive periodic calculations within a limited time, the computing resources can be saved.

[0097] Figure 8 The recommended schedule generation flow chart when there is no user input text setting sentence is shown as Figure 8 As shown, if there is no user input text setting sentence in the current time range, voice data and image data are obtained once every half hour according to a preset time period, and are input to the corresponding final voice schedule setting model and final image schedule setting model. The consistency of the second output result time information of the final voice schedule setting model and the final image schedule setting model is judged. If the recommended schedule time information of the second output result of the voice schedule setting model and the image schedule setting model is inconsistent (N), the system does not recommend; if the time information of the two is consistent (Y), after the fusion schedule setting model judges, the recommended schedule is output. Specifically, according to the time corresponding to the time, the schedule reminding unit is started, and the output result is reminded in the form of a recommendation. For example, the voice schedule setting model recommends buying takeout at 8 o'clock, and the image schedule setting model recommends running at 8 o'clock. After the fusion schedule setting model judges that it is better to run at 8 o'clock, the output recommended schedule is "It's 8 o'clock, let's start running".

[0098] The method of the embodiment supports multiple types of schedule setting inputs, including text input, voice input, image input and the like; user habits are analyzed from voice sample data and image sample data, and combined with text sample data to form multiple types of schedule setting models, and different algorithms are used for training according to different types of data input modes, so that the prediction result of each type of schedule setting model is more accurate; the schedule unit is processed by matrixing, and the time dimension is displayed in a better data form, the matrix shows the association between different time dimensions, which helps the subsequent fusion schedule setting model to more accurately extract the setting rules; the results of processing multiple types of sample data are input into the fusion schedule setting model, and the data set is classified and labeled again according to the same time dimension, which increases the accuracy of the fusion schedule setting model recommendation result; for the case that the data to be analyzed is a periodic statement, a secondary setting period setting mode is adopted after a one-time reminder is completed, which saves system resources, increases the accuracy of schedule setting, and saves system resources; the Wide&Deep model is used to improve the schedule setting efficiency and the coverage range of the schedule setting scene; the method optimizes the schedule data preprocessing mode according to the characteristics of schedule setting, and matrixes a single schedule element, which is beneficial to the output of the schedule setting model, improves the model accuracy, and further improves the accuracy of the schedule recommendation.

[0099] Figure 9 The structure of the embodiment of the schedule reminding device based on data extraction is shown. Figure 9 As shown in the figure, the device comprises a sub-model training module 901, a fusion model training module 902, a sub-model analysis module 903 and a fusion recommendation module 904.

[0100] The sub-model training module 901 is used to obtain multiple types of sample data, and for each type, the sample data of the type is used for model training to obtain the schedule setting model corresponding to the type.

[0101] In an optional manner, the multiple types include an input text type; the sub-model training module 901 is further used to obtain text setting statements and input time information corresponding to the text setting statements; the text setting statements are preprocessed and analyzed to determine a periodic attribute label; the input time information is matrix converted; the sample data of the input text type is obtained according to the preprocessed text input statement, the periodic attribute label and the matrix converted input time information; the sample data of the input text type is input into a decision tree training model for training to obtain a text schedule setting model as the schedule setting model corresponding to the input text type.

[0102] In an optional manner, the multiple types include: a voice type; the sub-model training module 901 is further configured to: read voice simulation signal data through a voice grabbing toolkit; convert the voice simulation signal data into a voice spectrum image; label the voice spectrum image in multiple dimensions to obtain sample data of the voice type; input the sample data of the voice type into a long short-term memory network model for training to obtain a voice schedule setting model as a schedule setting model corresponding to the voice type; and the multiple dimensions at least include: a time dimension, an event dimension, and a tone dimension.

[0103] In an optional manner, the multiple types include: an image type; the sub-model training module 901 is further configured to: obtain image data, pre-process the image data through a first neural network model to obtain effective image data; label the effective image data to obtain time information and event information, and construct sample data of the image type by using the time information and the event information; and input the sample data of the image type into a second neural network model for training to obtain an image schedule setting model as a schedule setting model corresponding to the image type.

[0104] The fusion model training module 902 is configured to perform fusion training on first output results of the schedule setting models corresponding to the multiple types through a preset fusion model to obtain a fusion schedule setting model.

[0105] In an optional manner, the first output results include: time information; the fusion model training module 902 is further configured to: classify the first output results of the schedule setting models corresponding to the multiple types according to a time dimension to obtain first output results corresponding to the same time information; input the first output results corresponding to the same time information into the preset fusion model, train in the preset fusion model according to weights corresponding to the multiple types to obtain corresponding log training results, adjust model parameters of the preset fusion model according to a loss function and the log training results, until a preset training end condition is met, and obtain the fusion schedule setting model.

[0106] The sub-model analysis module 903 is configured to obtain, from a user end, to-be-analyzed data of the multiple types, process the to-be-analyzed data of the corresponding types by using the schedule setting models corresponding to the multiple types, and obtain second output results of the multiple types.

[0107] In an optional manner, the sub-model analysis module 903 is further configured to: obtain, from the user end, voice simulation signal data generated within a preset time period as to-be-analyzed data of the voice type, and obtain, from the user end, image data generated within the preset time period as to-be-analyzed data of the image type; determine whether there is a text setting statement input by the user within a current time range; if yes, obtain the text setting statement from the user end as to-be-analyzed data of an input text type.

[0108] The fusion recommendation module 904 is configured to process the second output results of the multiple types by using a fusion schedule setting model, and output a recommended schedule.

[0109] In an optional manner, the fusion recommendation module 904 is further configured to: determine whether the second output results of the multiple types include a second output result of an input text type; if the second output result of the input text type is included, filter the second output results of the multiple types according to time information in the second output result of the input text type, input the filtered second output results to the fusion schedule setting model for processing, and output the recommended schedule; and if the second output result of the input text type is not included, filter second output results corresponding to the same time information from the second output results of the multiple types, input the second output results corresponding to the same time information to the fusion schedule setting model for processing, and output the recommended schedule.

[0110] The device of the embodiment supports multiple types of schedule setting inputs, including text input, voice input, image input and the like; user habits are analyzed from voice sample data and image sample data, and combined with text sample data to form multiple types of schedule setting models, and different algorithms are used for training according to different types of data input modes, so that the prediction result of each type of schedule setting model is more accurate; the schedule unit is processed by matrixing, and the time dimension is displayed in a better data form, the matrix shows the association between different time dimensions, which helps the subsequent fusion schedule setting model to more accurately extract setting rules; the results of processing multiple types of sample data are input to the fusion schedule setting model, and the data set is classified and labeled again according to the same time dimension, which increases the accuracy of the recommended result of the fusion schedule setting model; for the case that the data to be analyzed is a periodic statement, a periodical setting mode of secondary setting after one-time reminding is adopted, which saves system resources, increases the accuracy of schedule setting, and saves system resources.

[0111] The embodiment of the present application provides a non-volatile computer storage medium, which stores at least one executable instruction, and the computer executable instruction can execute a data extraction-based schedule reminding method in any method embodiment.

[0112] The executable instruction can be specifically used to make the processor perform the following operations:

[0113] Obtain multiple types of sample data, for each type, use the sample data of the type to train a model, and obtain a schedule setting model corresponding to the type;

[0114] Fuse the first output results of the multiple types of schedule setting models by using a preset fusion model to obtain a fusion schedule setting model.

[0115] Obtaining a plurality of types of to-be-analyzed data from a user end, processing the to-be-analyzed data of each type by using a schedule setting model corresponding to the type to obtain a second output result of each type;

[0116] Processing the second output result of each type by using a fusion schedule setting model to output a recommended schedule.

[0117] Figure 10 A structural schematic diagram of an embodiment of the computing device of the present application is shown, and the embodiment of the present application does not limit the specific implementation of the computing device.

[0118] As shown in Figure 10 , the computing device can include:

[0119] a processor, a communications interface, a memory, and a communications bus.

[0120] The processor, the communications interface, and the memory complete communication with each other through the communications bus. The communications interface is configured to communicate with network elements such as clients or other servers. The processor is configured to execute programs, and specifically can execute the related steps in the above-described one embodiment of the schedule reminding method based on data extraction.

[0121] Specifically, the program can include program code including computer operation instructions.

[0122] The processor can be a central processing unit (CPU) or an application specific integrated circuit (ASIC) or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the server can be processors of the same type, such as one or more CPUs; or can be processors of different types, such as one or more CPUs and one or more ASICs.

[0123] The memory is configured to store programs. The memory can include a high-speed RAM memory and can also include a non-volatile memory such as at least one disk memory.

[0124] The program can specifically be configured to cause the processor to perform the following operations:

[0125] Obtaining a plurality of types of sample data, for each type, performing model training by using the sample data of the type to obtain a schedule setting model corresponding to the type;

[0126] The first output results of the plurality of type corresponding schedule setting models are fused by a preset fusion model to obtain a fused schedule setting model;

[0127] The plurality of types of to-be-analyzed data are obtained from the user end, and the type corresponding schedule setting model is used to process the to-be-analyzed data of the corresponding type to obtain a second output result of the plurality of types;

[0128] The second output result of the plurality of types is processed by the fused schedule setting model to output a recommended schedule.

[0129] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other apparatus. Various general purpose systems can be used with programs in accordance with the teachings herein, or it can prove convenient to construct more specialized apparatus to perform the required method steps. The required structure for a variety of these systems will be apparent from the description above. In addition, the present embodiments are not described with reference to any particular programming language. It will be appreciated that a variety of programming languages can be used to implement the teachings of the application as described herein, and any references below to specific languages are provided for disclosure of enablement only.

[0130] In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the application can be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been described in detail in order to not obscure the understanding of this description.

[0131] Similarly, it is to be understood that the bricr-inventor's detailed description of exemplary embodiments of the application, for purposes of brevity, generally, describes various features only in their most relevant relation to that embodiment. However, it is not intended that all features be necessarily required in every case, as the various features can be used in a wide variety of settings and applications, even different application than those explicitly disclosed herein. Furthermore, it should be noted that the bricr-inventor's exemplary embodiments of the application can comprise various combinations and / or sub-combinations of the various features and / or functions described herein, and other applications and uses will suggest themselves to those of ordinary skill in the art after having the benefit of this description. Therefore, the bricr-inventor's application should not be construed as limited to the particular embodiments disclosed herein, but instead, should be construed in breadth, consistent with the appended claims, along with the full span of equivalents thereunder.

[0132] Those skilled in the art will appreciate that the modules in the apparatuses in the embodiments can be adapted and placed in one or more apparatuses other than the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and further can be divided into multiple sub-modules or sub-units or sub-components. Any combination of all the features disclosed in the present specification (including the accompanying claims, abstract and drawings), and any method or apparatus so disclosed, can be taken in any combination, except that at least some of such features and / or processes or units are mutually exclusive of each other. Each feature disclosed in the present specification (including the accompanying claims, abstract and drawings) can be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

[0133] Furthermore, those skilled in the art will appreciate that different embodiments of the application have different features and that not all embodiments of the application have the same features. It is therefore anticipated that each of the claims hereon includes at least one feature which is not shared by all embodiments of the application. For example, in the claims below, any of the claims can be read to include at least one of the features of the other claims.

[0134] Embodiments of various components of the present application can be implemented in hardware, software, or a combination thereof, and can be implemented with one or more computer programs or code that comprise computer readable program instructions that are executable by one or more processors. Embodiments of the present application can be implemented as a computer program product, which can include a computer readable medium having stored computer readable program instructions that can direct a processor to carry out the steps of the present application. Embodiments of the present application can also be implemented as a computer program product comprising a computer readable medium having stored computer readable program instructions that, when executed by one or more processors, carry out the steps of the present application. The computer readable program instructions can be stored on a computer readable medium, which can be any medium (or combination of media) that participates in providing computer readable program instructions to a processor for execution. By way of example, such computer readable media can include RAM (random access memory), ROM (read only memory), EPROM (erasable programmable read only memory), EEPROM (electrically erasable programmable read only memory), flash memory cards, DVD, digital tape, computer memory, computer readable storage media, etc. The computer readable program instructions can be stored on the computer readable medium using any appropriate medium, such as computer readable storage media, computer readable signals, etc.

[0135] It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word 'comprising' does not exclude the presence of elements or steps other than those listed in a claim. The word 'a' or 'an' preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In a unitary claim, several devices, apparatuses or means can be listed, comprising means for performing a certain function. The use of the term'means' in a claim is intended to cover one or more elements comprising the means for performing the function. The use of the terms first, second and third, etc. does not imply any ordering, but rather are used for naming purposes only. The steps of any of the methods disclosed herein do not have to be performed in the exact order disclosed, unless a step is explicitly described as following or preceding another step.

Claims

1. A data-extraction-based schedule reminding method, characterized by, The method comprises: acquiring a plurality of types of sample data, for each type, using the sample data of the type to perform model training to obtain a schedule setting model corresponding to the type; performing fusion training on first output results of the schedule setting models of the plurality of types by using a preset fusion model to obtain a fusion schedule setting model; the first output results comprise time information; acquiring a plurality of types of to-be-analyzed data from a user end, using the schedule setting models of the plurality of types to process the to-be-analyzed data of the corresponding types to obtain second output results of the plurality of types; using the fusion schedule setting model to process the second output results of the plurality of types to output a recommended schedule; wherein the acquiring a plurality of types of sample data, for each type, using the sample data of the type to perform model training to obtain a schedule setting model corresponding to the type further comprises: when the plurality of types include an input text type, acquiring a text setting statement and input time information corresponding to the text setting statement; preprocessing and analyzing the text setting statement to determine a periodic attribute label; performing matrix conversion on the input time information; obtaining sample data of the input text type according to the preprocessed text input statement, the periodic attribute label and the matrix-converted input time information; inputting the sample data of the input text type into a decision tree training model to perform training to obtain a text schedule setting model as the schedule setting model corresponding to the input text type; when the plurality of types include a voice type, reading voice simulation signal data by using a voice grabbing toolkit; converting the voice simulation signal data into a voice spectrum image; marking the voice spectrum image in a plurality of dimensions to obtain sample data of the voice type; inputting the sample data of the voice type into a long short-term memory network model to perform training to obtain a voice schedule setting model as the schedule setting model corresponding to the voice type; wherein the plurality of dimensions at least include a time dimension, an event dimension and a tone dimension; when the plurality of types include an image type, acquiring image data, preprocessing the image data by using a first neural network model to obtain effective image data; marking the effective image data to obtain time information and event information, and using the time information and the event information to construct sample data of the image type; inputting the sample data of the image type into a second neural network model to perform training to obtain an image schedule setting model as the schedule setting model corresponding to the image type; the performing fusion training on first output results of the schedule setting models of the plurality of types by using a preset fusion model to obtain a fusion schedule setting model further comprises: classifying the first output results of the schedule setting models of the plurality of types according to a time dimension to obtain first output results corresponding to the same time information; The first output result corresponding to the same time information is input into the preset fusion model, a corresponding log training result is obtained by training the preset fusion model according to weights corresponding to multiple types, model parameters of the preset fusion model are adjusted according to a loss function and the log training result, until a preset training end condition is met, and a fusion schedule setting model is obtained; The processing of the second output result of multiple types by using the fusion schedule setting model to output a recommended schedule further includes: determining whether the second output result of the input text type is included in the second output result of multiple types; if the second output result of the input text type is included, the second output result of multiple types is filtered according to time information in the second output result of the input text type, the filtered second output result is input into the fusion schedule setting model for processing, and a recommended schedule is output; if the second output result of the input text type is not included, the second output result corresponding to the same time information is filtered from the second output result of multiple types, and the second output result corresponding to the same time information is input into the fusion schedule setting model for processing, and a recommended schedule is output.

2. The method of claim 1, wherein, The obtaining of the multiple types of to-be-analyzed data from the user end further includes: obtaining voice simulation signal data generated within a preset time period from the user end as voice type to-be-analyzed data, and obtaining image data generated within a preset time period from the user end as image type to-be-analyzed data; determining whether there is a user input text setting statement within the current time range; if yes, the text setting statement is obtained from the user end as input text type to-be-analyzed data.

3. A data-extraction-based schedule reminding apparatus characterized by comprising: including: a sub-model training module for obtaining sample data of multiple types, for each type, using sample data of the type for model training to obtain a schedule setting model corresponding to the type; a fusion model training module for performing fusion training on first output results of schedule setting models corresponding to multiple types through a preset fusion model to obtain a fusion schedule setting model; a sub-model analysis module for obtaining to-be-analyzed data of multiple types from the user end, processing the to-be-analyzed data of the corresponding type by using the schedule setting model corresponding to the type to obtain second output results of multiple types; a fusion recommendation module for processing the second output results of multiple types by using the fusion schedule setting model to output a recommended schedule; The sub-model training module is further used for: when multiple types include an input text type, obtaining a text setting statement and input time information corresponding to the text setting statement; preprocessing and analyzing the text setting statement to determine a periodic attribute label; performing matrix conversion on the input time information; obtaining sample data of the input text type according to the preprocessed text input statement, the periodic attribute label and the matrix converted input time information; inputting the sample data of the input text type into a decision tree training model for training to obtain a text schedule setting model as the schedule setting model corresponding to the input text type; When the multiple types include a voice type, reading voice analog signal data through a voice scraping toolkit; converting the voice analog signal data into a voice spectrum image; labeling the voice spectrum image in multiple dimensions to obtain sample data of the voice type; inputting the sample data of the voice type into a long short-term memory network model for training to obtain a voice schedule setting model as a schedule setting model corresponding to the voice type; wherein the multiple dimensions at least include a time dimension, an event dimension, and a tone dimension; When the multiple types include an image type, obtaining image data, pre-processing the image data through a first neural network model to obtain effective image data; labeling the effective image data to obtain time information and event information, and using the time information and the event information to construct sample data of the image type; inputting the sample data of the image type into a second neural network model for training to obtain an image schedule setting model as a schedule setting model corresponding to the image type; The fusion model training module is further used for: classifying first output results of the schedule setting models corresponding to the multiple types according to a time dimension to obtain first output results corresponding to the same time information; inputting the first output results corresponding to the same time information into the preset fusion model, training in the preset fusion model according to weights corresponding to the multiple types to obtain corresponding log training results, adjusting model parameters of the preset fusion model according to a loss function and the log training results until a preset training end condition is met, and obtaining a fusion schedule setting model; The fusion recommendation module is further used for: judging whether the second output results of the multiple types include a second output result of an input text type; if the second output result of the input text type is included, filtering the second output results of the multiple types according to time information in the second output result of the input text type, inputting the filtered second output results into the fusion schedule setting model for processing, and outputting a recommended schedule; if the second output result of the input text type is not included, filtering second output results corresponding to the same time information from the second output results of the multiple types, inputting the second output results corresponding to the same time information into the fusion schedule setting model for processing, and outputting a recommended schedule.

4. A computing device, comprising: comprise: a processor, a memory, a communication interface, and a communication bus, the processor, the memory, and the communication interface completing communication with each other through the communication bus; the memory is used to store at least one executable instruction, and the executable instruction causes the processor to perform operations corresponding to the schedule reminding method based on data extraction according to any one of claims 1 or 2.

5. A computer storage medium, characterized in that The storage medium stores at least one executable instruction, and the executable instruction causes the processor to perform operations corresponding to the schedule reminding method based on data extraction according to any one of claims 1 or 2.

Images