Data screening method, device and equipment and storage medium thereof

Abstract

The embodiment of the invention belongs to the technical field of big data, is applied to a many-to-many data retrieval scene of a multi-screening condition and a multi-source data end, and relates to a data screening method, device and equipment and a storage medium thereof.The method comprises the steps that numeralized data participating in target data screening in the multi-source data end is obtained; carrying out data tuple ensemble arrangement on the numerical data to obtain an ensemble arrangement result taking the data tuple as an element; identifying a screening condition; and screening out a final data tuple corresponding to each group of data screening conditions through the screening conditions to finish the data screening. Before data screening, the data in the search domain is subjected to data tuple ensemble processing, screening conditions are also subjected to preposed sorting, and candidate data tuples are introduced into monotonic stack storage, so that when data screening is actually carried out, the data screening efficiency is improved, and the time consumption is saved.

Description

Technical Field

[0001] The present application relates to the field of big data technology, and is applied to many-to-many data retrieval scenarios with multiple screening conditions and multiple source data terminals, and in particular to a data screening method, device, equipment and storage medium thereof. Background Art

[0002] When users perform data queries, they directly search for data when the amount of original data is large. The time required varies depending on the configuration of the search device, the search method, and the search content. When users perform queries, if they search directly in the data source, when the amount of original data is large, directly judging the target result will result in a slow search or a pseudo-death pause due to the large amount of data that cannot be responded to.

[0003] Although there have been some improvements to the above phenomenon, there is still a lack of good improvement methods for many-to-many screening searches in big data application scenarios. For example, in big data processing scenarios where multiple search engines conduct comprehensive searches on multiple data storage terminals, or in scenarios where multiple search conditions are used to search multiple data terminals separately, the existing technology takes a long time and has low screening efficiency in many-to-many screening scenarios. Summary of the invention

[0004] The purpose of the embodiments of the present application is to propose a data screening method, device, equipment and storage medium thereof to solve the problem that the prior art takes a long time and has low screening efficiency in many-to-many screening scenarios.

[0005] In order to solve the above technical problems, the present application embodiment provides a data screening method, which adopts the following technical solutions:

[0006] A data screening method comprises the following steps:

[0007] Obtain numerical data from multiple source data terminals that are respectively involved in the screening of target data;

[0008] According to the difference of multi-source data terminals, the numerical data is aggregated into data tuples to obtain an aggregated aggregation result with data tuples as elements;

[0009] Identifying multiple groups of data screening conditions corresponding to the multi-source data ends, and comprehensive screening conditions corresponding to the multi-source data ends, wherein each group of data screening conditions includes numerical screening data of all data ends in the multi-source data ends in the current group;

[0010] Filtering all candidate data tuples corresponding to each set of data screening conditions from the aggregated sorting results by using the multiple sets of data screening conditions and the comprehensive screening conditions;

[0011] According to the preset optimal screening strategy, the final data tuples corresponding to each set of data screening conditions are screened out from all the candidate data tuples corresponding to each set of data screening conditions, thereby completing this data screening.

[0012] Furthermore, the step of performing data tuple collection and sorting on the digitized data according to the differences of the multi-source data terminals to obtain a collection sorting result with the data tuple as an element specifically includes:

[0013] According to the differences of the multi-source data terminals, the numerical data are sorted into arrays to obtain numerical data arrays corresponding to each data terminal;

[0014] According to the array subscript, the numerated data with the same array subscript is obtained from the numerated data array corresponding to each data end, and a sorting result with a set length of N and a data tuple length of M is generated, wherein M represents the number of data ends of the multi-source data ends, and N represents the maximum number of numerated data participating in the target data screening among the M data ends, and N and M are positive integers.

[0015] Furthermore, the step of obtaining the digitized data with the same array subscript from the digitized data arrays corresponding to each data end according to the array subscript, and generating a sorting result with a set length of N and a data tuple length of M specifically includes:

[0016] According to the preset array length calculation method, the length of the numerical data array corresponding to each data end is calculated;

[0017] By comparing two by two, selecting the longest numerical data array from all numerical data arrays corresponding to the multi-source data terminals;

[0018] The length of the longest numerical data array is set as the set length N;

[0019] Sequentially obtain the digitized data with the same array subscript from all digitized data arrays corresponding to the multi-source data end, add the digitized data with the same array subscript to the same data tuple, and generate N data tuples with a length of M;

[0020] The N data tuples with a generated length of M are subjected to set processing to obtain a sorting result with a set length of N and a data tuple length of M, wherein N represents the length of the longest digitized data array, and M represents the number of all digitized data arrays corresponding to the multi-source data terminals.

[0021] Further, the step of obtaining the digitized data with the same array subscript from all digitized data arrays corresponding to the multi-source data end in turn, adding the digitized data with the same array subscript to the same data tuple, and generating N data tuples with a length of M specifically includes:

[0022] Step 501, using a preset array element extraction method to extract the numerical data corresponding to the current array subscript of the current numerical data array to obtain an extraction result;

[0023] Step 502, if the extraction result is not a null value, then the numerical data is obtained, and the numerical data is added to the data tuple corresponding to the current array index;

[0024] Step 503, if the extraction result is a null value, then add a corresponding null value at the reserved position of the data tuple corresponding to the current array index;

[0025] Step 504, taking 0 to N in turn as the extraction control parameters of the preset array element extraction method, repeating steps 501 to 503 to generate N data tuples of length M.

[0026] Furthermore, after executing the step of identifying the multiple sets of data screening conditions corresponding to the multiple source data ends, the method further includes:

[0027] Selecting one data end from the multiple source data ends as the target data end;

[0028] Identify the numerical screening data corresponding to the target data end from each set of data screening conditions;

[0029] According to the size relationship of all the numerical screening data corresponding to the target data end, the multiple groups of data screening conditions are sorted to obtain the multiple groups of data screening conditions after sorting.

[0030] Furthermore, the step of sorting the plurality of sets of data screening conditions according to the size relationship of all numerical screening data corresponding to the target data end to obtain the plurality of sets of data screening conditions after sorting specifically includes:

[0031] Performing descending processing on all numerical screening data corresponding to the target data end to obtain a descending sequence;

[0032] Identify each set of data screening conditions corresponding to all numerical screening data in the descending sequence, and obtain a recognition result;

[0033] The multiple groups of data screening conditions are sorted according to the recognition results to obtain the multiple groups of data screening conditions after the sorting process.

[0034] Furthermore, after executing the step of selecting all candidate data tuples corresponding to each set of data screening conditions from the aggregated sorting results by using the multiple sets of data screening conditions and the comprehensive screening conditions, the method further includes:

[0035] Step 801, sequentially obtaining all candidate data tuples corresponding to each set of data screening conditions;

[0036] Step 802, based on the comprehensive screening condition, select a better candidate data tuple from the currently acquired candidate data tuple and the previously acquired candidate data tuple to obtain a screening result;

[0037] Step 803, according to the screening result, cache the better candidate data tuples corresponding to each set of data screening conditions into a target data cache stack, wherein each set of data screening conditions corresponds to a corresponding data cache stack;

[0038] Step 804, repeatedly executing steps 801 to 803, caching the optimal candidate data tuples corresponding to each set of data screening conditions into the target data cache stack;

[0039] The step of selecting the final data tuples corresponding to each set of data screening conditions from all the candidate data tuples corresponding to each set of data screening conditions according to the preset optimal screening strategy to complete the data screening specifically includes:

[0040] Acquire the optimal candidate data tuples corresponding to the plurality of sets of data screening conditions respectively from the target data cache stack;

[0041] The optimal candidate data tuples corresponding to the multiple sets of data screening conditions are taken as the corresponding final data tuples.

[0042] In order to solve the above technical problems, the embodiment of the present application also provides a data screening device, which adopts the following technical solution:

[0043] A data screening device, comprising:

[0044] A data acquisition module for participating in screening is used to obtain numerical data participating in target data screening from multiple source data terminals;

[0045] A collection arrangement module, used to collect and arrange the numerical data into data tuples according to the differences of multiple source data terminals, and obtain a collection arrangement result with data tuples as elements;

[0046] A screening condition identification module, used to identify multiple groups of data screening conditions corresponding to the multi-source data ends, and comprehensive screening conditions corresponding to the multi-source data ends, wherein each group of data screening conditions includes numerical screening data of all data ends in the multi-source data ends in the current group;

[0047] A candidate data tuple screening module, used to screen out all candidate data tuples corresponding to each set of data screening conditions from the aggregated sorting results through the multiple sets of data screening conditions and the comprehensive screening conditions;

[0048] The final data tuple screening module is used to screen out the final data tuples corresponding to each set of data screening conditions from all candidate data tuples corresponding to each set of data screening conditions according to a preset optimal screening strategy to complete this data screening.

[0049] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0050] A computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the above-mentioned data screening method when executing the computer-readable instructions.

[0051] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0052] A computer-readable storage medium stores computer-readable instructions, which implement the steps of the data screening method described above when executed by a processor.

[0053] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0054] The data screening method described in the embodiment of the present application is to obtain the numerical data respectively involved in the target data screening from the multi-source data end; perform data tuple collection and sorting on the numerical data to obtain the collection sorting result with data tuples as elements; identify the screening conditions; screen out all the candidate data tuples corresponding to each set of data screening conditions from the collection sorting result through the screening conditions; and screen out the final data tuples corresponding to each set of data screening conditions from all the candidate data tuples corresponding to each set of data screening conditions according to the preset optimal screening strategy to complete the data screening. Before data screening, the data in the search domain is processed into data tuples, and the screening conditions are also pre-sorted, and monotone stack storage is introduced for the candidate data tuples. When actually performing data screening, the data screening efficiency is improved and time consumption is saved. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0056] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0057] Figure 2 is a flow chart of an embodiment of a data screening method according to the present application;

[0058] Figure 3 yes Figure 2 A flowchart of a specific embodiment of step 202 is shown;

[0059] Figure 4 yes Figure 3 A flowchart of a specific embodiment of step 302 is shown;

[0060] Figure 5 yes Figure 4 A flowchart of a specific embodiment of step 404 is shown;

[0061] Figure 6 is a flowchart of a specific embodiment of sorting the multiple groups of data screening conditions in the data screening method described in the present application;

[0062] Figure 7 yes Figure 6 A flowchart of a specific embodiment of step 603 is shown;

[0063] Figure 8 is a flowchart of a specific embodiment of caching candidate data tuples into a target data cache stack in the data screening method described in the present application;

[0064] Fig. 9 is a structural schematic diagram of an embodiment of a data screening device according to the present application;

[0065] Fig.10 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0066] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0067] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0068] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0069] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0070] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0071] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to a laptop computer 1011, a tablet computer 1012 or a mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0072] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0073] It should be noted that the data screening method provided in the embodiment of the present application is generally executed by a server, and accordingly, the data screening device is generally set in the server.

[0074] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0075] Continue to refer Figure 2 , shows a flow chart of an embodiment of a data screening method according to the present application. The data screening method comprises the following steps:

[0076] Step 201, obtaining numerical data respectively involved in target data screening from multiple source data terminals.

[0077] In this embodiment, the multi-source data terminal includes exposure data or / and click data recording terminals corresponding to multiple application scenarios, for example: exposure data or / and click data of the same product on different shopping application APPs. Obviously, the corresponding data terminal not only records the exposure data or / and click data of the target product, but also records the exposure data or / and click data corresponding to all products. The numerical data participating in the target data screening in the multi-source data terminal can be understood as the exposure data or / and click data corresponding to all products on multiple shopping application APPs. The more products on each shopping application APP, the more corresponding exposure data or / and click data.

[0078] By obtaining the numerical data that participates in the target data screening from multiple source data terminals, it is convenient to perform pre-screening processing on the data in the big data scenario.

[0079] Step 202, according to the difference of multi-source data terminals, the digitized data is aggregated into data tuples to obtain an aggregated result with data tuples as elements.

[0080] In this embodiment, it is assumed that the multi-source data terminals are two different data terminals A and B, and the numerical data corresponding to the data terminal A are numberA1, numberA2, numberA3, numberA4, numberA5, numberA6, numberA7 and numberA8, and the numerical data corresponding to the data terminal B are numberB1, numberB2, numberB3, numberB4, numberB5, numberB6, numberB7 and numberB8;

[0081] Specifically, according to the differences of the multi-source data terminals, the numeralized data are sorted into data tuples to obtain a set sorting result with data tuples as elements. It can be understood that the numeralized data in the data terminal A and the data terminal B are sorted into data tuples to obtain a set sorting result of ListAB = [(numberA1, numberB1), (numberA2, numberB2), (numberA3, numberB3), (numberA4, numberB4), (numberA5, numberB5), (numberA6, numberB6), (numberA7, numberB7), (numberA8, numberB8)]. Obviously, (numberA1, numberB1) is a binary data tuple, and ListAB is a set with a length of 8, that is, the number of elements is 8.

[0082] Correspondingly, assuming that the multi-source data end also includes a C data end, and the numeralized data corresponding to the data end C are numberC1, numberC2, numberC3, numberC4, numberC5, numberC6, numberC7 and numberC8 respectively, the numeralized data in the data ends A, B and C are sorted into data tuples to obtain the set sorting result of ListABC = [(numberA1, numberB1, numberC1), (numberA2, numberB2, numberC2), (numberA3, numberB3, numberC3), (numberA4, numberB4, numberC4), (numberA5, numberB5, numberC5), (numberA6, numberB6, numberC6), (numberA7, numberB7, numberC7), (numberA8, numberB8, numberC8)], obviously, (numberA1, numberB1, numberC1) is a three-dimensional data tuple, and ListABC is a set with a length of 8, that is, the number of elements is 8.

[0083] Therefore, it can be understood that, assuming that the number of data ends of the multi-source data end is M and the set length is N, the finally generated set sorting result with data tuples as elements is actually the sorting result with a set length of N and a data tuple length of M, that is, M metadata tuples.

[0084] By utilizing the differences in multiple source data terminals, the numeralized data is aggregated and organized into data tuples to obtain aggregated organization results with data tuples as elements, so as to facilitate subsequent faster and more efficient data screening.

[0085] Step 203, identifying multiple groups of data screening conditions corresponding to the multi-source data terminals and comprehensive screening conditions corresponding to the multi-source data terminals, wherein each group of data screening conditions includes numerical screening data of all data terminals in the multi-source data terminals in the current group.

[0086] Specifically, the multiple sets of data filtering conditions corresponding to the multi-source data terminals can be understood as, for example, product a, the target filtering click volumes corresponding to the data terminals A, B and C the day before yesterday, yesterday and today respectively. Therefore, the multiple sets of data filtering conditions include three sets of filtering conditions, corresponding to the data filtering conditions of the day before yesterday, yesterday and today respectively. Moreover, the target filtering click volumes are different at different data terminals and at different times. For example, one set of data filtering conditions among the multiple sets of data filtering conditions corresponding to the multi-source data terminals is to filter out numerical data greater than queryn1 from data terminal A, filter out numerical data greater than queryn2 from data terminal B, and filter out numerical data greater than queryn3 from data terminal C, then (queryn1, queryn2, queryn3) is this set of data filtering conditions.

[0087] Furthermore, the comprehensive screening conditions corresponding to the multi-source data terminals, for example: requiring that the numerical data of the screened data terminal A is greater than queryn1, the numerical data of the screened data terminal B is greater than queryn2, the numerical data of the screened data terminal C is greater than queryn3, and the sum of the numerical data of the three screened data terminals is greater than 2 times (queryn1+queryn2+queryn3).

[0088] By identifying the multiple groups of data screening conditions corresponding to the multi-source data terminals and the comprehensive screening conditions corresponding to the multi-source data terminals, it is convenient to perform many-to-many data screening from the aggregated sorting results with data tuples as elements.

[0089] Step 204: All candidate data tuples corresponding to each set of data screening conditions are screened out from the aggregated sorting results using the multiple sets of data screening conditions and the comprehensive screening conditions.

[0090] Specifically, assuming that the candidate data tuples satisfying each of the above-mentioned sets of data filtering conditions are not unique, that is, all candidate data tuples corresponding to each set of data filtering conditions are filtered out from the aggregated sorting results through the multiple sets of data filtering conditions and the comprehensive filtering conditions, thereby ensuring the comprehensiveness of the screening and avoiding screening omissions.

[0091] Step 205 , according to the preset optimal screening strategy, the final data tuples corresponding to each set of data screening conditions are screened out from all the candidate data tuples corresponding to each set of data screening conditions, thereby completing the data screening.

[0092] Specifically, the preset optimal screening strategy, for example, selects the candidate data tuple with the largest data element and value from all candidate data tuples corresponding to each set of data screening conditions as the final data tuple. The preset optimal screening strategy is set by the specific screening personnel when actually screening data, for example, obtaining the optimal value, the worst value, the majority, etc.

[0093] In this embodiment, the numerical data participating in the target data screening are obtained from the multi-source data terminals; the numerical data are aggregated and sorted into data tuples to obtain the aggregated sorting results with data tuples as elements; the screening conditions are identified; all the candidate data tuples corresponding to each set of data screening conditions are screened out from the aggregated sorting results by the screening conditions; the final data tuples corresponding to each set of data screening conditions are screened out from all the candidate data tuples corresponding to each set of data screening conditions according to the preset optimal screening strategy, thus completing the data screening. Before data screening, the data in the search domain are aggregated into data tuples, and the screening conditions are also pre-sorted, and monotone stack storage is introduced for the candidate data tuples, which improves the data screening efficiency and saves time consumption when actually performing data screening.

[0094] Continue to refer Figure 3 , in some optional implementations, Figure 3 yes Figure 2 The flowchart of a specific embodiment of step 202 shown includes the following steps:

[0095] Step 301, according to the difference of multiple source data terminals, the numerical data is arrayed to obtain the numerical data array corresponding to each data terminal;

[0096] Specifically, continue with the implementation in the above step 202, that is, firstly organize the numerical data corresponding to each data end into an array according to the differences in the multi-source data ends, and obtain the numerical data array corresponding to each data end, so that subsequently only data needs to be obtained from the numerical data array corresponding to each data end to construct and generate the corresponding set.

[0097] Step 302, according to the array subscript, obtain the numeralized data with the same array subscript from the numeralized data array corresponding to each data end, and generate a sorting result with a set length of N and a data tuple length of M, wherein M represents the number of data ends of the multi-source data ends, and N represents the maximum number of numeralized data participating in the target data screening among the M data ends, and N and M are positive integers.

[0098] Specifically, according to the array subscript, the numerated data with the same array subscript is obtained from the numerated data array corresponding to each data end, and a sorting result with a set length of N and a data tuple length of M is generated, that is, the numerated data under the same array are added to the same data tuple, and a set sorting result with data tuples as elements is generated.

[0099] Continue to refer Figure 4 , in some optional implementations, Figure 4 yes Figure 3 The flowchart of a specific embodiment of step 302 shown includes the following steps:

[0100] Step 401, calculating the length of the digitized data array corresponding to each data end according to a preset array length calculation method;

[0101] Specifically, the preset array length calculation method, for example, calculates the length of the numerical data array corresponding to each data end through the length attribute in the Ar ray class of the Java language.

[0102] Step 402, by comparing two by two, selecting the longest numerical data array from all numerical data arrays corresponding to the multi-source data terminals;

[0103] Specifically, it can be understood that the number of digitized data recorded in each data end is different, and therefore, the lengths of all digitized data arrays corresponding to the multi-source data ends are different.

[0104] Step 403, setting the length of the longest numerical data array to the set length N;

[0105] Specifically, the length of the longest numerical data array is set to the set length N, thereby ensuring that all numerical data participating in the target data screening acquired in step 201 are screened.

[0106] Step 404, sequentially obtaining the digitized data with the same array subscript from all digitized data arrays corresponding to the multi-source data end, adding the digitized data with the same array subscript to the same data tuple, and generating N data tuples with a length of M;

[0107] Step 405, performs set processing on the N data tuples of the generated length M to obtain a sorting result with a set length of N and a data tuple length of M, wherein N represents the length of the longest digitized data array, and M represents the number of all digitized data arrays corresponding to the multi-source data terminals.

[0108] By utilizing the differences in multiple source data terminals, the numeralized data is aggregated and organized into data tuples to obtain aggregated organization results with data tuples as elements, so as to facilitate subsequent faster and more efficient data screening.

[0109] Continue to refer Figure 5 , in some optional implementations, Figure 5 yes Figure 4 The flowchart of a specific embodiment of step 404 shown includes the following steps:

[0110] Step 501, using a preset array element extraction method to extract the numerical data corresponding to the current array subscript of the current numerical data array to obtain an extraction result;

[0111] Specifically, the preset array element extraction method, for example: arr[i], wherein i represents the array subscript, the value range of i is 0 to arr length-1, and arr length represents the array length.

[0112] Step 502, if the extraction result is not a null value, then the numerical data is obtained, and the numerical data is added to the data tuple corresponding to the current array index;

[0113] Step 503, if the extraction result is a null value, then add a corresponding null value at the reserved position of the data tuple corresponding to the current array index;

[0114] Step 504, taking 0 to N in turn as the extraction control parameters of the preset array element extraction method, repeating steps 501 to 503 to generate N data tuples of length M.

[0115] Through null value judgment, it is ensured that the elements in the N generated data tuples are all M, avoiding the generation of inconsistent length data tuples, that is, only M-element data tuples are generated, and data tuples with inconsistent numbers of elements such as one-element, two-element, three-element, M-element, etc. are not generated, so as to facilitate subsequent data acquisition and screening.

[0116] Continue to refer Figure 6 In some optional implementations, after step 203, a step of sorting the multiple sets of data screening conditions is also included. Figure 6 It is a flowchart of a specific embodiment of sorting the multiple groups of data screening conditions in the data screening method described in the present application, comprising the following steps:

[0117] Step 601, selecting any one of the multiple source data ends as a target data end;

[0118] Step 602, identifying the numerical screening data corresponding to the target data end from each set of data screening conditions;

[0119] Step 603, sorting the plurality of sets of data screening conditions according to the size relationship of all numerical screening data corresponding to the target data end, to obtain the plurality of sets of data screening conditions after sorting.

[0120] The multiple sets of data screening conditions are sorted based on the size relationship of all numerical screening data corresponding to the target data end to obtain the sorted multiple sets of data screening conditions, so as to facilitate subsequent data screening in combination with the size relationship and save screening time.

[0121] Continue to refer Figure 7 , in some optional implementations, Figure 7 yes Figure 6 The flowchart of a specific embodiment of step 603 shown includes the following steps:

[0122] Step 701, performing descending processing on all numerical screening data corresponding to the target data end to obtain a descending sequence;

[0123] Step 702, identifying each set of data screening conditions corresponding to all numerical screening data in the descending sequence, and obtaining a recognition result;

[0124] Step 703: sorting the plurality of sets of data screening conditions according to the recognition result to obtain the plurality of sets of data screening conditions after the sorting.

[0125] The plurality of groups of data screening conditions are sorted by the size relationship of all numerical screening data corresponding to the target data end to obtain the plurality of groups of data screening conditions after sorting, so as to facilitate the subsequent combination with the sorting sequence to perform data screening and save the screening time.

[0126] Continue to refer Figure 8 In some optional implementations, after step 204, a step of caching the candidate data tuple into a target data cache stack is also included. Figure 8 This is a flowchart of a specific embodiment of caching candidate data tuples into a target data cache stack in the data screening method described in the present application, comprising the following steps:

[0127] Step 801, sequentially obtaining all candidate data tuples corresponding to each set of data screening conditions;

[0128] Step 802, based on the comprehensive screening condition, select a better candidate data tuple from the currently acquired candidate data tuple and the previously acquired candidate data tuple to obtain a screening result;

[0129] Specifically, the comprehensive screening condition, for example, takes the candidate data tuple when the sum of the numerical data screened from multiple data ends is the maximum value as a preferred candidate data tuple.

[0130] Step 803, according to the screening result, cache the better candidate data tuples corresponding to each set of data screening conditions into a target data cache stack, wherein each set of data screening conditions corresponds to a corresponding data cache stack;

[0131] Step 804 , repeatedly executing steps 801 to 803 , and caching the optimal candidate data tuples corresponding to each set of data screening conditions into the target data cache stack.

[0132] It can be understood that the preset optimal screening strategy can be a supplementary condition of the comprehensive screening condition, or a collaborative condition of the comprehensive screening condition, and is specifically set according to the actual application scenario.

[0133] In this embodiment, the steps of filtering out the final data tuples corresponding to each set of data filtering conditions from all the candidate data tuples corresponding to each set of data filtering conditions according to the preset optimal filtering strategy to complete this data filtering include: obtaining the optimal candidate data tuples corresponding to the multiple sets of data filtering conditions from the target data cache stack; and taking the optimal candidate data tuples corresponding to the multiple sets of data filtering conditions as the corresponding final data tuples.

[0134] Specifically, the target data cache stack is a monotonic stack, that is, each candidate data tuple has only one push and pop operation. This approach greatly reduces the time complexity, reduces the time for data screening, and improves data screening efficiency.

[0135] By adding the candidate data tuples obtained each time into the target data cache stack in combination with the preset optimal screening strategy and comprehensive screening conditions, the optimal candidate data tuples corresponding to the multiple sets of data screening conditions can be directly obtained from the target data cache stack later.

[0136] The data screening method described in this embodiment is mainly used in data screening scenarios from multiple data ends under multiple screening conditions in big data scenarios. It introduces the collection of data in the search domain with data tuples as elements, sorting and serializing of multiple screening conditions, and monotonic stack storage of candidate data tuples. When actually screening big data, it improves data screening efficiency and saves time compared to direct screening.

[0137] The present application obtains numerical data respectively involved in the target data screening from multi-source data terminals; performs data tuple collection and sorting on the numerical data to obtain a collection sorting result with data tuples as elements; identifies the screening conditions; screens out all candidate data tuples corresponding to each set of data screening conditions from the collection sorting result through the screening conditions; and screens out the final data tuples corresponding to each set of data screening conditions from all candidate data tuples corresponding to each set of data screening conditions according to the preset optimal screening strategy to complete the data screening. Before data screening, the data in the search domain is processed into data tuples, and the screening conditions are also pre-sorted, and monotone stack storage is introduced for the candidate data tuples. When actually performing data screening, the data screening efficiency is improved and time consumption is saved.

[0138] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0139] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0140] In the embodiment of the present application, the numerical data respectively participating in the target data screening in the multi-source data end are obtained; the numerical data are aggregated and sorted into data tuples to obtain the aggregated sorting result with data tuples as elements; the screening conditions are identified; all the candidate data tuples corresponding to each set of data screening conditions are screened out from the aggregated sorting result by the screening conditions; the final data tuples corresponding to each set of data screening conditions are screened out from all the candidate data tuples corresponding to each set of data screening conditions according to the preset optimal screening strategy, so as to complete the data screening. Before the data screening, the data in the search domain are aggregated into data tuples, and the screening conditions are also pre-sorted, and monotone stack storage is introduced for the candidate data tuples, which improves the data screening efficiency and saves time consumption when the data screening is actually performed.

[0141] Further references Fig. 9 , as a response to the above Figure 2 In order to realize the method shown in the figure, the present application provides an embodiment of a data screening device, and the device embodiment is Figure 2Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0142] like Fig. 9 As shown, the data screening device 900 described in this embodiment includes: a data acquisition module 901 for participating in screening, a collection arrangement module 902, a screening condition identification module 903, a candidate data tuple screening module 904 and a final data tuple screening module 905. Among them:

[0143] The data acquisition module 901 is used to acquire numerical data that are respectively involved in the target data screening from multiple source data terminals;

[0144] The collection arrangement module 902 is used to collect and arrange the numerical data into data tuples according to the differences of the multi-source data terminals, and obtain a collection arrangement result with data tuples as elements;

[0145] A screening condition identification module 903, used to identify multiple groups of data screening conditions corresponding to the multi-source data terminals, and comprehensive screening conditions corresponding to the multi-source data terminals, wherein each group of data screening conditions includes numerical screening data of all data terminals in the multi-source data terminals in the current group;

[0146] A candidate data tuple screening module 904 is used to screen out all candidate data tuples corresponding to each set of data screening conditions from the aggregated sorting results through the multiple sets of data screening conditions and the comprehensive screening conditions;

[0147] The final data tuple screening module 905 is used to screen out the final data tuples corresponding to each set of data screening conditions from all candidate data tuples corresponding to each set of data screening conditions according to a preset optimal screening strategy, so as to complete this data screening.

[0148] The present application obtains numerical data respectively involved in the target data screening from multi-source data terminals; performs data tuple collection and sorting on the numerical data to obtain a collection sorting result with data tuples as elements; identifies the screening conditions; screens out all candidate data tuples corresponding to each set of data screening conditions from the collection sorting result through the screening conditions; and screens out the final data tuples corresponding to each set of data screening conditions from all candidate data tuples corresponding to each set of data screening conditions according to the preset optimal screening strategy to complete the data screening. Before data screening, the data in the search domain is processed into data tuples, and the screening conditions are also pre-sorted, and monotone stack storage is introduced for the candidate data tuples. When actually performing data screening, the data screening efficiency is improved and time consumption is saved.

[0149] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0150] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0151] To solve the above technical problems, the present application also provides a computer device. Fig.10 , Fig.10 This is a basic structural block diagram of the computer device in this embodiment.

[0152] The computer device 10 includes a memory 10a, a processor 10b, and a network interface 10c which are interconnected and communicated via a system bus. It should be noted that: Fig.10 Only a computer device 10 having a component memory 10a, a processor 10b, and a network interface 10c is shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital processor (DSP), an embedded device, etc.

[0153] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0154] The memory 10a includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10a can be an internal storage unit of the computer device 10, such as a hard disk or memory of the computer device 10. In other embodiments, the memory 10a can also be an external storage device of the computer device 10, such as a plug-in hard disk equipped on the computer device 10, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 10a can also include both the internal storage unit of the computer device 10 and its external storage device. In this embodiment, the memory 10a is generally used to store the operating system and various application software installed on the computer device 10, such as a computer-readable instruction of a data screening method, etc. In addition, the memory 10a can also be used to temporarily store various types of data that have been output or are to be output.

[0155] The processor 10b may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 10b is generally used to control the overall operation of the computer device 10. In this embodiment, the processor 10b is used to run the computer-readable instructions stored in the memory 10a or process data, such as computer-readable instructions for running the data screening method.

[0156] The network interface 10c may include a wireless network interface or a wired network interface. The network interface 10c is generally used to establish a communication connection between the computer device 10 and other electronic devices.

[0157] The computer device proposed in this embodiment belongs to the field of big data technology and is applied to many-to-many data retrieval scenarios with multiple screening conditions and multiple source data terminals. The present application obtains the numerical data that respectively participates in the screening of target data from the multiple source data terminals; performs data tuple collection and sorting on the numerical data to obtain a collection sorting result with data tuples as elements; identifies the screening conditions; selects all candidate data tuples corresponding to each set of data screening conditions from the collection sorting result through the screening conditions; selects the final data tuples corresponding to each set of data screening conditions from all candidate data tuples corresponding to each set of data screening conditions according to the preset optimal screening strategy, and completes this data screening. By performing data tuple collection processing on the data in the search domain before data screening, and pre-sorting the screening conditions, and introducing monotone stack storage for the candidate data tuples, the data screening efficiency is improved and time consumption is saved when actually performing data screening.

[0158] The present application also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by a processor to enable the processor to perform the steps of the data screening method as described above.

[0159] The computer-readable storage medium proposed in this embodiment belongs to the field of big data technology and is applied to many-to-many data retrieval scenarios with multiple screening conditions and multiple source data terminals. This application obtains the numerical data that respectively participates in the screening of target data from the multiple source data terminals; performs data tuple collection and sorting on the numerical data to obtain a collection sorting result with data tuples as elements; identifies the screening conditions; selects all candidate data tuples corresponding to each set of data screening conditions from the collection sorting results through the screening conditions; selects the final data tuples corresponding to each set of data screening conditions from all candidate data tuples corresponding to each set of data screening conditions according to the preset optimal screening strategy, and completes this data screening. By performing data tuple collection processing on the data in the search domain before data screening, and pre-sorting the screening conditions, and introducing monotone stack storage for the candidate data tuples, the data screening efficiency is improved and time consumption is saved when actually performing data screening.

[0160] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0161] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A data screening method, characterized in that: The steps include: Obtain numerical data from multiple source data terminals that are respectively involved in the screening of target data; According to the difference of multi-source data terminals, the numerical data is aggregated into data tuples to obtain an aggregated aggregation result with data tuples as elements; Identifying multiple groups of data screening conditions corresponding to the multi-source data ends, and comprehensive screening conditions corresponding to the multi-source data ends, wherein each group of data screening conditions includes numerical screening data of all data ends in the multi-source data ends in the current group; Filtering all candidate data tuples corresponding to each set of data screening conditions from the aggregated sorting results by using the multiple sets of data screening conditions and the comprehensive screening conditions; According to the preset optimal screening strategy, the final data tuples corresponding to each set of data screening conditions are screened out from all the candidate data tuples corresponding to each set of data screening conditions, thereby completing this data screening.

2. The data screening method according to claim 1, characterized in that: The step of performing data tuple collection and sorting on the digitized data according to the differences of the multi-source data terminals to obtain a collection sorting result with the data tuple as an element specifically includes: According to the differences of the multi-source data terminals, the numerical data are sorted into arrays to obtain numerical data arrays corresponding to each data terminal; According to the array subscript, the numerated data with the same array subscript is obtained from the numerated data array corresponding to each data end, and a sorting result with a set length of N and a data tuple length of M is generated, wherein M represents the number of data ends of the multi-source data ends, and N represents the maximum number of numerated data participating in the target data screening among the M data ends, and N and M are positive integers.

3. The data screening method according to claim 2, characterized in that: The step of obtaining the digitized data with the same array subscript from the digitized data arrays corresponding to each data end according to the array subscript, and generating a sorting result with a set length of N and a data tuple length of M specifically includes: According to the preset array length calculation method, the length of the numerical data array corresponding to each data end is calculated; By comparing two by two, selecting the longest numerical data array from all numerical data arrays corresponding to the multi-source data terminals; The length of the longest numerical data array is set as the set length N; Sequentially obtain the digitized data with the same array subscript from all digitized data arrays corresponding to the multi-source data end, add the digitized data with the same array subscript to the same data tuple, and generate N data tuples with a length of M; The N data tuples with a generated length of M are subjected to set processing to obtain a sorting result with a set length of N and a data tuple length of M, wherein N represents the length of the longest digitized data array, and M represents the number of all digitized data arrays corresponding to the multi-source data terminals.

4. The data screening method according to claim 3, characterized in that: The step of sequentially obtaining the digitized data with the same array subscript from all digitized data arrays corresponding to the multi-source data terminals, adding the digitized data with the same array subscript to the same data tuple, and generating N data tuples with a length of M specifically includes: Step 501, using a preset array element extraction method to extract the numerical data corresponding to the current array subscript of the current numerical data array to obtain an extraction result; Step 502, if the extraction result is not a null value, then the numerical data is obtained, and the numerical data is added to the data tuple corresponding to the current array index; Step 503, if the extraction result is a null value, then add a corresponding null value at the reserved position of the data tuple corresponding to the current array index; Step 504, taking 0 to N in turn as the extraction control parameters of the preset array element extraction method, repeating steps 501 to 503 to generate N data tuples of length M.

5. The data screening method according to claim 1, characterized in that: After executing the step of identifying the multiple sets of data screening conditions corresponding to the multiple source data ends, the method further includes: Selecting one data end from the multiple source data ends as the target data end; Identify the numerical screening data corresponding to the target data end from each set of data screening conditions; According to the size relationship of all the numerical screening data corresponding to the target data end, the multiple groups of data screening conditions are sorted to obtain the multiple groups of data screening conditions after sorting.

6. The data screening method according to claim 5, characterized in that: The step of sorting the plurality of sets of data screening conditions according to the size relationship of all numerical screening data corresponding to the target data end to obtain the plurality of sets of data screening conditions after sorting specifically includes: Performing descending processing on all numerical screening data corresponding to the target data end to obtain a descending sequence; Identify each set of data screening conditions corresponding to all numerical screening data in the descending sequence, and obtain a recognition result; The multiple groups of data screening conditions are sorted according to the recognition results to obtain the multiple groups of data screening conditions after the sorting process.

7. The data screening method according to claim 1 or 6, characterized in that: After executing the step of filtering out all candidate data tuples corresponding to each set of data filtering conditions from the aggregated sorting results by using the multiple sets of data filtering conditions and the comprehensive filtering conditions, the method further includes: Step 801, sequentially obtaining all candidate data tuples corresponding to each set of data screening conditions; Step 802, based on the comprehensive screening condition, select a better candidate data tuple from the currently acquired candidate data tuple and the previously acquired candidate data tuple to obtain a screening result; Step 803, according to the screening result, cache the better candidate data tuples corresponding to each set of data screening conditions into a target data cache stack, wherein each set of data screening conditions corresponds to a corresponding data cache stack; Step 804, repeatedly executing steps 801 to 803, caching the optimal candidate data tuples corresponding to each set of data screening conditions into the target data cache stack; The step of selecting the final data tuples corresponding to each set of data screening conditions from all the candidate data tuples corresponding to each set of data screening conditions according to the preset optimal screening strategy to complete the data screening specifically includes: Acquire the optimal candidate data tuples corresponding to the plurality of sets of data screening conditions respectively from the target data cache stack; The optimal candidate data tuples corresponding to the multiple sets of data screening conditions are taken as the corresponding final data tuples.

8. A data screening device, characterized in that: include: A data acquisition module for participating in screening is used to obtain numerical data participating in target data screening from multiple source data terminals; A collection arrangement module, used to collect and arrange the numerical data into data tuples according to the differences of multiple source data terminals, and obtain a collection arrangement result with data tuples as elements; A screening condition identification module, used to identify multiple groups of data screening conditions corresponding to the multi-source data ends, and comprehensive screening conditions corresponding to the multi-source data ends, wherein each group of data screening conditions includes numerical screening data of all data ends in the multi-source data ends in the current group; A candidate data tuple screening module, used to screen out all candidate data tuples corresponding to each set of data screening conditions from the aggregated sorting results through the multiple sets of data screening conditions and the comprehensive screening conditions; The final data tuple screening module is used to screen out the final data tuples corresponding to each set of data screening conditions from all candidate data tuples corresponding to each set of data screening conditions according to a preset optimal screening strategy to complete this data screening.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the data screening method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the data screening method according to any one of claims 1 to 7 are implemented.

Images