Data scanning method and related equipment

Abstract

The embodiment of the invention discloses a data scanning method and related equipment. The related equipment can comprise a data scanning device, electronic equipment, a computer program product and a computer readable storage medium. According to the embodiment of the invention, after at least one data scanning request is received, at least one data storage position is identified in a data index of a target database based on data scanning information carried in the data scanning request, and then first data corresponding to the data storage position is read in the target database by adopting a first scanning operator; according to the data scanning request, a first scanning operator is adopted to read first data corresponding to the data storage position in the target database, a reading state corresponding to the first scanning operator is evaluated, then, according to an evaluation result, a second scanning operator is adopted to read second data corresponding to the data storage position in the target database, and the first data and the second data serve as target data corresponding to the data scanning request; according to the scheme, the scanning operators can be adaptively switched, so that the data scanning efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of communication, in particular to a data scanning method and related equipment, which can include a data scanning device, an electronic device, a computer program product and a computer readable storage medium. BACKGROUND

[0002] In recent years, with the rapid development of Internet technology, the scanning operators for data scanning in database systems are also increasingly diversified. When the current database system performs data scanning, the cost of scanning can be estimated before the scanning starts, so that different scanning operators are selected to perform data scanning.

[0003] In the research and practice of the prior art, the present inventors have found that in the process of estimating the scanning cost, there are sampling errors and timeliness problems in the statistical information, and the selection rate and the cardinality estimation based on the independence and uniformity assumptions will bring bias, and the cost model has fitting errors, so that it is difficult to obtain a truly low-cost scanning operator, and therefore the efficiency of data scanning is low. SUMMARY

[0004] Embodiments of the present application provide a data scanning method and related equipment, which can include a data scanning device and a computer readable storage medium, and can improve the efficiency of data scanning.

[0005] A data scanning method comprises:

[0006] receiving at least one data scanning request, the data scanning request carrying data scanning information;

[0007] based on the data scanning information, identifying at least one data storage location in the data index of the target database;

[0008] using a first scanning operator to read first data corresponding to the data storage location in the target database, and evaluating the reading state corresponding to the first scanning operator;

[0009] According to the evaluation result, using a second scanning operator to read second data corresponding to the data storage location in the target database, and taking the first data and the second data as the target data corresponding to the data scanning request.

[0010] Correspondingly, the present application provides a data scanning device, comprising:

[0011] a receiving unit configured to receive at least one data scanning request, the data scanning request carrying data scanning information;

[0012] identify at least one data storage location in a data index of a target database based on the data scan information;

[0013] evaluate a reading state of the first scan operator based on the first data read from the target database using the first scan operator;

[0014] read second data corresponding to the data storage location from the target database using a second scan operator based on the evaluation result, and take the first data and the second data as target data corresponding to the data scan request.

[0015] Optionally, in some embodiments, the evaluation unit can be specifically configured to record the reading state of reading the first data in the target database to obtain reading state information corresponding to the first data, generate target reading parameters corresponding to the first scan operator based on the reading state information, and compare the target reading parameters with preset first reading parameter indicators to obtain the evaluation result of the reading state.

[0016] Optionally, in some embodiments, the reading unit can be specifically configured to filter out data storage locations of unread data from the data storage locations based on the first data when the target reading parameters reach the preset first reading parameter indicators to obtain remaining data storage locations, and read second data corresponding to the remaining data storage locations from the target database using the second scan operator.

[0017] Optionally, in some embodiments, the reading unit can be specifically configured to filter out data storage locations corresponding to the first data from the data storage locations to obtain read data storage locations, and delete the read data storage locations from the data storage locations to obtain remaining data storage locations.

[0018] Optionally, in some embodiments, the reading unit can be specifically configured to sort the remaining data storage locations using the second scan operator to obtain a sorting bitmap corresponding to the remaining data storage locations, and read data corresponding to the remaining data storage locations from the target database based on the sorting bitmap to obtain second data.

[0019] Optionally, in some embodiments, the reading unit can be specifically configured to continue reading the first data corresponding to the data storage location in the target database by using the first scan algorithm when the target reading parameter reaches the preset first reading parameter index, and update the target reading parameter; and read the second data corresponding to the data storage location in the target database by using a second scan algorithm when the updated target reading parameter reaches a preset second reading parameter index.

[0020] Optionally, in some embodiments, the evaluation unit can be specifically configured to return to the step of reading the first data corresponding to the data storage location in the target database by using the first scan algorithm when the target reading parameter does not reach the preset first reading parameter index, until the data storage location is read completely, so as to obtain the target data corresponding to the data scanning request; and return the target data to the at least one scanning terminal corresponding to the data scanning request.

[0021] Optionally, in some embodiments, the evaluation unit can be specifically configured to determine a correlation coefficient between the data index and a heap table of the target database based on the data storage location and the first data, the correlation coefficient indicating a correlation degree between the data index and the heap table; and the evaluation of the reading state corresponding to the first scan algorithm comprises: evaluating the reading state corresponding to the first scan algorithm when the correlation coefficient is less than a preset correlation coefficient threshold.

[0022] Optionally, in some embodiments, the evaluation unit can be specifically configured to continue reading the first data corresponding to the data storage location in the target database by using the first scan algorithm when the correlation coefficient is greater than or equal to the preset correlation coefficient threshold; take the first data as the target data corresponding to the data scanning request, and return the target data to the at least one scanning terminal corresponding to the data scanning request.

[0023] Optionally, in some embodiments, the evaluation unit can be specifically configured to initialize an execution component corresponding to the target database to obtain a target execution component and a target data cache space corresponding to the target execution component; read the data corresponding to the data storage location in the target database according to the target execution component and the first scan algorithm to obtain first data, and add the first data to the target data cache space.

[0024] Optionally, in some embodiments, the evaluation unit can be specifically configured to initialize the preset scan operator corresponding to the execution component to obtain an initialized execution component, the preset scan operator including a first scan operator and a second scan operator; construct an evaluation component corresponding to the preset scan operator, and take the initialized execution component and the evaluation component as a target execution component; and generate a target data cache space corresponding to the target execution component based on the data scan information.

[0025] Optionally, in some embodiments, the evaluation unit can be specifically configured to randomly select a target data storage location in the data storage location by using the target execution component; and read data corresponding to the target data storage location in the target database to obtain first data.

[0026] Optionally, in some embodiments, the evaluation unit can be specifically configured to predict scan resource information corresponding to the data scan request based on the data scan information and the data storage location; and select a first scan operator from a candidate scan operator set according to the scan resource information.

[0027] In addition, an electronic device is further provided in the embodiment of the present application, which comprises a processor and a memory, the memory stores an application program, and the processor is used to run the application program in the memory to execute the data scan method provided in the embodiment of the present application.

[0028] In addition, a computer readable storage medium is further provided in the embodiment of the present application, which stores a plurality of instructions, the instructions are suitable for being loaded by a processor to execute the steps in any one of the data scan methods provided in the embodiment of the present application.

[0029] In addition, a computer program product is further provided in the embodiment of the present application, which comprises a computer program or instructions, and the computer program or instructions are executed by a processor to realize the steps in the data scan method provided in the embodiment of the present application.

[0030] The embodiment of the application receives at least one data scanning request, identifies at least one data storage location in a data index of a target database based on data scanning information carried in the data scanning request, then reads first data corresponding to the data storage location in the target database by using a first scanning operator, evaluates a reading state corresponding to the first scanning operator, then reads second data corresponding to the data storage location in the target database by using a second scanning operator according to the evaluation result, and takes the first data and the second data as target data corresponding to the data scanning request. Since the scheme can read data in the target database by using the first scanning operator, evaluate the reading state of the data reading, switch the first scanning operator to the second scanning operator according to the evaluation result, switch the selection of the scanning operator to the execution stage, smoothly switch the scanning operator according to the real execution feedback (reading state), and thus obtain a scanning operator with low real overhead to perform data scanning, the efficiency of data scanning can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative effort.

[0032] Figure 1 is a scene diagram of the data scanning method provided by the embodiment of the application;

[0033] Figure 2 is a flow diagram of the data scanning method provided by the embodiment of the application;

[0034] Figure 3 is a diagram of switching the scanning operator by using a preset reading parameter index provided by the embodiment of the application;

[0035] Figure 4 is a whole flow diagram of the initialization state and the execution state provided by the embodiment of the application;

[0036] Figure 5 is a whole architecture diagram of the data scanning provided by the adaptive scanning operator switching provided by the embodiment of the application;

[0037] Figure 6 is a whole flow diagram of the data scanning provided by the adaptive scanning operator switching provided by the embodiment of the application;

[0038] Figure 7 is a scanning result diagram provided by the embodiment of the application, which only uses index scanning;

[0039] Figure 8 This is a schematic diagram of the scan result after switching from index scan to bitmap scan, provided in an embodiment of this application.

[0040] Figure 9 This is another schematic flowchart of the data scanning method provided in the embodiments of this application;

[0041] Figure 10 This is a schematic diagram of the structure of the data scanning device provided in the embodiments of this application;

[0042] Figure 11 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0043] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] This application provides a data scanning method and related equipment. The related equipment may include a data scanning device, an electronic device, a computer program product, and a computer-readable storage medium. The data scanning device may be integrated into an electronic device, which may be a server or a terminal, etc.

[0045] The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial intelligence platforms. The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0046] For example, see Figure 1For example, if the data scanning device is integrated in an electronic device, the electronic device can identify at least one data storage location in the data index of the target database based on the data scanning information carried in the data scanning request after receiving the data scanning request, then read first data corresponding to the data storage location in the target database by using a first scanning operator, evaluate the reading state of the first scanning operator, then read second data corresponding to the data storage location in the target database by using a second scanning operator according to the evaluation result, and take the first data and the second data as target data corresponding to the data scanning request, thereby improving the efficiency of data scanning.

[0047] The data scanning method provided in the embodiments of the present application relates to database technology in the cloud infrastructure technology of cloud technology. The embodiments of the present application can identify at least one data storage location in the data index of the target database based on the data scanning information, read first data corresponding to the data storage location in the target database by using a first scanning operator, evaluate the reading state of the first scanning operator, read second data corresponding to the data storage location in the target database by using a second scanning operator according to the evaluation result, and take the first data and the second data as target data corresponding to the data scanning request, thereby improving the efficiency of data scanning.

[0048] The cloud technology refers to a kind of hosting technology that unifies series resources such as hardware, software and network in wide area network or local area network to realize the calculation, storage, processing and sharing of data. The cloud technology can be understood as the general term of network technology, information technology, integration technology, management platform technology, application technology and the like based on cloud computing business model application, can form a resource pool, use on demand, flexible and convenient. Cloud computing technology will become an important support. The background service of technical network system needs a large amount of calculation and storage resources, such as video website, picture website and more portal website. With the high development and application of Internet industry, every item may have its own identification mark in the future, which needs to be transmitted to the background system for logical processing. Different levels of data will be processed separately, and various industry data need strong system support, which can only be realized by cloud computing.

[0049] The database can be simply regarded as an electronic file cabinet, i.e. a place for storing electronic files, and users can add, query, update and delete data in the files. The so-called "database" is a collection of data stored together in a certain way, shared by multiple users, with as little redundancy as possible, and independent of application programs.

[0050] A database management system (DBMS) is a computer software system designed for managing databases, typically with functions such as storage, retrieval, security, backup, and other basic functions. Database management systems can be classified according to the database model they support, such as relational, XML (Extensible Markup Language); or by the type of computer they support, such as server clusters, mobile phones; or by the query language they support, such as SQL (Structured Query Language), XQuery; or by the performance focus, such as maximum size, highest speed; or other ways of classification. Regardless of the classification method used, some DBMSs can cross categories, such as supporting multiple query languages at the same time.

[0051] In the specific embodiments of the present application, the first data, the second data, or other related data in the target database of the object are involved, and when the embodiments of the present application are applied to specific products or technologies, permission or consent is required, and the collection, use, and processing of the related data need to comply with relevant laws, regulations, and standards of the country and region.

[0052] The following will be described in detail. It should be noted that the order of description of the following embodiments is not limited as the preferred order of the embodiments.

[0053] The present embodiment will be described from the perspective of a data scanning device, which can be integrated in an electronic device, which can be a server, a terminal, or other devices. The terminal can include a tablet computer, a notebook computer, a personal computer (PC), a wearable device, a virtual reality device, or other smart devices that can perform data scanning.

[0054] A data scanning method includes:

[0055] Receiving at least one data scanning request carrying data scanning information; based on the data scanning information, identifying at least one data storage location in the data index of the target database; using a first scanning operator to read first data corresponding to the data storage location in the target database, and evaluating the reading state of the first scanning operator; according to the evaluation result, using a second scanning operator to read second data corresponding to the data storage location in the target database, and taking the first data and the second data as target data corresponding to the data scanning request.

[0056] AsFigure 2 As shown in the figure, the specific flow of the data scanning method is as follows:

[0057] 101. Receive at least one data scanning request.

[0058] The data scanning request can be understood as a request for data scanning or data reading in the target database. The data scanning request carries data scanning information. The so-called data scanning information can be understood as information for indicating the scanning range of data scanning. The data scanning request can include a to-be-scanned data identifier, a to-be-scanned data identifier interval or range, a data scanning condition, or other information indicating the data scanning range, etc.

[0059] The way of receiving at least one data scanning request can be various, and can be as follows:

[0060] For example, at least one data scanning request sent by a scanning terminal or a client can be directly accepted, the data scanning request carrying data scanning information, or a data scanning page including a scanning data input control is displayed, and a data scanning request carrying data scanning information corresponding to an input operation on the scanning data input control is generated in response to the input operation, or at least one data scanning request sent by a third-party scanning platform can be received, etc.

[0061] 102. Identify at least one data storage location in the data index of the target database based on the data scanning information.

[0062] The data storage location can be understood as a location in the target database where the data corresponding to the data scanning information is stored. The type of data storage location can be various, such as a row number, a data block identifier, a data block location, an index identifier, or other location types, etc.

[0063] The way of identifying at least one data storage location in the data index of the target database based on the data scanning information can be various, and can be as follows:

[0064] For example, the data index of the target database can be obtained, at least one data identifier, data range identifier, or data interval identifier can be obtained in the data scanning information, and at least one data storage location corresponding to the data identifier, data range identifier, or data interval identifier can be identified in the data index, or the data index of the target database can be obtained, the index information corresponding to the data scanning information can be filtered out in the data index, and at least one data storage location can be identified in the index information, etc.

[0065] 103. read the first data corresponding to the data storage location in the target database by using the first scan operator, and evaluate the read state corresponding to the first scan operator.

[0066] The first scan operator can be understood as a scan operator for scanning data in the target database. The scan operator can be understood as a scan method or algorithm for scanning data in a database system. The scan operator can have various types, such as full table (sequential), index scan, bitmap scan, or other scan methods. The full table (sequential) scan algorithm has a time complexity of 0(n), but because it scans data in the order of disk organization, there is no random read problem, so it has an advantage when the selection rate is high (the cardinality is large). The index scan algorithm has a time complexity of 0(log(n)), and because it uses a B+ tree index, it has an advantage when the selection rate is low (the cardinality is small), and the overhead caused by random reading is controllable. The bitmap scan algorithm has a time complexity of 0(log(n)), which also uses a B+ tree, so it has an advantage when the selection rate is medium, because it forms a bitmap by sorting tid and masks the overhead of random reading.

[0067] The first scan operator can be understood as a scan operator for scanning data in the target database. The scan operator can be understood as a scan method or algorithm for scanning data in a database system. The scan operator can have various types, such as full table (sequential), index scan, bitmap scan, or other scan methods. The full table (sequential) scan algorithm has a time complexity of 0(n), but because it scans data in the order of disk organization, there is no random read problem, so it has an advantage when the selection rate is high (the cardinality is large). The index scan algorithm has a time complexity of 0(log(n)), and because it uses a B+ tree index, it has an advantage when the selection rate is low (the cardinality is small), and the overhead caused by random reading is controllable. The bitmap scan algorithm has a time complexity of 0(log(n)), which also uses a B+ tree, so it has an advantage when the selection rate is medium, because it forms a bitmap by sorting tid and masks the overhead of random reading.

[0068] S1. read the first data corresponding to the data storage location in the target database by using the first scan operator.

[0069] For example, the execution component corresponding to the target database can be initialized to obtain the target execution component and the target data cache space corresponding to the target execution component, read the data corresponding to the data storage location in the target database according to the target execution component and the first scan operator, obtain the first data, and add the first data to the target data cache space.

[0070] The execution component can be understood as an executor for executing data scanning or reading in the target database, which can be a physical or virtual component. The execution component corresponding to the target database can be initialized in various ways, such as initializing the corresponding preset scan operator in the execution component to obtain the initialized execution component, the preset scan operator can include the first scan operator and the second scan operator, constructing the evaluation component corresponding to the preset scan operator, and taking the initialized execution component and the evaluation component as the target execution component, generating the target data cache space corresponding to the target execution component based on the data scanning information.

[0071] The initialization of the preset scan operator corresponding to the execution component is for preparing the preset scan operator.

[0072] The evaluation component can be an evaluator for evaluating the switching of scan operators. The evaluator can be a virtual or physical component. The evaluation component is used to switch between the first scan operator and the second scan operator during the data scanning process. For example, if the preset scan operators include index scan and bitmap scan, the evaluation component can switch between the index scan and the bitmap scan.

[0073] The target data cache space can be used to store the tid buffer of the ID of the tuple. The first data and the second data are stored in the target data cache space, so that the tuple is not returned to the scanning terminal or the client too early.

[0074] After the initialization of the execution component corresponding to the target database, the data at the data storage location in the target database can be read according to the initialized target execution component and the first scan operator, so as to obtain the first data. The data at the data storage location in the target database can be read in various ways according to the target execution component and the first scan operator. For example, if the first scan operator is an index scan, the target execution component can be used to randomly select a target data storage location in the data storage location, read the data at the target data storage location in the target database, and obtain the first data.

[0075] Optionally, in some embodiments, the first scan operator can be selected from the candidate scan operator set before the first data at the data storage location in the target database is read by the first scan operator. The first scan operator can be selected from the candidate scan operator set in various ways. For example, the scan resource information corresponding to the data scanning request can be predicted based on the data scanning information and the data storage location, and the first scan operator can be selected from the candidate scan operator set according to the scan resource information.

[0076] The scan resource information can be understood as the estimated computing resources or overheads required for data scanning. The first scan operator can be selected from the candidate scan operator set in various ways based on the scan resource information. For example, the reference scan resource information of each candidate scan operator in the candidate scan operator set can be obtained, the scan resource information can be compared with the reference scan resource information, and the first scan operator can be selected from the candidate scan operator set based on the comparison result.

[0077] S2, evaluating the read state corresponding to the first scan operator.

[0078] The read state can be understood as an execution state of reading the first data by the target execution component through the first scan operator. The execution state can include state information when reading the first data, statistical information when reading the first data, computing resource consumption information when reading the first data, or other state information indicating a state in the reading process, and the like.

[0079] The read state corresponding to the first scan operator can be evaluated in various ways, and the specific evaluation can be as follows:

[0080] For example, the read state of reading the first data can be recorded in the target database to obtain read state information corresponding to the first data. Based on the read state information, a target read parameter corresponding to the first scan operator is generated. The target read parameter is compared with a preset first read parameter index to obtain an evaluation result of the read state.

[0081] The target read parameter can include a parameter of a specific index, such as the number of read rows, the number of read positions, disk access efficiency, and the like. The target read parameter can also include a parameter of a comprehensive calculation index, such as a consumption or computing resource parameter calculated based on statistical information, actual row number, disk access efficiency, and the like. The target read parameter corresponding to the first scan operator can be generated in various ways based on the target read state information. For example, when the target read parameter includes a parameter of a specific index, the parameter of the specific index can be directly extracted from the read state information to obtain the target read parameter corresponding to the first scan operator. When the target read parameter includes a parameter of a comprehensive calculation index, at least one initial index parameter can be extracted from the read state information, and the initial index parameters are fused to obtain the target read parameter corresponding to the first scan operator. For example, when the target read parameter is the number of read rows, the number of read rows can be directly extracted from the read state information to obtain the target read parameter corresponding to the first scan operator. When the target read parameter is the computing consumption, the statistical information, the number of read rows, and the disk access efficiency can be extracted from the read state information, and the computing consumption is calculated based on the statistical information, the number of read rows, and the disk access efficiency to obtain the target read parameter corresponding to the first scan operator, and the like.

[0082] After generating the target read parameter corresponding to the first scan operator based on the read state information, the target read parameter can be compared with the preset first read parameter index to obtain an evaluation result of the read state. The comparison of the target read parameter with the preset first read parameter index can be in various manners. For example, taking the index type of the preset first read parameter index as a data threshold as an example, the target read parameter can be directly compared with the threshold corresponding to the preset first read parameter index. When the threshold corresponding to the preset first read parameter index is a maximum value, if the target read parameter is less than the threshold corresponding to the preset first read parameter index, it can be determined that the target read parameter reaches the preset first read parameter index. When the threshold corresponding to the preset first read parameter index is a minimum value, if the target read parameter is greater than the threshold corresponding to the preset first read parameter index, it can be determined that the target read parameter reaches the preset first read parameter index. Or, taking the preset first read parameter index as a read parameter condition as an example, the read parameter condition can be compared with the target read parameter. When the target read parameter satisfies the read parameter condition, it can be determined that the target read parameter reaches the first read parameter index. When the target read parameter does not satisfy the read parameter condition, it can be determined that the target read parameter does not reach the first read parameter index. Or, taking the preset first read parameter index as a read parameter range as an example, the read parameter range can be compared with the target read parameter. When the target read parameter falls within the read parameter range, it can be determined that the target read parameter reaches the preset first read parameter index. When the target read parameter does not fall within the read parameter range, it can be determined that the target read parameter does not reach the preset first read parameter index, and so on.

[0083] In the process of smooth switching of different scan operators, a preset read parameter index (Threshold) can be set. When the preset read parameter index is reached, the switching of the scan operator can be performed. The preset read parameter index can be a certain index, such as the actual number of rows, or a certain comprehensive calculated index, for example, the overhead based on the statistical information, the actual number of rows, and the disk access efficiency. Taking the first scan operator as an index scan, the second scan operator as a bitmap scan, and the third scan operator as a table scan as an example, the switching of the scan operator can be as shown in FIG. 1. It can be found that the preset first read parameter index (Threshold1) can be the switching point of the index scan and the bitmap scan, and another read parameter index (Threshold2) can be the switching point of the table scan and the bitmap scan. Figure 3

[0084] ​It should be noted that the preset first reading parameter index can be a pre-set reading parameter index, or can be calculated according to the data scanning request. There are various ways to calculate the reading parameter index, for example, an optimizer or optimization component can estimate or predict the reading parameter index based on the data scanning information carried in the data scanning request, and take the estimated or predicted reading parameter index as the preset first reading parameter index. In addition, if the estimated reading parameter index (Threshold) is close to the estimated scanning resource information (scanning cost) for selecting the scanning operator, it is considered that the current optimizer may have deviation in screening the operator (the overhead of the two operators is at the critical point, and the optimizer cost may be estimated inaccurately). Therefore, the optimizer or component needs to have the ability to calculate the preset reading parameter index (preset first reading parameter index and preset second reading parameter index) that needs to be switched, and determine whether the switching of the scanning operator needs to be performed by the target execution component including the evaluation component according to the preset reading parameter index.

[0085] Optionally, in some embodiments, before evaluating the reading state corresponding to the first scanning operator, the correlation coefficient between the index and the heap table can also be considered to determine whether the reading state corresponding to the first scanning operator needs to be evaluated, for example, based on the data storage location and the first data, the correlation coefficient between the data index and the heap table of the target database can be determined, and when the correlation coefficient is less than a preset correlation coefficient threshold, the reading state corresponding to the first scanning operator is evaluated.

[0086] The correlation coefficient can indicate the correlation between the data index and the heap table. Based on the data storage location and the first data, there are various ways to determine the correlation coefficient between the data index and the heap table of the target database, for example, the data storage order of the first data is identified in the heap table of the target database, the data storage location corresponding to the first data is sorted based on the data index, and the storage location order is obtained. The data storage order and the storage location order are compared to obtain the correlation coefficient between the data index and the heap table of the target database.

[0087] Optionally, in some embodiments, after determining the correlation coefficient between the data index and the heap table of the target database based on the data storage location and the first data, it can further include: when the correlation coefficient is greater than or equal to a preset correlation coefficient threshold, continuing to use the first scanning operator to read the first data corresponding to the data storage location in the target database, taking the first data as the target data corresponding to the data scanning request, and returning the target data to at least one scanning terminal corresponding to the data scanning request.

[0088] In the scheme, when the reading state corresponding to the first scan operator is evaluated, that is, the preset first reading parameter threshold is enabled, the correlation coefficient between the data index and the heap table can be considered. The lower the correlation coefficient, the more the switching of the scan operator needs to be considered. Taking the index scan with the first scan operator as an example, the index scan can be switched to bitmap scan (second scan operator).

[0089] 104、According to the evaluation result, the second data corresponding to the data storage position in the target database is read by using the second scan operator, and the first data and the second data are used as the target data corresponding to the data scan request.

[0090] The second data can be understood as the data corresponding to the data storage position read in the target database by using the second scan operator. The second scan operator can be understood as another scan operator other than the first scan operator. For example, taking the index scan with the first scan operator as an example, the second scan operator can be bitmap scan, etc.

[0091] The second data corresponding to the data storage position in the target database can be read by using the second scan operator in various ways according to the evaluation result, which can be as follows:

[0092] For example, when the target reading parameter reaches the preset first reading parameter threshold, the data storage positions of the unread data are filtered out in the data storage positions based on the first data, the remaining data storage positions are obtained, and the second data corresponding to the remaining data storage positions in the target database is read by using the second scan operator.

[0093] When the target reading parameter reaches the preset first reading parameter threshold, the data storage positions of the unread data can be filtered out in the data storage positions based on the first data in various ways, such as filtering out the data storage positions corresponding to the first data in the data storage positions, obtaining the read data storage positions, deleting the read data storage positions in the data storage positions, and obtaining the remaining data storage positions.

[0094] After the data storage positions of the unread data are filtered out in the data storage positions based on the first data, the second data corresponding to the remaining data storage positions in the target database can be read by using the second scan operator. The second data corresponding to the remaining data storage positions in the target database can be read by using the second scan operator in various ways, such as sorting the remaining data storage positions by using the second scan operator to obtain the sorted bitmap corresponding to the remaining storage positions, reading the data corresponding to the remaining storage positions in the target database based on the sorted bitmap, and obtaining the second data.

[0095] Optionally, in some embodiments, the second scan operator is adopted to read the data stored in the target database as the second data corresponding to the child according to the evaluation result can further include: when the target reading parameter reaches the preset first reading parameter index, the first scan operator is continuously adopted to read the first data corresponding to the data storage location in the target database, and the target reading parameter is updated, and when the updated target reading parameter reaches the preset second reading parameter index, the second scan operator is adopted to read the second data corresponding to the data storage location in the target database.

[0096] The preset second reading parameter index can be a reading parameter index of a next level or degree of the preset first reading parameter index. When the preset first reading parameter index is a maximum value, the preset second reading parameter index is a maximum value greater than the preset first reading parameter index. When the preset first reading parameter index is a minimum value, the preset second reading parameter index is a minimum value less than the preset first reading parameter index. When the preset first parameter index is a parameter range, the parameter range of the preset second reading parameter index is greater than the parameter range of the preset first parameter index. When the preset first parameter index is a reading parameter condition, the condition range of the reading parameter condition of the preset second reading parameter index is greater than or wider than the reading parameter condition of the preset first parameter index. The preset second reading parameter index is mainly used for the function of supplementing and switching the preset first reading parameter index. For example, taking the reading row threshold of 15 rows as the preset first reading parameter index, if the target data corresponding to the data storage location has a total of 16 rows, the scan operator is switched after 15 rows are reached, and 1 row is read after the switching, which completes the data reading. At this time, the switching has little significance, and therefore, the preset second reading parameter index can be used for switching after data accumulation. For example, the preset second reading parameter index is 30 rows, the first scan operator is continuously used for switching when 15 rows are reached, and the switching of the scan operator is performed until the 30th row, which can indicate that the switching of the scan operator has significance or is worth switching.

[0097] After the second data corresponding to the data storage location in the target database is read by the second scan operator, the first data and the second data can be used as the target data corresponding to the data scanning request.

[0098] After obtaining the target data corresponding to the data scanning request, the target data can be returned to at least one scanning terminal corresponding to the data scanning request, or the target data can be displayed.

[0099] Optionally, in some embodiments, when the target reading parameter does not reach the preset first reading parameter index, the step of reading the first data corresponding to the data storage location in the target database by using the first scan operator is returned to be executed until all the data storage locations are read, the target data corresponding to the data scanning request is obtained, and the target data is returned to the at least one scan terminal corresponding to the data scanning request.

[0100] In the process of reading the target data in the target database by using the first scan operator and the second scan operator, a target execution component (Adaptive Scan Operator, ASO) corresponding to the target database can be used for execution, and the target execution component can include an evaluation component. The target execution component usually includes three execution stages in the process of executing data scanning: an initialization stage, an execution stage, and a cleanup stage. In the initialization stage, mainly includes: generating a tid buffer (target data buffer space) for saving the ID of the tuple, avoiding the tuple from being returned to the scan terminal or the client too early, in addition, initializing the preset scan operator (subplan), so that the preset scan operator (the first scan operator and the second scan operator) is in a standby state, and finally, an evaluator (evaluation component) is established, in order to switch between the first scan operator and the second scan operator. The execution state can include reading the data in the target database by using the first scan operator and the second scan operator, and recording the execution state (reading state), then, the reading state is evaluated by the evaluation component, the evaluation here is mainly through the preset reading parameter index, if the evaluation reaches the preset reading parameter index (the preset first reading parameter index and the preset second reading parameter index), the first scan operator is smoothly switched to the second scan operator according to the execution state, for example, taking the first scan operator as index scanning and the second scan operator as bitmap scanning as an example, the index scanning can be smoothly switched to bitmap scanning. The overall flow of the initialization state and the execution state can be as shown in Figure 4 The cleanup stage can return the first data and the second data cached in the target data buffer space (tid buffer) to the at least one scan terminal corresponding to the data scanning request after the execution stage, and delete the first data and the second data in the target data buffer space, or directly delete the target data buffer space and restore the state of the subplan.

[0101] Taking an index scan as the first scan operator and a bitmap scan as the second scan operator as an example, this solution can switch the selection of scan operators (data scan operators) to the execution phase. The execution component (executor) of the target database can smoothly switch between the index scan and bitmap scan operators based on actual execution feedback. The overall architecture can be as follows: Figure 5 As shown. During the data scanning process, the overall data scanning process can be described as follows: Figure 6 As shown, the Adaptive Scan Operator uses an index scan to read data, obtaining the first data and recording the execution status (the reading status of the first data). Simultaneously, an evaluation component assesses whether the target reading parameters in the execution status corresponding to the index scan have reached a threshold (preset first and second reading parameter indicators). If the threshold is reached, the system smoothly switches to a bitmap scan to read the second data, based on the execution status. Then, the first and second data are used as the target data for the data scan request. Taking table t1 as an example, if only an index scan is used, the scan result can be as follows: Figure 7 As shown, the scan time is 18.872ms. If the adaptive switching scan operator of this scheme switches from index scan to bitmap scan during the scan process, the scan result can be as follows: Figure 8 As shown, it took 11.975ms, thus improving the efficiency of data scanning.

[0102] As can be seen from the above, after receiving at least one data scanning request, this embodiment identifies at least one data storage location in the data index of the target database based on the data scanning information carried in the data scanning request. Then, a first scanning operator is used to read the first data corresponding to the data storage location in the target database, and the reading status corresponding to the first scanning operator is evaluated. Then, based on the evaluation result, a second scanning operator is used to read the second data corresponding to the data storage location in the target database, and the first data and the second data are used as the target data corresponding to the data scanning request. Since this scheme can use the first scanning operator to read data in the target database and evaluate the reading status of the data reading, and switch the first scanning operator to the second scanning operator based on the evaluation result, the selection of scanning operators is switched to the execution stage. The switching of scanning operators is smooth based on the actual execution feedback (reading status), thereby obtaining a scanning operator with truly low overhead for data scanning, thus improving the efficiency of data scanning.

[0103] Based on the method described in the above embodiments, the following examples will provide further detailed explanations.

[0104] In this embodiment, the data scanning device is specifically integrated in an electronic device, the electronic device is a server, the first scanning operator is an index scan, the second scanning operator is a bitmap scan, and the target data buffer space is a tid buffer.

[0105] As shown in FIG. 1, a data scanning method includes the following specific processes. Figure 9

[0106] 201. The server receives at least one data scanning request.

[0107] For example, the server can directly receive at least one data scanning request sent by a scanning terminal or a client, the data scanning request carrying data scanning information, or display a data scanning page including a scanning data input control, generate a data scanning request carrying data scanning information corresponding to an input operation of the scanning data input control in response to the input operation, or receive at least one data scanning request sent by a third-party scanning platform, etc.

[0108] 202. The server identifies at least one data storage location in a data index of a target database based on the data scanning information.

[0109] For example, the server can obtain a data index of the target database, identify at least one data storage location corresponding to a data identifier, a data range identifier or a data interval identifier in the data index based on the data identifier, the data range identifier or the data interval identifier in the data scanning information, or obtain a data index of the target database, filter out index information corresponding to the data scanning information in the data index, and identify at least one data storage location in the index information, etc.

[0110] 203. The server reads first data corresponding to the data storage location in the target database by index scanning.

[0111] For example, the server can initialize a preset scanning operator corresponding to an execution component to obtain an initialized execution component, the preset scanning operator can include index scanning and bitmap scanning, construct an evaluation component corresponding to the preset scanning operator, and use the initialized execution component and the evaluation component as a target execution component, generate a tid buffer corresponding to the target execution component based on the data scanning information.

[0112] The server randomly filters out a target data storage location in the data storage location by using the target execution component, reads data corresponding to the target data storage location in the target database to obtain first data, and stores the first data to the tid buffer.

[0113] ​Optionally, in some embodiments, the server can further predict scan resource information corresponding to the data scan request based on the data scan information and the data storage location, obtain reference scan resource information of each candidate scan operator in the candidate scan operator set, compare the scan resource information with the reference scan resource information, and filter out the index scan from the candidate scan operator set based on a comparison result.

[0114] 204. The server evaluates a read state corresponding to the index scan.

[0115] For example, the server can record a read state of reading the first data in the target database to obtain read state information corresponding to the first data.

[0116] When the target read parameter includes a parameter of a certain specific index, the server can directly extract the parameter of the specific index from the read state information to obtain the target read parameter corresponding to the index scan. When the target read parameter includes a parameter of a certain comprehensive calculated index, the server can extract at least one initial index parameter from the read state information and fuse the initial index parameter to obtain the target read parameter corresponding to the index scan. For example, when the target read parameter is a read row number, the server can directly extract the read row number from the read state information to obtain the target read parameter corresponding to the index scan. For example, when the target read parameter is a calculation overhead, the server can extract statistical information, a read row number, and a disk access efficiency from the read state information, calculate the calculation overhead based on the statistical information, the read row number, and the disk access efficiency, and obtain the target read parameter corresponding to the index scan. The above is only an example.

[0117] When the index type of the preset first reading parameter index is a data threshold, the server can directly compare the target reading parameter with the threshold corresponding to the preset first reading parameter index. When the threshold corresponding to the preset first reading parameter index is a maximum value, if the target reading parameter is less than the threshold corresponding to the preset first reading parameter index, it can be determined that the target reading parameter reaches the preset first reading parameter index. When the threshold corresponding to the preset first reading parameter index is a minimum value, if the target reading parameter is greater than the threshold corresponding to the preset first reading parameter index, it can be determined that the target reading parameter reaches the preset first reading parameter index. Or, when the preset first reading parameter index is a reading parameter condition, the reading parameter condition can be compared with the target reading parameter. When the target reading parameter satisfies the reading parameter condition, it can be determined that the target reading parameter reaches the first reading parameter index. When the target reading parameter does not satisfy the reading parameter condition, it can be determined that the target reading parameter does not reach the first reading parameter index. Or, when the preset first reading parameter index is a reading parameter range, the reading parameter range can be compared with the target reading parameter. When the target reading parameter falls within the reading parameter range, it can be determined that the target reading parameter reaches the preset first reading parameter index. When the target reading parameter does not fall within the reading parameter range, it can be determined that the target reading parameter does not reach the preset first reading parameter index, and so on.

[0118] Optionally, in some embodiments, the server can also identify the data storage order of the first data in the heap table of the target database, sort the data storage positions corresponding to the first data based on the data index, obtain the storage position order, compare the data storage order and the storage position order, and thus obtain the correlation coefficient between the data index and the heap table of the target database. When the correlation coefficient is less than a preset correlation coefficient threshold, the reading state corresponding to the index scan is evaluated.

[0119] Optionally, in some embodiments, when the correlation coefficient is greater than or equal to the preset correlation coefficient threshold, the server can continue to use the index scan to read the first data corresponding to the data storage position in the target database, take the first data as the target data corresponding to the data scan request, and return the target data to the at least one scanning terminal corresponding to the data scan request.

[0120] 205、the server reads the second data corresponding to the data storage position in the target database using the bitmap scan according to the evaluation result.

[0121] For example, when the target reading parameter reaches the preset first reading parameter index, the server screens the data storage location corresponding to the first data in the data storage location, obtains the read data storage location, deletes the read data storage location in the data storage location, and thus obtains the remaining data storage location. The remaining data storage location is sorted by bitmap scanning, and the sorting bitmap corresponding to the remaining storage location is obtained. Based on the sorting bitmap, the data corresponding to the remaining storage location is read from the target database, the second data is obtained, and the second data is stored in the tid buffer.

[0122] Optionally, in some embodiments, when the target reading parameter reaches the preset first reading parameter index, the server can continue to read the first data corresponding to the data storage location in the target database by index scanning, update the target reading parameter, and when the updated target reading parameter reaches the preset second reading parameter index, read the second data corresponding to the data storage location in the target database by bitmap scanning.

[0123] Optionally, in some embodiments, when the target reading parameter does not reach the preset first reading parameter index, the server can return to execute the step of reading the first data corresponding to the data storage location in the target database by index scanning until all the data storage locations are read, obtain the target data corresponding to the data scanning request, and return the target data to the at least one scanning terminal corresponding to the data scanning request.

[0124] 206、The server returns the first data and the second data as the target data corresponding to the data scanning request.

[0125] For example, the server can directly return the first data and the second data as the target data corresponding to the data scanning request.

[0126] Optionally, in some embodiments, the server also returns the target data to the at least one scanning terminal or client corresponding to the data scanning request.

[0127] From the above, the server of the embodiment of the present application receives at least one data scanning request, identifies at least one data storage location in the data index of the target database based on the data scanning information carried in the data scanning request, then reads the first data corresponding to the data storage location in the target database by index scanning, evaluates the reading state corresponding to the index scanning, then reads the second data corresponding to the data storage location in the target database by bitmap scanning according to the evaluation result, and takes the first data and the second data as the target data corresponding to the data scanning request. Since this scheme can read data in the target database by index scanning, evaluate the reading state of data reading, switch index scanning to bitmap scanning according to the evaluation result, switch the selection of the scanning operator to the execution phase, smoothly switch the scanning operator according to the real execution feedback (reading state), and thus obtain a scanning operator with low real overhead for data scanning, the efficiency of data scanning can be improved.

[0128] To better implement the above method, the embodiment of the present application further provides a data scanning device, which can be integrated in an electronic device, such as a server or a terminal device. The terminal device can include a tablet computer, a notebook computer, and / or a personal computer, etc.

[0129] For example, as shown in Figure 10 The data scanning device can include a receiving unit 301, an identifying unit 302, an evaluating unit 303, and a reading unit 304, as follows:

[0130] (1) The receiving unit 301;

[0131] The receiving unit 301 is configured to receive at least one data scanning request, and the data scanning request carries data scanning information.

[0132] For example, the receiving unit 301 can be specifically configured to receive a data scanning request sent by at least one scanning terminal or client, and the data scanning request carries data scanning information.

[0133] (2) The identifying unit 302;

[0134] The identifying unit 302 is configured to identify at least one data storage location in the data index of the target database based on the data scanning information.

[0135] For example, the identification unit 302 can be specifically configured to obtain a data index of the target database, identify at least one data identifier, data range identifier or data interval identifier in the data scanning information, and identify at least one data storage location corresponding to the data identifier, data range identifier or data interval identifier in the data index; or, obtain a data index of the target database, filter out index information corresponding to the data scanning information in the data index, and identify at least one data storage location in the index information.

[0136] (3) the evaluation unit 303;

[0137] The evaluation unit 303 is configured to read the first data corresponding to the data storage location in the target database by using the first scan operator, and evaluate the reading state corresponding to the first scan operator.

[0138] For example, the evaluation unit 303 can be specifically configured to initialize the execution component corresponding to the target database, obtain a target execution component and a target data cache space corresponding to the target execution component, read the data corresponding to the data storage location in the target database according to the target execution component and the first scan operator, obtain the first data, add the first data to the target data cache space, record the reading state of reading the first data in the target database, obtain the reading state information corresponding to the first data, generate the target reading parameter corresponding to the first scan operator based on the reading state information, compare the target reading parameter with the preset first reading parameter index, and obtain the evaluation result of the reading state.

[0139] (4) the reading unit 304;

[0140] The reading unit 304 is configured to read the second data corresponding to the data storage location in the target database by using the second scan operator according to the evaluation result, and take the first data and the second data as the target data corresponding to the data scanning request.

[0141] For example, the reading unit 304 can be specifically configured to, when the target reading parameter reaches the preset first reading parameter index, filter out the data storage location of the unread data in the data storage location based on the first data, obtain the remaining data storage location, and read the second data corresponding to the remaining data storage location in the target database by using the second scan operator.

[0142] In specific implementation, each unit above can be implemented as an independent entity, or can be combined as the same or several entities, and the specific implementation of each unit above can be referred to the method embodiments above, which will not be described here.

[0143] As can be seen from the above, in this embodiment of the application, after the receiving unit 301 receives at least one data scanning request, the identification unit 302 identifies at least one data storage location in the data index of the target database based on the data scanning information carried in the data scanning request. Then, the evaluation unit 303 uses a first scanning operator to read the first data corresponding to the data storage location in the target database and evaluates the reading status corresponding to the first scanning operator. Then, the reading unit 304 uses a second scanning operator to read the second data corresponding to the data storage location in the target database according to the evaluation result, and uses the first data and the second data as the target data corresponding to the data scanning request. Since this scheme can use the first scanning operator to read data in the target database and evaluate the reading status of the data reading, and switch the first scanning operator to the second scanning operator according to the evaluation result, the selection of the scanning operator is switched to the execution stage. The switching of the scanning operator is smooth according to the actual execution feedback (reading status), thereby obtaining the scanning operator with the lower overhead for data scanning, thus improving the efficiency of data scanning.

[0144] This application also provides an electronic device, such as... Figure 11 As shown, it illustrates a structural schematic diagram of the electronic device involved in the embodiments of this application, specifically:

[0145] The electronic device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will understand that... Figure 11 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0146] The processor 401 is the control center of the electronic device, connecting various parts of the device via various interfaces and lines. It executes software programs and / or modules stored in the memory 402, and calls data stored in the memory 402, to perform various functions and process data. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 401.

[0147] The memory 402 can be used to store software programs and modules, and the processor 401 can execute various function applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; and the data storage area can store data created according to the use of the electronic device, etc. In addition, the memory 402 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device. Accordingly, the memory 402 can also include a memory controller to provide the processor 401 with access to the memory 402.

[0148] The electronic device also includes a power supply 403 for powering the various components. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. The power supply 403 can also include one or more direct current or alternating current power supplies, a recharging system, a power supply fault detection circuit, a power supply converter or inverter, a power supply status indicator, and the like.

[0149] The electronic device can also include an input unit 404, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls.

[0150] Although not shown, the electronic device can also include a display unit and the like, which will not be described here. Specifically, in the present embodiment, the processor 401 in the electronic device will load the executable file corresponding to the process of one or more application programs into the memory 402 according to the following instructions, and run the application programs stored in the memory 402 by the processor 401, so as to realize various functions, as follows:

[0151] receiving at least one data scanning request carrying data scanning information; identifying at least one data storage location in a data index of a target database based on the data scanning information; reading first data corresponding to the data storage location in the target database by using a first scanning operator, and evaluating a reading state of the first scanning operator; reading second data corresponding to the data storage location in the target database by using a second scanning operator according to an evaluation result, and taking the first data and the second data as target data corresponding to the data scanning request.

[0152] For example, the electronic device can receive a data scanning request sent by at least one scanning terminal or client, the data scanning request carrying data scanning information. Obtain a data index of a target database, and identify at least one data identifier, data range identifier or data interval identifier in the data scanning information, and identify at least one data storage location corresponding to the data identifier, data range identifier or data interval identifier in the data index. Alternatively, obtain a data index of a target database, filter index information corresponding to the data scanning information in the data index, and identify at least one data storage location in the index information. Initialize an execution component corresponding to the target database to obtain a target execution component and a target data cache space corresponding to the target execution component, read data corresponding to the data storage location in the target database according to the target execution component and the first scanning operator to obtain first data, add the first data to the target data cache space, record the reading state of reading the first data in the target database to obtain reading state information corresponding to the first data, generate target reading parameters corresponding to the first scanning operator based on the reading state information, compare the target reading parameters with preset first reading parameter indicators to obtain an evaluation result of the reading state. When the target reading parameters reach the preset first reading parameter indicators, filter the data storage locations of unread data in the data storage location based on the first data to obtain remaining data storage locations, and read second data corresponding to the remaining data storage locations in the target database using a second scanning operator.

[0153] The specific implementation of each operation can refer to the foregoing embodiments, which will not be repeated here.

[0154] As can be seen from the above, after receiving at least one data scanning request, the embodiments of the present application identify at least one data storage location in the data index of the target database based on the data scanning information carried in the data scanning request, then read first data corresponding to the data storage location in the target database using a first scanning operator, and evaluate the reading state corresponding to the first scanning operator. Then, according to the evaluation result, a second scanning operator is used to read second data corresponding to the data storage location in the target database, and the first data and the second data are used as target data corresponding to the data scanning request. Since this scheme can use a first scanning operator to read data in the target database and evaluate the reading state of data reading, and according to the evaluation result, the first scanning operator is switched to a second scanning operator, so that the selection of the scanning operator is switched to the execution phase, and the switching of the scanning operator is smoothly performed according to the real execution feedback (reading state), and then a scanning operator with a low real overhead is obtained for data scanning, so that the efficiency of data scanning can be improved.

[0155] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions or by controlling relevant hardware by instructions, which can be stored in a computer readable storage medium and loaded and executed by a processor.

[0156] To this end, the embodiments of the present application provide a computer readable storage medium, which stores a plurality of instructions capable of being loaded by a processor to execute the steps in any one of the data scanning methods provided by the embodiments of the present application. For example, the instructions can execute the following steps:

[0157] receiving at least one data scanning request carrying data scanning information; identifying at least one data storage location in a data index of a target database based on the data scanning information; reading first data corresponding to the data storage location in the target database by using a first scanning operator, and evaluating a reading state corresponding to the first scanning operator; reading second data corresponding to the data storage location in the target database by using a second scanning operator according to the evaluation result, and taking the first data and the second data as target data corresponding to the data scanning request.

[0158] For example, a data scanning request carrying data scanning information is received, which is sent by at least one scanning terminal or client. The data index of a target database is obtained, at least one data identifier, data range identifier or data interval identifier is obtained in the data scanning information, and at least one data storage location corresponding to the data identifier, data range identifier or data interval identifier is identified in the data index. Alternatively, the data index of the target database is obtained, index information corresponding to the data scanning information is filtered out in the data index, and at least one data storage location is identified in the index information. An execution component corresponding to the target database is initialized to obtain a target execution component and a target data cache space corresponding to the target execution component, data corresponding to the data storage location is read in the target database according to the target execution component and a first scanning operator to obtain first data, the first data is added to the target data cache space, a reading state of reading the first data is recorded in the target database to obtain reading state information corresponding to the first data, target reading parameters corresponding to the first scanning operator are generated based on the reading state information, the target reading parameters are compared with preset first reading parameter indicators to obtain an evaluation result of the reading state. When the target reading parameters reach the preset first reading parameter indicators, the data storage locations of unread data are filtered out in the data storage location based on the first data to obtain remaining data storage locations, and second data corresponding to the remaining data storage locations is read in the target database by using a second scanning operator.

[0159] The specific implementation of the above operations can refer to the foregoing embodiments, which will not be described here again.

[0160] The computer readable storage medium can include a read only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0161] Due to the instructions stored in the computer readable storage medium, the steps in any of the data scanning methods provided by the embodiments of the present application can be executed, thus the beneficial effects of any of the data scanning methods provided by the embodiments of the present application can be achieved, which will be described in detail in the foregoing embodiments, and will not be described here again.

[0162] According to an aspect of the present application, a computer program product or a computer program is provided, which includes computer instructions stored in a computer readable storage medium. A processor of an electronic device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the method provided in any of the various optional implementations of the data scanning aspect or the data reading aspect.

[0163] The above describes the data scanning method and the related device provided by the embodiments of the present application in detail, which can include a data scanning device, an electronic device, a computer program product and a computer readable storage medium. The principle and implementation manner of the present application are described by applying specific examples in this paper. The above description of the embodiments is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range will be changed. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A data scanning method, characterized by, The method comprises the following steps: receiving at least one data scanning request, the data scanning request carrying data scanning information; based on the data scanning information, identifying at least one data storage location in the data index of the target database; using a first scanning operator to read first data corresponding to the data storage location in the target database, and evaluating the reading state corresponding to the first scanning operator; according to the evaluation result, using a second scanning operator to read second data corresponding to the data storage location in the target database, and taking the first data and the second data as the target data corresponding to the data scanning request.

2. The data scanning method of claim 1, wherein, The evaluation of the reading state corresponding to the first scanning operator comprises: recording the reading state of reading the first data in the target database to obtain reading state information corresponding to the first data; based on the reading state information, generating target reading parameters corresponding to the first scanning operator; comparing the target reading parameters with preset first reading parameter indicators to obtain the evaluation result of the reading state.

3. The data scanning method of claim 2, wherein, According to the evaluation result, using a second scanning operator to read second data corresponding to the data storage location in the target database comprises: when the target reading parameters reach the preset first reading parameter indicators, based on the first data, filtering out data storage locations of unread data in the data storage location to obtain remaining data storage locations; using the second scanning operator to read second data corresponding to the remaining data storage locations in the target database.

4. The data scanning method of claim 3, wherein, Based on the first data, filtering out data storage locations of unread data in the data storage location to obtain remaining data storage locations comprises: filtering out data storage locations corresponding to the first data in the data storage location to obtain read data storage locations; deleting the read data storage locations in the data storage location to obtain the remaining data storage locations.

5. The data scanning method of claim 3, wherein, Using the second scanning operator to read second data corresponding to the remaining data storage locations in the target database comprises: using the second scanning operator to sort the remaining data storage locations to obtain a sorting bitmap corresponding to the remaining data storage locations; based on the sorting bitmap, reading data corresponding to the remaining data storage locations in the target database to obtain second data.

6. The data scanning method of claim 2, wherein, According to the evaluation result, using a second scanning operator to read second data corresponding to the data storage location in the target database comprises: when the target reading parameters reach the preset first reading parameter indicators, continue to use the first scanning operator to read first data corresponding to the data storage location in the target database, and update the target reading parameters; when the updated target reading parameters reach the preset second reading parameter indicators, use the second scanning operator to read second data corresponding to the data storage location in the target database.

7. The data scanning method of claim 2, wherein, After comparing the target reading parameters with the preset first reading parameter indicators to obtain the evaluation result of the reading state, the method further comprises the following steps: When the target reading parameter does not reach the preset first reading parameter index, returning to execute the step of reading the first data corresponding to the data storage location in the target database by using the first scanning operator until all the data storage locations are read, and obtaining the target data corresponding to the data scanning request; Returning the target data to at least one scanning terminal corresponding to the data scanning request.

8. The data scanning method of claim 1, wherein, Before the step of evaluating the reading state corresponding to the first scanning operator, the method further comprises: Based on the data storage location and the first data, determining a correlation coefficient between the data index and the heap table of the target database, the correlation coefficient indicating a correlation degree between the data index and the heap table; The step of evaluating the reading state corresponding to the first scanning operator comprises: when the correlation coefficient is less than a preset correlation coefficient threshold, evaluating the reading state corresponding to the first scanning operator.

9. The data scanning method of claim 8, wherein, After the step of determining the correlation coefficient between the data index and the heap table of the target database based on the data storage location and the first data, the method further comprises: When the correlation coefficient is greater than or equal to the preset correlation coefficient threshold, continuing to read the first data corresponding to the data storage location in the target database by using the first scanning operator; Taking the first data as the target data corresponding to the data scanning request, and returning the target data to at least one scanning terminal corresponding to the data scanning request.

10. The data scanning method of claim 1, wherein, The step of reading the first data corresponding to the data storage location in the target database by using the first scanning operator comprises: Initializing an execution component corresponding to the target database to obtain a target execution component and a target data cache space corresponding to the target execution component; According to the target execution component and the first scanning operator, reading data corresponding to the data storage location in the target database to obtain first data, and adding the first data to the target data cache space.

11. The data scanning method of claim 10, wherein, The step of initializing the execution component corresponding to the target database to obtain a target execution component and a target data cache space corresponding to the target execution component comprises: Initializing a preset scanning operator corresponding to the execution component to obtain an initialized execution component, the preset scanning operator comprising a first scanning operator and a second scanning operator; Building an evaluation component corresponding to the preset scanning operator, and taking the initialized execution component and the evaluation component as a target execution component; Based on the data scanning information, generating a target data cache space corresponding to the target execution component.

12. The data scanning method of claim 10, wherein, The step of reading data corresponding to the data storage location in the target database according to the target execution component and the first scanning operator to obtain first data comprises: Randomly screening a target data storage location in the data storage location by using the target execution component; Reading data corresponding to the target data storage location in the target database to obtain first data.

13. The data scanning method of claim 1, wherein, Before the step of reading the first data corresponding to the data storage location in the target database by using the first scanning operator, the method further comprises: predict, based on the data scan information and data storage locations, scan resource information corresponding to the data scan request; filter, according to the scan resource information, a first scan operator from a candidate scan operator set.

14. A data scanning device, characterized by The method comprises the steps of: receiving at least one data scan request, the data scan request carrying data scan information; identifying, based on the data scan information, at least one data storage location in a data index of a target database; evaluating a reading state of the first scan operator corresponding to the reading of the data storage location in the target database by the first scan operator; reading, according to the evaluation result, second data corresponding to the data storage location in the target database by a second scan operator, and taking the first data and the second data as target data corresponding to the data scan request.

15. An electronic device, comprising: The computer program / instruction is executed by the processor to implement the steps in the data scan method of any one of claims 1 to 13.

16. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instruction is executed by the processor to implement the steps in the data scan method of any one of claims 1 to 13.

17. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a plurality of instructions adapted to be loaded by the processor to execute the steps in the data scan method of any one of claims 1 to 13.

Images