Method, system, device and medium for identifying data blocks in a data table

Abstract

The present disclosure provides a data block identification method, system, device and medium in a data table, relating to the field of data identification, and the identification comprises: querying valid data bits of a data table to be processed in a preset direction; when the first valid data bit is queried, determining that the valid data bit is data of a current data field and pressing into a detection stack; the position to be detected is popped out from the detection stack in turn, and whether the adjacent position of the valid data bit has a new valid data bit is judged, if yes, the boundary range of the current data field is expanded according to the adjacent position with the new valid data bit; all data positions on the boundary range of the current data field which have not been verified are pressed into the detection stack, and then the above steps are returned until the detection stack is empty, and the current data field is output. The present disclosure can accurately obtain the data field even in the case of serious data loss, and solves the problem of low efficiency of manual operation.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of data recognition, and in particular to a data block recognition method, system, device and medium in a data table. BACKGROUND

[0002] At present, the data warehouse based on the Hadoop framework of the big data system derives many data visualization frameworks and schemes, such as the open source frameworks Grafana and SuperSet. These frameworks can be well compatible with the big data framework system, based on the underlying database, and through simple configuration, a good data visualization effect can be obtained. However, the open source framework must be deployed before use, and the data migration is poor. In many application scenarios, such as industry research reports, academic sharing reports and government reports, PPT, an Office tool, is often used for data display, and the popular open source visualization tool cannot assume this role.

[0003] The technical solution to solve the current problem is to develop an automatic Office report making system based on the data warehouse, which periodically generates PPT reports according to templates. The current situation is that analysts are overwhelmed with a large amount of repetitive work, and once the data of the report is updated over time, this work is also long-term and complex.

[0004] Therefore, it is very important to access Office to the data warehouse as part of data visualization display, but the data blocks in the data table and the data blocks of the source data of the graph or table are difficult to match, resulting in that the updated data cannot be accurately inserted into the updated graph or table, and manual operation also leads to low work efficiency. In addition, the connected domain algorithm based on data matrix is mostly applied in image recognition direction. There are two mainstream algorithms at present, Two-Pass algorithm and Seed-Filling algorithm. These two algorithms can be effectively applied to ordinary pixel point connected domain analysis, but are not suitable for the current application direction. SUMMARY

[0005] The technical problem to be solved by the present disclosure is to overcome the defects that the data blocks in the data table cannot be accurately and effectively recognized and manual operation is inefficient in the prior art, and to provide a data block recognition method, system, device and medium in a data table.

[0006] The present disclosure solves the above technical problems by the following technical solutions:

[0007] In a first aspect, a data block recognition method in a data table is provided, the method comprising:

[0008] querying valid data bits in a preset direction for a data table to be processed;

[0009] When the first valid data bit is found, it is determined that the valid data bit is data of a current data field, and the valid data bit is pushed into a to-be-detected stack;

[0010] The to-be-detected stack is popped to detect positions one by one, and it is determined whether a neighboring position of the valid data bit has a new valid data bit, if yes, a boundary range of the current data field is expanded according to the neighboring position having the new valid data bit;

[0011] All data positions of the boundary range of the current data field that have not been verified are pushed into the to-be-detected stack, and then the step of popping the to-be-detected stack to detect positions one by one is returned to, until the to-be-detected stack is empty, and the current data field is output.

[0012] Preferably, the step of outputting the current data field further includes: continuing to take the data table after the current data field as a data table to be processed, and then returning to the step of searching for a valid data bit in a preset direction from the data table to be processed, to continue searching for a new valid data bit, until all new valid data bits are found, and the process is ended.

[0013] Preferably, the step of searching for a valid data bit in a preset direction from the data table to be processed specifically includes: searching for a valid data bit from left to right by row from a starting position of the data table to be processed.

[0014] Preferably, in the step of determining whether a neighboring position of the valid data bit has a new valid data bit, if a coordinate of the valid data bit is (x, y), coordinates of the neighboring position of the valid data bit are (x-1, y), (x+1, y), (x-1, y+1), (x, y+1) and (x+1, y+1) respectively.

[0015] Preferably, the step of expanding the boundary range of the current data field according to the neighboring position having the new valid data bit specifically includes: comparing horizontal and vertical coordinate values of the neighboring position having the new valid data bit with minimum and maximum horizontal and vertical coordinate values of the current data field respectively, and taking minimum and maximum values in sequence, and expanding the minimum and maximum values as boundary coordinates of the current data field.

[0016] In a second aspect, a system for identifying a data block in a data table is provided, and the system includes:

[0017] A searching module is configured to search for a valid data bit in a preset direction from a data table to be processed;

[0018] A pushing module is configured to, when the first valid data bit is found, determine that the valid data bit is data of a current data field, and push the valid data bit into a to-be-detected stack;

[0019] an expanding module, configured to pop out the position to be detected from the stack in sequence, and determine whether the adjacent position of the valid data bit has a new valid data bit, and if so, expand the boundary range of the current data field according to the adjacent position having the new valid data bit;

[0020] an output module, configured to push all the data positions not verified on the boundary range of the current data field into the stack, and then call the expanding module until the stack is empty, and output the current data field.

[0021] Preferably, the output module is specifically configured to continue the data table after the current data field as a data table to be processed, and then call the query module to continue to query the new valid data bit until all the new valid data bits are queried.

[0022] Preferably, the query module is specifically configured to query the valid data bit from left to right by row from the starting position of the data table to be processed.

[0023] Preferably, if the coordinate of the valid data bit is (x, y), the coordinates of the adjacent position of the valid data bit are (x-1, y), (x+1, y), (x-1, y+1), (x, y+1) and (x+1, y+1) respectively.

[0024] Preferably, the expanding module is specifically configured to compare the horizontal and vertical coordinate values of the adjacent position having the new valid data bit with the minimum and maximum horizontal and vertical coordinate values of the current data field respectively, and sequentially take the minimum value and the maximum value, and expand the minimum value and the maximum value as the boundary coordinates of the current data field.

[0025] In a third aspect, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the data block identification method in the data table when executing the computer program.

[0026] In a fourth aspect, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program, and the computer program is executable on a processor to implement the data block identification method in the data table.

[0027] The positive progress effect of the present disclosure is that the method provided by the present disclosure can effectively identify the data block in the data table, accurately obtain the data field even in the case of serious data missing, and eliminate the inefficient operation of manual observation, thereby ensuring the stable update of the data block in the Office to the data warehouse, and providing a foundation for realizing the function of taking the Office tool as a visual tool of the data warehouse. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 A flowchart of a data block identification method in a data table provided for Embodiment 1 of the present disclosure;

[0029] Figure 2 A neighbor detection position map in a data block identification method in a data table provided for Embodiment 1 of the present disclosure.

[0030] Figure 3 An effect diagram of a data block identification method in a data table provided for Embodiment 1 of the present disclosure.

[0031] Figure 4 An effect diagram of a traditional connected component algorithm.

[0032] Figure 5 A first effect diagram of an application example of a data block identification method in a data table provided for Embodiment 1 of the present disclosure.

[0033] Figure 6 A second effect diagram of an application example of a data block identification method in a data table provided for Embodiment 1 of the present disclosure.

[0034] Figure 7 A third effect diagram of an application example of a data block identification method in a data table provided for Embodiment 1 of the present disclosure.

[0035] Figure 8 A fourth effect diagram of an application example of a data block identification method in a data table provided for Embodiment 1 of the present disclosure.

[0036] Figure 9 A first effect diagram of another application example of a data block identification method in a data table provided for Embodiment 1 of the present disclosure.

[0037] Figure 10 A second effect diagram of another application example of a data block identification method in a data table provided for Embodiment 1 of the present disclosure.

[0038] Figure 11 A third effect diagram of another application example of a data block identification method in a data table provided for Embodiment 1 of the present disclosure.

[0039] Figure 12 A fourth effect diagram of another application example of a data block identification method in a data table provided for Embodiment 1 of the present disclosure.

[0040] Figure 13 A fifth effect diagram of another application example of a data block identification method in a data table provided for Embodiment 1 of the present disclosure.

[0041] Figure 14 Figure 6 is a schematic diagram of a sixth effect of another application example of the method for identifying a data block in a data table provided in Embodiment 1 of the present disclosure.

[0042] Figure 15 Figure 1 is a schematic diagram of a structure of a system for identifying a data block in a data table provided in Embodiment 2 of the present disclosure.

[0043] Figure 16 Figure 4 is a schematic diagram of a structure of an electronic device provided in Embodiment 3 of the present disclosure. DETAILED DESCRIPTION

[0044] The present disclosure is further illustrated by way of examples below, but the present disclosure is not limited in the scope of the examples.

[0045] Embodiment 1

[0046] The present embodiment provides a method for identifying a data block in a data table, Figure 1 Figure 3 is a schematic diagram of a flow of the method for identifying a data block in a data table provided in Embodiment 1 of the present disclosure, as Figure 1 The method comprises:

[0047] Step 101: Querying valid data bits in the data table to be processed in a preset direction.

[0048] In this step 101, valid data bits are queried from left to right by row from the starting position of the data table to be processed.

[0049] The position of a cell with data in the data table is referred to as a valid data bit.

[0050] Step 102: When the first valid data bit is queried, determining that the valid data bit is data of a current data field, and pushing the valid data bit into a detection stack.

[0051] Step 103: The position to be detected is popped out from the detection stack in sequence, and it is determined whether the adjacent position of the valid data bit has a new valid data bit, if yes, expanding the boundary range of the current data field according to the adjacent position with the new valid data bit.

[0052] In expanding the boundary range of the data field, the position of the adjacent position needs to be located. Different from the four-adjacent-position search method and the eight-adjacent-position search method in the traditional algorithm, the five-adjacent-position search method is selected in the present embodiment. If the coordinates of the valid data bit are (x, y), the coordinates of the adjacent position of the valid data bit are (x-1, y), (x+1, y), (x-1, y+1), (x, y+1) and (x+1, y+1) respectively. As Figure 2As shown, the black position in the figure is the position to be detected, and the gray part is five positions to be detected of the position to be detected. If the detected position has valid data bits, the coordinate of the position is used to expand the boundary range of the current data field.

[0053] In this step 103, the horizontal and vertical coordinate values of the position with new valid data bits are compared with the minimum and maximum horizontal and vertical coordinate values of the current data field, respectively, and the minimum and maximum values are taken in turn, and the minimum and maximum values are expanded as the boundary coordinates of the current data field.

[0054] In this step 103, the horizontal and vertical coordinate values of the position with new valid data bits are compared with the minimum and maximum horizontal and vertical coordinate values of the current data field, respectively, and the minimum and maximum values are taken in turn, and the minimum and maximum values are expanded as the boundary coordinates of the current data field.

[0055] Step 104, all unverified data positions on the boundary range of the current data field are pushed into the stack to be detected, and then step 103 is returned until the stack to be detected is empty, and the current data field is output.

[0056] In this step 103, the horizontal and vertical coordinate values of the position with new valid data bits are compared with the minimum and maximum horizontal and vertical coordinate values of the current data field, respectively, and the minimum and maximum values are taken in turn, and the minimum and maximum values are expanded as the boundary coordinates of the current data field.

[0057] In the specific implementation, the above step 104 further includes:

[0058] Step 105, the data table after the current data field is removed continues to be used as the data table to be processed, and then step 101 is returned to continue to query new valid data bits until all new valid data bits are queried, and the process is ended.

[0059] In this step 103, the horizontal and vertical coordinate values of the position with new valid data bits are compared with the minimum and maximum horizontal and vertical coordinate values of the current data field, respectively, and the minimum and maximum values are taken in turn, and the minimum and maximum values are expanded as the boundary coordinates of the current data field.

[0060] In this embodiment, as shown in FIG. 1, the data table is divided into a plurality of data fields, and the data fields are arranged in a matrix form. Figure 3As shown in the figure, the gray block area is the data block obtained by the method, wherein the (4, 6) position (i.e., the horizontal coordinate value of the position is 4, and the vertical coordinate value is 6, and the same below) is determined as the largest data block on the left side of the data table, which obviously meets the expectation and the conventional use of Excel, and all the data blocks can be accurately distinguished even in the case of serious missing data in the entire data block. Figure 4 The effect diagram obtained by using the traditional connected domain exploration method. Since the eight adjacent positions of the (4, 6) position do not have valid data, the (4, 6) position will be determined as a data block by the traditional algorithm, which does not meet the expectation.

[0061] In the embodiment, the data table is traversed from the (1, 1) position. Figure 5 As can be seen, in the 5*5 matrix, only the data on the diagonal line exists, and most of the data is missing. Among them, the light gray position is defined as the starting search point of the data domain, the black position is the valid data bit detected in the last round, which is used to expand the boundary range of the overall data domain, and the dark gray position is the newly appeared detection point after the overall data domain range is expanded. These newly appeared detection points will also enter the to-be-detected stack and wait to be detected. In the Figure 5 , the starting position (1, 1) of the data domain is detected first, then the adjacent positions of (1, 1) are detected, and the (2, 2) position is detected to have a valid data bit. The (1, 1) to (2, 2) are set as the valid region of the current data domain (i.e., the minimum coordinate value in the region is (1, 1), and the maximum coordinate value is (2, 2)), and the dark gray region, i.e., the position not detected in the current data domain, is pressed into the to-be-detected stack for detection, so as to advance, as shown in Figure 6 , Figure 7 and Figure 8 , the entire data domain can be successfully obtained.

[0062] In the embodiment, as shown in Figure 9 , the light gray part (1, 1) position is the starting position of the data table, which is detected from the current position. The (2, 2) position in the five adjacent positions of the position is detected to have valid data, so the (2, 2) position is expanded as the boundary coordinate of the current data domain. Similarly, the (3, 3) position is expanded, Figure 10 , the (2, 4) position is expanded. Figure 11 In Figure 12 , when the (3, 4) position is detected, it is observed that the (4, 5) and (5, 4) two valid data positions are not in the five adjacent positions of any other valid data bit. However, when the (1, 5) position is determined as a valid data bit, the y value of the boundary coordinate of the current data domain will be expanded to 5. At this time, the null data point of the dark gray position will also be pressed into the to-be-detected stack. When the (3, 5) position is detected, the (4, 5) position will be expanded into the current data domain, as shown in Figure 13 . Similarly, as shown inFigure 14 As shown, when the position (5, 4) is detected, the x value of the boundary coordinate of the current data field is expanded to 5, at this time, the area in the data field is detected as a 5x5 data matrix, at this time, all the valid data bits in the current data field have been detected, and the current data field is output. Figures 9 to 14 The advantages of the present disclosure over conventional algorithms are explained, and how the present disclosure identifies valid data bits of non-adjacent bits into data blocks is described in detail.

[0063] The data block identification method of the data table of the embodiment replaces the manual operation method, and can accurately distinguish the data regions of each data block even in the case of serious data loss in the data table. The connectivity algorithm is innovatively modified to make the program more suitable for user use, eliminate the inefficient operation of manual observation, and ensure stable updating of the data blocks in the Office to the data warehouse, thereby providing a foundation for realizing the function of the Office tool as a visual tool for the data warehouse.

[0064] Embodiment 2

[0065] The embodiment provides a data block identification system in a data table, which can implement the data block identification method in the data table in embodiment 1, Figure 15 A structural schematic diagram of the data block identification system in the data table provided by the embodiment is shown in Figure 15 As shown, the system comprises:

[0066] The query module 1 is configured to query valid data bits in the data table to be processed in a preset direction;

[0067] The stack module 2 is configured to determine the valid data bit as the data of the current data field when the first valid data bit is queried, and press the valid data bit into the detection stack;

[0068] The expansion module 3 is configured to pop the position to be detected from the detection stack in sequence, and determine whether the adjacent position of the valid data bit has a new valid data bit, if yes, expand the boundary range of the current data field according to the adjacent position having the new valid data bit;

[0069] The output module 4 is configured to press all the data positions on the boundary range of the current data field and not verified into the detection stack, and then call the expansion module 3, until the detection stack is empty, and output the current data field.

[0070] In the specific implementation, the output module 4 is specifically configured to continue the data table after the current data field as the data table to be processed, and then call the query module 1 to continue to query the new valid data bit, until all the new valid data bits are queried.

[0071] In specific implementation, the query module 1 is specifically used to query valid data bits starting from the starting position of the data table to be processed from left to right by row.

[0072] In a specific implementation, if the coordinates of the valid data bit are (x, y), the coordinates of the adjacent bits of the valid data bit are (x-1, y), (x+1, y), (x-1, y+1), (x, y+1) and (x+1, y+1).

[0073] In specific implementation, the expansion module 3 is specifically used to compare the horizontal and vertical coordinate values ​​of the adjacent bits with the new valid data bits with the minimum and maximum horizontal and vertical coordinate values ​​of the current data domain, take the minimum and maximum values ​​in turn, and expand the minimum and maximum values ​​to the boundary coordinates of the current data domain.

[0074] The data block recognition system in the data table of this embodiment, through the mutual cooperation between various modules, can accurately distinguish the data area of ​​each data block even when there is a serious lack of data in the data table, solve the problem of inefficiency of manual operation, ensure the stable update of data blocks in Office to the data warehouse, and lay the foundation for realizing the function of using Office tools as a visualization tool for data warehouses.

[0075] Example 3

[0076] This embodiment provides an electronic device, Figure 16 This is a structural diagram of an electronic device provided in this embodiment. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for identifying data blocks in the data table in the above-mentioned embodiment 1 is implemented. Figure 16 The electronic device 30 shown is only an example and should not limit the functionality and scope of use of the embodiments of the present disclosure. Figure 16 As shown, the electronic device 30 may be a general-purpose computing device, such as a server device. Components of the electronic device 30 may include, but are not limited to, the at least one processor 31, the at least one memory 32, and a bus 33 connecting different system components (including the memory 32 and the processor 31).

[0077] The bus 33 includes a data bus, an address bus, and a control bus.

[0078] The memory 32 may include a volatile memory, such as a random access memory (RAM) 321 and / or a cache memory 322 , and may further include a read-only memory (RAM) 323 .

[0079] The memory 32 can also include a program tool 325 (or utility) having a set (at least one) of program modules 324, such as an operating system, one or more application programs, other program modules, and program data, and each of these examples, or some combination thereof, can include implementation of a network environment.

[0080] The processor 31 performs various function applications and data processing by running the computer program stored in the memory 32, such as the data block identification method in the data table in the above embodiment 1.

[0081] The electronic device 30 can also communicate with one or more external devices 34. Such communication can occur via the Input / Output (I / O) interface 35. Still yet, such electronic device 30 can communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or the public network, such as the Internet) via the network adapter 36. As Figure 16 illustrated, the network adapter 36 communicates with the other modules of the electronic device 30 through the bus 33. It should be appreciated that although not shown, other hardware and / or software modules could be used in conjunction with the electronic device 30. Such as, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0082] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, such division is merely exemplary and not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into embodied by multiple units / modules.

[0083] Embodiment 4

[0084] The present embodiment provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the data block identification method in the data table in the above embodiment 1.

[0085] Among them, the more specific readable storage medium that can be adopted can include but is not limited to: portable disk, hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical storage device, magnetic storage device or any suitable combination of the above.

[0086] In possible implementation manners, the present disclosure can also be implemented in the form of a program product, which includes program codes for causing a terminal device to perform steps in the method for identifying data blocks in a data table in the above-mentioned embodiment 1 when the program product is run on the terminal device.

[0087] Wherein, the program codes for executing the present disclosure can be written in any combination of one or more programming languages, and can be executed entirely on the user device, partially on the user device, as a separate software package, partially on the user device and partially on a remote device, or entirely on a remote device.

[0088] Although the specific implementation manners of the present disclosure are described above, those skilled in the art should understand that this is only an illustration, and the protection scope of the present disclosure is defined by the appended claims. Those skilled in the art can make various changes or modifications to these implementation manners without departing from the principles and essence of the present disclosure, and these changes and modifications all fall within the protection scope of the present disclosure.

Claims

1. A method for identifying data blocks in a data table, characterized in that: The method comprises: Query the valid data bits of the data table to be processed according to the preset direction; When the first valid data bit is found, the valid data bit is determined to be data in the current data field, and the valid data bit is pushed into the to-be-detected stack; Popping out positions to be tested from the to-be-tested stack in sequence, and determining whether an adjacent position of the valid data bit has a new valid data bit, wherein if the coordinates of the valid data bit are (x, y), then the coordinates of the adjacent positions of the valid data bit are (x-1, y), (x+1, y), (x-1, y+1), (x, y+1), and (x+1, y+1); If the adjacent bit of the valid data bit has a new valid data bit, then the boundary range of the current data domain is expanded according to the adjacent bit with the new valid data bit; specifically, the method includes: comparing the horizontal and vertical coordinate values ​​of the adjacent bit with the minimum and maximum horizontal and vertical coordinate values ​​of the current data domain, taking the minimum value and the maximum value in turn, and expanding the minimum value and the maximum value as the boundary coordinates of the current data domain; Push all unverified data positions on the boundary range of the current data domain into the to-be-detected stack, then return to the step of popping out the to-be-detected positions in sequence through the to-be-detected stack until the to-be-detected stack is empty, and then output the current data domain.

2. The method for identifying data blocks in a data table according to claim 1, wherein: After the step of outputting the current data field, the method includes: continuing to use the data table after removing the current data field as the data table to be processed, then returning to the step of searching for valid data bits in a preset direction in the data table to be processed, continuing to search for new valid data bits until all new valid data bits have been found, and then terminating the process; And / or, the step of searching the data table to be processed for valid data bits in a preset direction specifically includes: searching for valid data bits starting from the starting position of the data table to be processed from left to right by row.

3. A system for identifying data blocks in a data table, characterized in that: The system comprises: A query module, used to query the valid data bits of the data table to be processed according to a preset direction; A stacking module is used to determine that the first valid data bit is data of the current data field when the first valid data bit is found, and to push the valid data bit into the stack to be detected; an expansion module, configured to pop out positions to be detected in sequence from the to-be-detected stack, and determine whether an adjacent position of the valid data bit has a new valid data bit, wherein if the coordinates of the valid data bit are (x, y), the coordinates of the adjacent positions of the valid data bit are (x-1, y), (x+1, y), (x-1, y+1), (x, y+1), and (x+1, y+1), respectively; if the adjacent position of the valid data bit has a new valid data bit, then the boundary range of the current data domain is expanded according to the adjacent position with the new valid data bit; The expansion module is specifically used to compare the horizontal and vertical coordinate values ​​of the adjacent bit with the new valid data bit with the minimum and maximum horizontal and vertical coordinate values ​​of the current data domain, take the minimum value and the maximum value in turn, and expand the minimum value and the maximum value as the boundary coordinates of the current data domain; The output module is used to push all unverified data positions on the boundary range of the current data domain into the stack to be detected, and then call the expansion module until the stack to be detected is empty, and then output the current data domain.

4. The system for identifying data blocks in a data table according to claim 3, wherein: The output module is specifically used to continue to use the data table after removing the current data field as the data table to be processed, and then call the query module to continue to query new valid data bits until all new valid data bits have been queried; And / or, the query module is specifically configured to query valid data bits starting from a starting position of the data table to be processed from left to right by row.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for identifying a data block in the data table according to any one of claims 1 to 2 is implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for identifying a data block in a data table according to any one of claims 1 to 2 is implemented.

Images