A shared memory morphological data access method, system and device

Abstract

The invention proposes a shared memory morphological data access method. A main process initializes and allocates a shared memory space and clears all data in the shared memory space. Fork is called to split the main process into a child process as a foreground task process. The main process of the child process is split into a parent process. The parent process is used as a background monitoring process and is split into child processes as a foreground task process. Data of each thread is written into a data.union subunit. Waitpid is called to make the background monitoring process sleep and wait for the foreground task process to end. After the background monitoring process wakes up, the pointer base address of base_address.atomic is switched to latch all recorded data. All latched data in the shared memory space are parsed and compressed and packaged to be stored in a designated location. The foreground task process and the background monitoring process are all terminated. The two processes access data in the form of shared memory without the loss of kernel state switching. The data analysis of the background monitoring process does not affect the data writing and task execution of the foreground task process, and does not cause timing disturbance.

Description

Technical Field

[0001] The present invention relates to the field of data access technology, and in particular to a shared memory morphological data access method, system and device. Background Art

[0002] Currently, the LINUX operating system has been widely used in various embedded devices and servers, and a large number of complex high-performance software have emerged in various industries. These high-performance software have strict time requirements for data processing. In terms of hardware signals, there are real-time observation instruments such as logic analyzers and oscilloscopes to assist, but in software execution, there is actually a lack of effective real-time observation methods.

[0003] The DEBUG mode is generally used during the application development period of the LINUX operating system, which contains a large amount of printed output for observing the dynamic execution process. These outputs themselves are processed using string functions, which process information byte by byte, with slow calculation speed and low efficiency.

[0004] You can't have your cake and eat it too. Existing technologies cannot take into account both the high-performance operation and observable status of programs. Real-time programs that run at high performance may not be able to meet the speed requirements for normal operation in DEBUG mode, and the collected execution data also loses its important reference significance for real-time analysis. On the other hand, even in RELEASE mode, even if the fastest memory device is used to output string information, the delay is still at the microsecond level due to the need to switch to kernel mode to access virtual devices, which is still a thousand times slower than the actual clock speed of the processor. It is still inadequate to describe the microscopic operation status, and it also generates timing disturbances for fully real-time execution programs, which may affect the actual execution process during multi-threaded scheduling.

[0005] In view of this, it is necessary to provide a new low-energy shared memory data access method, system and device to overcome the above-mentioned defects. Summary of the invention

[0006] The purpose of the present invention is to provide a method, system and device for accessing data in the form of shared memory, which parses data into two independent processes: real-time binary tracing and non-real-time on-demand textual translation, and can trigger data analysis on demand to avoid indiscriminate waste of processor computing power by useless information; and split the main process into a foreground task process, which serves as a background monitoring process. The two processes access data in the form of shared memory, without the loss of kernel state switching, and the data analysis of the background monitoring process does not affect the data writing and task execution of the foreground task process, and will not cause timing disturbance.

[0007] In order to achieve the above object, the technical solution provided by the present invention is implemented as follows: a shared memory morphological data access method, comprising the following steps:

[0008] The main process initializes and allocates shared memory space and clears all data in the shared memory space. It calls fork to split the main process into child processes as foreground task processes. The main process of the child processes is the parent process. The parent process is the background monitoring process and writes the data of each thread into the data.union subunit.

[0009] Call waitpid to make the background monitoring process sleep and wait for the foreground task process to end. After the background monitoring process wakes up, it switches the pointer base address of base_address.atomic to latch all recorded data, parses all latched data in the shared memory space, compresses and packages them and stores them in the specified location.

[0010] The foreground task process and the background monitoring process all terminate execution.

[0011] The process of using the foreground task process to write data to each thread is as follows:

[0012] Perform an atomic increment operation on the data of the alloc_serial.atomic unit in the control and status module. The calculated result after the operation is used as the serial code to perform a logical AND operation with N-1 to obtain the write offset, and then combine it with the data in base_address.atomic to obtain the actual written record address;

[0013] Obtain the exclusive write right of the record grouping module, write the calculated sequence code into the serial subunit of the fixed-length record unit to record the number, write the ID of each thread into the belongs subunit, and keep the timestamp as 0 or register the current nanosecond timestamp;

[0014] Write the data to be recorded into the data.union subunit, and register the enumeration type of the data record in the type.enum subunit.

[0015] Determine the reason for the background monitoring process to wake up. If the external signal triggers the background monitoring process to wake up, return to call waitpid to make the background monitoring process sleep and wait for the foreground task process to end;

[0016] Otherwise, check the exit status code of the foreground task process in the control and status module. If there is no status code information, it means an abnormal crash, then the background monitoring process restores the priority and prepares to split again. If there is status code information, it means a normal exit, then the shared memory space is released, and the foreground task process and the background monitoring process all end execution.

[0017] The data analysis process using the background monitoring process is as follows:

[0018] Switch the pointer base address of the base_address.atomic unit in the control and status module, and transform another record group module into a write area, thereby locking all the data of the current record group module;

[0019] To synchronize the data completely, scan the locked record group module to mark the start and end points of the continuous interval containing the maximum sequence number; add timestamps from the start to the end point. While the timestamps are added, the data is parsed into a standardized text string according to the type requirements of the type.enum subunit.

[0020] The text string information is written into the memory disk file one by one, compressed and packaged, and named and stored according to the trigger reason and time.

[0021] On the other hand, the present invention also provides a system of a shared memory morphological data access method, comprising: a control and status module and two record grouping modules;

[0022] The control and status module includes base_address.atomic, alloc_serial.atomic and exit_status, the record grouping module includes a plurality of offset units, each offset unit includes a fixed-length record unit,

[0023] The base_address.atomic in the control and status module is the base address of the data record, the alloc_serial.atomtic in the control and status module is the total serial number of the data record, and the exit_status in the control and status module is the status code of the process exit. The reason why the parent and child processes exit execution can be inferred through the status code.

[0024] The serial subunit in the fixed-length recording unit is used to record the number, the belongs subunit in the fixed-length recording unit is used to record the thread ID, the type.enum subunit in the fixed-length recording unit is used to record the enumeration number of the type, the timestamp subunit in the fixed-length recording unit is used to record the timestamp, and the data.union subunit in the fixed-length recording unit is used to record binary data.

[0025] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the shared memory morphological data access method are implemented.

[0026] A non-transitory computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the shared memory morphological data access method.

[0027] A computer program product comprises a computer program, wherein when the computer program is executed by a processor, the steps of the shared memory morphological data access method are implemented.

[0028] Compared with the existing technology, the beneficial effect lies in that data is parsed into two independent processes, real-time binary tracing and non-real-time on-demand textual translation, which can trigger data analysis on demand and avoid indiscriminate waste of processor computing power by useless information; and the main process is split into the foreground task process and the split process as the background monitoring process. The two processes access data in the form of shared memory without the loss of kernel state switching, and nanosecond data tracking capability can be obtained. The data analysis of the background monitoring process does not affect the data writing and task execution of the foreground task process. It automatically records abnormal crashes and has built-in soft watchdog capability.

[0029] Other features and advantages of the present invention will be set forth in the following description, and part will be apparent from the description, or can be understood by implementation of the present invention. Features and advantages of the present invention can be realized and obtained by the elements and combinations specifically pointed out in the appended application scope. These and other features of the present invention will become more clearly understood according to the following description and the appended claims, or can be understood by the implementation of the embodiments described in the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0031] Figure 1 A schematic diagram of a shared memory access data system provided by the present invention.

[0032] Figure 2 The present invention provides a flow chart of a shared memory access method. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and beneficial technical effect of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be understood that the specific implementation methods described in this specification are only for explaining the present invention, not for limiting the present invention.

[0034] It should be understood that the terms "first", "second", and "third" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly indicate the number of technical features indicated. Therefore, a feature defined as "first", "second", or "third" may explicitly or implicitly include one or more of the features. In addition, the meaning of "multiple" and "several" refers to two or more, unless otherwise clearly and specifically defined.

[0035] See also Figure 1 to Figure 2 , the present invention provides a shared memory morphological data access method, including a main process splitting a child process as a foreground task process, the main process splitting the child process is the parent process, the parent process is used as a background monitoring process, the foreground task process runs and processes and writes data to each thread;

[0036] Among them, the data writing of each thread of the foreground task process, that is, the record writing can occur at any running time, and the specific process is as follows:

[0037] a1, perform an atomic increment operation on the data in the alloc_serial.atomic unit in the control and status module. The calculated result after the operation is used as the serial code to perform a logical AND operation with N-1 to obtain the write offset, and then combined with the data in the base_address.atomic unit to obtain the actual written record address;

[0038] a2, obtain the exclusive write right of a corresponding record group module, write the calculated sequence code into the serial subunit of the fixed-length record unit to record the number, write the thread ID into the belongs subunit of the fixed-length record unit (the timestamp subunit is used to record the timestamp), and keep the timestamp as 0 or register the current nanosecond timestamp (which requires the delay of consuming additional access to the device);

[0039] a3, write the data of each thread into the data.union subunit of the fixed-length record unit, and register the enumeration type of the data record in the type.enum subunit.

[0040] Among them, the specific processing process of the main process is:

[0041] b1, the main process initializes and allocates shared memory space;

[0042] b2, clear all data in the shared memory space;

[0043] b3, call fork to split the main process into a child process as the foreground task process, the main process of the child process is split into the parent process, and the parent process is used as the background monitoring process;

[0044] b4, lower the priority of the background monitoring process to avoid affecting the foreground task process during data analysis;

[0045] b5, calling waitpid to make the background monitoring process sleep and wait for the foreground task process to end. The sleep can also be interrupted by an external signal;

[0046] b6, after the background monitoring process wakes up, it immediately switches the pointer base address of base_address.atomic to latch all recorded data, and then parses all the latched data in the shared memory space and compresses and packages them to store them in the specified location;

[0047] b7, determine the reason for the background monitoring process to wake up. If it is triggered by an external signal, return to b5. Otherwise, check the foreground task process exit status code in the control and status module. If there is no status code information, it means an abnormal crash, then continue to b8. If there is a status code information, it means a normal exit, then continue to b9.

[0048] b8, the monitoring process restores the priority and prepares to split again, returning to b2, and also serves as a watchdog;

[0049] b9, release the shared memory, and both the foreground task process and the background monitoring process terminate execution.

[0050] Among them, if the external signal is triggered or the foreground task process exits (that is, it only occurs when the external signal is triggered or the foreground task process exits), the background monitoring process performs data analysis and record reading. The specific processing process is as follows:

[0051] c1, switch the pointer base address of base_address.atomic in the control and status module, and transform another record group module into a write area, thereby locking all the data of the current record group module;

[0052] c2, wait for several milliseconds for the data to be fully synchronized;

[0053] c3, scan the locked record grouping module and mark the starting point and end point of the continuous interval containing the maximum sequence number;

[0054] c4, add timestamps from the start point to the end point, that is, if the timestamp of a record is 0, its value is the timestamp of the previous record, that is, the timestamp is also 0. This is only to preserve the order in various analysis and comparison operations of data records;

[0055] c5, while completing the timestamp, the data is parsed into a standardized text string according to the type requirements of the type.enum subunit. The text string includes but is not limited to the record number, timestamp, thread ID and other entry information that can be sorted, retrieved or sorted;

[0056] c6, the text string information is written into the memory disk file of the morphological access data system one by one, compressed and packaged, and then named and stored according to the trigger reason and time.

[0057] The external signal that the background monitoring process can receive in this method refers to the inter-process signal delivery mechanism in the Linux system, which can usually be sent by the operator through the kill command. The background monitoring process can capture specific signals and immediately start parsing the current data record when triggered by the external signal. If the foreground task process crashes unexpectedly, the background monitoring process will also automatically parse the last execution status data and restart the foreground task process.

[0058] The present invention also provides a shared memory morphological access data system, comprising: a control and status module and two record grouping modules (respectively, record grouping module 0 and record grouping module 1);

[0059] The control and status module includes base_address.atomic, alloc_serial.atomic and exit_status, the record grouping module includes a plurality of offset units (the plurality of offset units are offset 0 unit, offset 1 unit ... offset N unit), each offset unit includes a fixed-length record unit,

[0060] The base_address in the control and status module is the base address of the data record, and the switching is to point to a location in one of the record group modules or another record group module as the location where the actual data is written, while the other units in the remaining record group modules are in a locked state that will not be written;

[0061] The alloc_serial.atomtic in the control and status module is the total serial number of the data record, and the exit_status unit in the control and status module is the status code of the process exit, through which the reason why the child process exited execution can be inferred: if it is an active exit, there will be an exit status code information, otherwise it is a passive result caused by process abnormality.

[0062] Among them, the serial subunit in the fixed-length record unit is used to record the number, the belongs subunit in the fixed-length record unit is used to record the thread ID, the type.enum subunit in the fixed-length record unit is used to record the enumeration number of the type, the timestamp subunit in the fixed-length record unit is the timestamp, and the data.union subunit in the fixed-length record unit is used to record binary data.

[0063] The present invention also provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the shared memory morphological data access method when executing the program.

[0064] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the shared memory morphological data access method are implemented.

[0065] The present invention also provides a computer program product, including a computer program, which implements the steps of the shared memory morphological data access method when executed by a processor.

[0066] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0067] In particular, according to some embodiments of the present disclosure, the process described above can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processing device, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0068] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a mission data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such a propagated mission data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0069] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperTextTransferProtocol), and may be interconnected with any form or medium of digital task data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an adhoc peer-to-peer network), as well as any currently known or future developed network.

[0070] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: in response to detecting a query operation on a production collaboration document in a switch production line management application, determines the network connection status of the switch production line management application; in response to determining that the network connection status of the switch production line management application represents an offline state, replaces the web page entry information corresponding to the production collaboration document with target entry file information, and loads target web page resource information to display the web page of the production collaboration document offline in the switch production line management application, wherein the target entry file information is file information of a pre-downloaded entry file corresponding to the web page of the production collaboration document, and the target web page resource information is locally stored resource information corresponding to the web page; in response to determining that the network connection status of the switch production line management application represents an online state, and the web page resource information corresponding to the production collaboration document is not stored locally, downloads the web page resource information of the web page from the production line document server, wherein the web page resource information includes an entry file and resource information; according to the web page resource information, displays the web page of the production collaboration document in the switch production line management application, and stores the web page resource information in a local database.

[0071] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including product-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0072] The present invention is not limited to what is described in the specification and implementation modes, and therefore additional advantages and modifications can be easily realized by those skilled in the art. Therefore, without departing from the spirit and scope of the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details, representative devices, and illustrative examples shown and described herein.

Claims

1. A shared memory morphological data access method, characterized in that: The following steps are involved: The main process initializes and allocates shared memory space and clears all data in the shared memory space. The shared memory space includes a control and status module and two record grouping modules. Fork is called to split the main process into a child process as the foreground task process. The main process that splits the child process is the parent process, and the parent process is used as the background monitoring process. The foreground task process writes each thread data into the data.union subunit of one of the two record grouping modules. After the background monitoring process lowers its priority, it calls waitpid to put the background monitoring process to sleep and wait for the foreground task process to end. After the background monitoring process wakes up, it switches the pointer base address of base_address.atomic in the control and status module, converts another record grouping module into a write area, thereby locking all the data of the current record grouping module, parsing all the data latched in the shared memory space, compressing and packaging them and storing them in the specified location. The foreground task process and the background monitoring process all terminate execution.

2. The shared memory morphological data access method according to claim 1, characterized in that: The process of using the foreground task process to write data to each thread is as follows: Perform an atomic increment operation on the data of the alloc_serial.atomic unit in the control and status module. The calculated result after the operation is used as the serial code for logical AND operation to obtain the write offset, which is combined with the data in base_address.atomic to obtain the actual written record address; Obtain the exclusive write right of the record grouping module, write the calculated sequence code into the serial subunit of the fixed-length record unit to record the number, write the ID of each thread into the belongs subunit, and keep the timestamp as 0 or register the current nanosecond timestamp; Write the data to be recorded into the data.union subunit, and register the enumeration type of the data record in the type.enum subunit.

3. The shared memory morphological data access method according to claim 1, characterized in that: Determine the reason for the background monitoring process to wake up. If the external signal triggers the background monitoring process to wake up, return to call waitpid to make the background monitoring process sleep and wait for the foreground task process to end; Otherwise, check the exit status code of the foreground task process in the control and status module. If there is no status code information, it means an abnormal crash, then the background monitoring process restores the priority and prepares to split again. If there is status code information, it means a normal exit, then the shared memory space is released, and the foreground task process and the background monitoring process all end execution.

4. The shared memory morphological data access method according to claim 1, characterized in that: To synchronize the data completely, scan the locked record group module to mark the start and end points of the continuous interval containing the maximum sequence number; add timestamps from the start to the end point. While the timestamps are added, the data is parsed into a standardized text string according to the type requirements of the type.enum subunit. The text string information is written into the memory disk file one by one, compressed and packaged, and named and stored according to the trigger reason and time.

5. A system for accessing data in a shared memory according to any one of claims 1 to 4, characterized in that: The control and status module includes base_address.atomic, alloc_serial.atomic and exit_status, the record grouping module includes a plurality of offset units, each offset unit includes a fixed-length record unit, The base_address.atomic in the control and status module is the base address of the data record, the alloc_serial.atomtic in the control and status module is the total serial number of the data record, and the exit_status in the control and status module is the status code of the process exit. The reason why the parent and child processes exit execution can be inferred through the status code.

6. The system of the shared memory morphological data access method as claimed in claim 5, characterized in that: The serial subunit in the fixed-length recording unit is used to record the number, the belongs subunit in the fixed-length recording unit is used to record the thread ID, the type.enum subunit in the fixed-length recording unit is used to record the enumeration number of the type, the timestamp subunit in the fixed-length recording unit is used to record the timestamp, and the data.union subunit in the fixed-length recording unit is used to record binary data.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the shared memory morphological data access method as described in any one of claims 1 to 4 are implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the shared memory morphological data access method as described in any one of claims 1 to 4 are implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the shared memory morphological data access method as described in any one of claims 1 to 4 are implemented.

Images