Digital government affair management method and system based on big data

Abstract

The invention discloses a digital government affair management method and system based on big data, and the method comprises the steps: receiving submitted government affair items in real time, and transmitting the government affair items to a first message queue; dequeuing the government affair items in the first message queue in sequence, persistently storing all the government affair items in a database in a coroutine manner, and distributing the government affair items to corresponding processing units; after processing of the processing units corresponding to the government affair items is completed, the processing results are returned to the second message queue for merging into the queue; the processing results in the second message queue are dequeued in sequence, all the processing results are persistently stored in the database in a coroutine mode, connection with the corresponding government affair items is established in a key mode except for the corresponding government affair items, and meanwhile the processing results are transmitted back to the submission position of the corresponding government affair items. The digital government affair management method and the digital government affair management system can meet the requirement of high availability in a high-concurrency scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of big data processing, and particularly relates to a digital government affair management method and system based on big data. BACKGROUND

[0002] The current government affair processing and management process of units at all levels is carried out in a digital manner, and a digital government affair system has been widely used. The digital system has the advantages of greatly improving the government affair processing efficiency, reducing human intervention and paper file circulation, breaking department barriers, realizing cross-level and cross-department collaboration, improving government affair transparency, reducing government affair processing links and time limit, and reducing government affair management cost through preset standardized processes and automatic processing.

[0003] However, the existing digital government affair management system will be blocked when facing a large number of concurrent accesses during the use peak, which will cause system overload and response delay or even system crash. In addition, the use of synchronous data processing mode in the digital government affair management system will reduce the processing efficiency of the system, which is not conducive to meeting the high availability requirement of the system in a high-concurrency scenario. SUMMARY

[0004] The purpose of the present application is to provide a digital government affair management method and system based on big data, which aims to solve the problem that the existing digital government affair management system will be blocked when facing a large number of concurrent accesses during the use peak, which will cause system overload and response delay or even system crash, and the use of synchronous data processing mode in the digital government affair management system will reduce the processing efficiency of the system, which is not conducive to meeting the high availability requirement of the system in a high-concurrency scenario.

[0005] In view of the above problems, the present application provides a digital government affair management method and system based on big data.

[0006] The first aspect of the present application provides a digital government affair management method based on big data, which comprises the following steps: Step 1: Real-time receiving of submitted government affairs, sending to the first message queue and enqueuing; Step 2: Dequeue the government affairs in the first message queue one by one, and store all the government affairs in the database in the form of coroutines, and distribute them to the corresponding processing unit; Step 3: After the corresponding processing unit of each government affair is processed, the processing result is returned to the second message queue and enqueued; Step 4: The processing results in the second message queue are dequeued in turn, and all processing results are respectively stored in the database in the form of a coroutine, and are linked with the corresponding government affairs matters by an external key, while the processing results are returned to the submission place of the corresponding government affairs matters.

[0007] Preferably, the step 1 specifically comprises: Step 1.1: Establish a transmission channel by adopting SSL / TLS two-way authentication, and receive the submission request of government affairs matters in real time through a distributed server interface; Step 1.2: Convert the government affairs matters received in real time into a JSON format, and attach a globally unique identification string generated by a snowflake algorithm, as well as the identity of the submission requester and the timestamp of the submission request, to generate a standardized packaged government affair matter; Step 1.3: Establish a FIFO message queue as a first message queue, and enqueue the standardized packaged government affair matter through the enqueue interface of the first message queue.

[0008] Preferably, the step 2 specifically comprises: Step 2.1: Dequeue the standardized packaged government affair matter in the first message queue in turn according to the first-in-first-out principle; Step 2.2: Establish a coroutine pool, and allocate a coroutine from the coroutine pool for each dequeued standardized packaged government affair matter; Step 2.3: Each coroutine asynchronously stores the corresponding standardized packaged government affair matter in the database with the attached identification string as the primary key, and distributes it to the corresponding processing unit.

[0009] Preferably, the step 3 specifically comprises: Step 3.1: After the corresponding processing unit of each government affair matter is processed, send the processing result to the distributed server interface, and the distributed server interface receives the sending request of the processing result in real time; Step 3.2: Convert the processing result received in real time into a JSON format, and attach a globally unique identification string generated by a snowflake algorithm, as well as the identity of the processing unit and the timestamp of the processing completion, to generate a standardized packaged processing result; Step 3.3: Establish a FIFO message queue as a second message queue, and enqueue the standardized packaged processing result through the enqueue interface of the second message queue.

[0010] Preferably, the step 4 specifically comprises: Step 4.1: Dequeue the standardized packaged processing result in the second message queue in turn according to the first-in-first-out principle; Step 4.2: reuse the coroutine pool in step 2.2, and allocate a coroutine from the coroutine pool for each dequeued standardized encapsulated processing result; Step 4.3: each coroutine asynchronously stores the corresponding standardized encapsulated processing result in the database with an additional identification string as the primary key, associates the corresponding government affair matter identification string through the foreign key, and returns the standardized encapsulated processing result to the corresponding government affair matter submission place according to the identity of the corresponding government affair matter submission requester.

[0011] The second aspect of the present application provides a digital government affair management system based on big data, which is used for the above-mentioned digital government affair management method based on big data, and comprises: A receiving module is configured to receive the submitted government affair matters in real time, send them to the first message queue and enqueue them; A first coroutine module is configured to dequeue the government affair matters in the first message queue one by one, store all the government affair matters in the database in the form of coroutines, and distribute them to the corresponding processing units; A processing module is configured to return the processing results to the second message queue and enqueue them after the corresponding processing units of each government affair matter are processed; A second coroutine module is configured to dequeue the processing results in the second message queue one by one, store all the processing results in the database in the form of coroutines, and establish a contact with the corresponding government affair matters through the foreign key, and return the processing results to the corresponding government affair matter submission place.

[0012] The third aspect of the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned digital government affair management method based on big data when executing the computer program.

[0013] The fourth aspect of the present application provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the above-mentioned digital government affair management method based on big data when executed by a processor.

[0014] The fifth aspect of the present application provides a computer program product comprising a computer program or instructions, and the computer program or instructions implement the steps of the above-mentioned digital government affair management method based on big data when executed by a processor.

[0015] The present application has the following advantages: (1) Based on the big data processing method, the message queue is used in the digital government management method and system to buffer the traffic, smooth the traffic peak, and achieve traffic peak reduction during the peak period. When facing a large number of concurrent accesses, the system will not be blocked and cause overload and crash; (2) When performing persistent storage and process processing within the digital government management system, coroutines are used in an asynchronous manner, which can handle multiple tasks concurrently, helping the system meet high availability requirements in high concurrency scenarios and improve the system throughput. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 This is an overall flow chart of a digital government management method based on big data.

[0018] Figure 2 This is an overall structural diagram of a digital government management system based on big data. DETAILED DESCRIPTION

[0019] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0020] Example 1: like Figure 1 As shown, the embodiment of the present application provides a digital government affairs management method based on big data, the method comprising: Step 1: Receive submitted government affairs in real time, send them to the first message queue and enqueue them.

[0021] Step 1 specifically includes: Step 1.1: Use SSL / TLS two-way authentication to establish a transmission channel. Receive government affairs submission requests in real time through the distributed server's API gateway interface. Verify the received government affairs, including required field verification and format compliance testing. Intercept illegal format requests and return a preset error code. Step 1.2: Convert the real-time received government affairs into JSON format, and attach the globally unique identification string generated by the snowflake algorithm, as well as the identity of the requestor and the timestamp of the request submission, to generate a standardized packaged government affair; Step 1.3: Establish a FIFO message queue as the first message queue, which contains an enqueue interface and a dequeue interface. The standardized packaged government affairs are enqueued through the enqueue interface of the first message queue, where FIFO represents a first-in, first-out message queue.

[0022] Step 2: Dequeue the government affairs in the first message queue one by one, and store all government affairs in the database in a coroutine manner, and distribute them to the corresponding processing units. The database is a relational database, and through the first message queue, it can avoid a large number of persistent storage operations under high concurrency scenarios, preventing the database from crashing.

[0023] Step 2 specifically includes: Step 2.1: Dequeue the standardized packaged government affairs in the first message queue in a first-in, first-out manner; Step 2.2: Establish a coroutine pool and allocate a coroutine from the coroutine pool for each dequeued standardized packaged government affair; Specifically, coroutine is a high-efficiency way to achieve asynchronous processing. Through coroutine, multiple tasks can be processed concurrently in a single-threaded manner, avoiding thread switching overhead, and handling high-concurrency I / O tasks in the current scenario. Modern programming languages such as Python, Go, Kotlin, and Java support coroutine.

[0024] Step 2.3: Each coroutine asynchronously stores the corresponding standardized packaged government affair in the database with the attached identification string as the primary key, and distributes it to the corresponding processing unit. Coroutine allows tasks to be suspended and resumed during execution, sequentially implementing asynchronous behavior. For example, when waiting for a database response, the coroutine can suspend the current task and let other coroutines execute, and then resume after the database returns a response. This approach avoids thread blocking and improves concurrency efficiency.

[0025] Step 3: After the processing unit corresponding to each government affair is processed, the processing result is returned to the second message queue and enqueued.

[0026] Step 3 specifically includes: Step 3.1: After the processing unit corresponding to each government affair is processed, the processing result is sent to the distributed server interface, which receives the request for sending the processing result in real time; Step 3.2: Convert the processing result received in real time into JSON format, and attach the globally unique identification string generated by the snowflake algorithm, as well as the identity of the processing unit and the timestamp of the completion of processing, to generate a standardized encapsulated processing result; Step 3.3: Establish a FIFO message queue as a second message queue, and enqueue the standardized encapsulated processing result through the enqueue interface of the second message queue.

[0027] Step 4: Dequeue the processing results in the second message queue one by one, and store all the processing results in the database in a coroutine manner, and establish a connection with the corresponding government affairs matter by foreign key, and return the processing result to the submission place of the corresponding government affairs matter.

[0028] Step 4 specifically includes: Step 4.1: Dequeue the standardized encapsulated processing result in the second message queue one by one according to the first-in, first-out principle; Step 4.2: Reuse the coroutine pool in step 2.2 to allocate a coroutine to each dequeued standardized encapsulated processing result from the coroutine pool; Specifically, based on the design principle of high cohesion and low coupling, the coroutine pool in step 2.2 is reused in the current step, which can reduce the resource overhead caused by repeatedly creating a coroutine pool, and also improve the scalability of this part of the method.

[0029] Step 4.3: Each coroutine asynchronously stores the corresponding standardized encapsulated processing result in the database with the attached identification string as the primary key, associates it with the corresponding government affairs matter identification string through the foreign key, and returns the standardized encapsulated processing result to the submission place of the corresponding government affairs matter according to the identity of the corresponding government affairs matter submission requester.

[0030] In summary, the digital government management method based on big data provided by the embodiments of the present application has the following technical effects: (1) The processing method based on big data uses a message queue for traffic buffering in the digital government management method and system, smooths traffic peaks, and realizes traffic peak shaving during peak periods, so that the system does not overload and crash due to blocking phenomena when facing a large number of concurrent accesses; (2) The coroutine is used in an asynchronous manner when performing persistent storage and process processing inside the digital government management system, which can handle multiple tasks concurrently, and is conducive to meeting the high availability requirements of the system in a high concurrency scenario, and improving the throughput of the system.

[0031] Embodiment Two: Based on the same inventive concept as the digital government management method based on big data in Embodiment One, such as Figure 2As shown, the present application provides a big data-based digital government affair management system, which comprises: A receiving module, which is configured to receive submitted government affairs in real time, send to a first message queue and enqueue; A first coroutine module, which is configured to dequeue the government affairs in the first message queue in sequence, persistently store all the government affairs to a database in a coroutine manner, and distribute to corresponding processing units; A processing module, which is configured to return the processing results to a second message queue and enqueue after the corresponding processing units of each government affair are processed; A second coroutine module, which is configured to dequeue the processing results in the second message queue in sequence, persistently store all the processing results to the database in a coroutine manner, establish a contact with the corresponding government affairs by a foreign key manner, and return the processing results to the submission of the corresponding government affairs.

[0032] Through the foregoing detailed description of the big data-based digital government affair management method, those skilled in the art can clearly know the big data-based digital government affair management system in the present embodiment. Since the system corresponds to the method disclosed in the embodiment, the system is described simply, and the relevant part can be referred to the method part.

[0033] Embodiment three In the embodiment three, a computer device is provided, which comprises a memory and a processor. The memory stores a computer program. The processor implements the steps of the big data-based digital government affair management method when executing the computer program.

[0034] Embodiment four In the embodiment four, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by a processor to implement the steps of the big data-based digital government affair management method.

[0035] Embodiment five In the embodiment five, a computer program product is provided, which comprises a computer program or instructions. The computer program or instructions are executed by a processor to implement the steps of the big data-based digital government affair management method.

[0036] The technical features of the above embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described. However, as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present disclosure.

[0037] The foregoing description of the disclosed embodiments enables a person skilled in the art to make or use the application. Modifications of these embodiments will occur to persons of skill in the art, and that the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Therefore, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A digital government affairs management method based on big data, characterized in that: The method comprises: Step 1: Receive submitted government affairs in real time, send them to the first message queue and add them to the queue; Step 2: Dequeue the government affairs in the first message queue one by one, store all government affairs in the database persistently in a coroutine manner, and distribute them to the corresponding processing units; Step 3: After each government matter is processed by its corresponding processing unit, the processing result is returned to the second message queue and queued; Step 4: Dequeue the processing results in the second message queue in sequence, and store all the processing results persistently in the database in the form of a coroutine, and establish a connection with the corresponding government affairs in the form of a foreign key, and at the same time send the processing results back to the submission location of the corresponding government affairs.

2. A digital government affairs management method based on big data as claimed in claim 1, characterized in that: The step 1 specifically includes: Step 1.1: Use SSL / TLS two-way authentication to establish a transmission channel and receive government affairs submission requests in real time through the distributed server interface; Step 1.2: Convert the government affairs received in real time into JSON format and append a globally unique identification string generated by the snowflake algorithm, as well as the identity of the requesting party and the timestamp of the request, to generate a standardized encapsulated government affairs document. Step 1.3: Establish a FIFO message queue as the first message queue, and enqueue the standardized encapsulated government affairs through the enqueue interface of the first message queue.

3. A digital government affairs management method based on big data as claimed in claim 2, characterized in that: The step 2 specifically includes: Step 2.1: Dequeue the standardized encapsulated government affairs in the first message queue in sequence according to the first-in-first-out principle; Step 2.2: Create a coroutine pool and assign a coroutine from the coroutine pool to each dequeued standardized encapsulated government affairs item; Step 2.3: Each coroutine asynchronously stores the corresponding standardized encapsulated government affairs in the database with the attached identification string as the primary key, and distributes it to the corresponding processing unit.

4. A digital government affairs management method based on big data as claimed in claim 1, characterized in that: The step 3 specifically includes: Step 3.1: After each government matter is processed by its corresponding processing unit, the processing result is sent to the distributed server interface, and the distributed server interface receives the request to send the processing result in real time; Step 3.2: Convert the received processing results into JSON format and append a globally unique identification string generated by the snowflake algorithm, along with the processing unit's identity and the processing completion timestamp, to generate a standardized encapsulated processing result. Step 3.3: Establish a FIFO message queue as the second message queue, and enqueue the standardized encapsulated processing results through the enqueue interface of the second message queue.

5. A digital government affairs management method based on big data as claimed in claim 4, characterized in that: The step 4 specifically includes: Step 4.1: Dequeue the standardized encapsulated processing results in the second message queue in sequence according to the first-in-first-out principle; Step 4.2: Reuse the coroutine pool in step 2.2 and assign a coroutine from the coroutine pool to each dequeued standardized encapsulated processing result; Step 4.3: Each coroutine asynchronously stores the corresponding standardized encapsulated processing results in the database with the attached identification string as the primary key, associates it with the corresponding government affairs identification string through the foreign key, and returns the standardized encapsulated processing results to the submission location of the corresponding government affairs according to the identity identification of the party submitting the corresponding government affairs.

6. A digital government affairs management system based on big data, characterized in that: The system comprises: A receiving module, the receiving module is used to receive submitted government affairs in real time, send them to the first message queue and queue them; A first coroutine module, configured to sequentially dequeue government affairs items from the first message queue, persistently store all government affairs items in a database in a coroutine manner, and distribute them to corresponding processing units; A processing module, wherein after each government matter is processed by the corresponding processing unit, the processing module returns the processing result to the second message queue and queues it; The second coroutine module is used to dequeue the processing results in the second message queue in sequence, store all the processing results persistently in the database in the form of a coroutine, establish a connection with the corresponding government affairs in the form of a foreign key, and at the same time return the processing results to the submission location of the corresponding government affairs.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of a digital government management method based on big data as described in any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a digital government management method based on big data as described in any one of claims 1 to 5 are implemented.

9. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of a digital government management method based on big data as described in any one of claims 1 to 5 are implemented.

Images