Content compliance detection replacement method, system and medium under micro front-end architecture

Through the combination of Nginx three-layer structure and AI detection model, efficient and low-cost content compliance detection replacement under the micro front-end architecture is achieved, solving the problems of low efficiency and high cost in traditional audits, and improving detection accuracy and performance.

CN118509197BActive Publication Date: 2024-11-22CHINA CONSTR THIRD ENG BUREAU GRP CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410539980.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-04-30
Publication Date
2024-11-22
Estimated Expiration
2044-04-30

AI Technical Summary

Technical Problem

Under the micro front-end architecture, traditional legal contracts + manual review cannot effectively ensure the compliance of third-party service content, especially after the integration of massive information services, resulting in low efficiency, high cost of content compliance detection, and inability to meet the business Development needs.

Method used

Adopt Nginx three-layer structure detection and replacement mechanism, unpack, replace and packetize third-party service resources through pre-configured AI detection model and Lua extension, to realize content compliance detection and replacement, and use the combination of local AI and cloud services. Carry out risk detection and replacement to reduce server resource consumption.

Benefits of technology

It improves the efficiency and performance of content compliance detection, realizes dynamic risk content replacement without human intervention, reduces operation and maintenance pressure and cost, supports compliance detection of all types of files, and improves detection accuracy and server performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118509197B_ABST
    Figure CN118509197B_ABST
Patent Text Reader

Abstract

The present invention discloses a content compliance detection and replacement method, system and medium under a micro-frontend architecture, the method comprising: obtaining a third-party service resource request; based on a pre-configured Nginx three-layer structure detection and replacement mechanism, sequentially unpacking, replacing and packaging the requested third-party service resources to complete content compliance detection and replacement. The present invention realizes content compliance detection and replacement with good performance, high efficiency and low cost under a micro-frontend architecture through a three-layer structure detection and replacement mechanism.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a content compliance detection and replacement method, system and medium under a micro-front-end architecture. Background Art

[0002] With the rapid development of the mobile Internet industry, the word compliance has appeared in more and more laws and regulations in recent years. Content compliance and data compliance have become an indispensable part of the development of enterprise informatization.

[0003] Traditional enterprise information platforms are relatively simple (such as common official websites), all content comes from the company's own internal information, and content review is completed by the company. However, with the popularization of mobile Internet, more personalized and customized content appears in enterprise information platforms, so a large number of third-party services will also be integrated into enterprise information platforms, which also makes content compliance more and more stressful for enterprise information operators.

[0004] With the development of micro-frontend architecture, it is becoming easier and easier to integrate third-party service content. Various information services built based on different front-end technology systems and technical frameworks can be quickly integrated into the enterprise information platform through micro-frontend technology, blurring the boundaries between proprietary services and third-party services, enhancing the consistency of user experience, and increasing user stickiness. This has greatly improved the development efficiency of enterprise informationization and shortened the R&D cycle. Therefore, enterprise informationization with micro-frontend architecture is increasingly valued by enterprise technical leaders. However, one disadvantage of micro-frontend is that the code scripts and JavaScript files of different service parties will be sent to the user's browser through the micro-frontend main application agent, and third-party JavaScript scripts can load other third-party JavaScript scripts when the user's browser is running. This will cause the enterprise informationization platform as the main application to bear corresponding risks once there is a risk in the third-party content.

[0005] Based on the dynamic, variable and hot-updatable behavior of JavaScript scripts, when massive information services are integrated by micro-frontends, traditional content compliance based on legal contracts + manual review can no longer meet the needs of business development. There is an urgent need for an efficient and fast technical means to integrate third-party services and ensure the content compliance of third-party services. Summary of the invention

[0006] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides a content compliance detection and replacement method, system and medium under a micro-front-end architecture. Through a three-layer detection and replacement mechanism, content compliance detection and replacement with good performance, high efficiency and low cost is realized under a micro-front-end architecture.

[0007] According to one aspect of the present invention, a content compliance detection replacement method under a micro-frontend architecture is provided, comprising:

[0008] Get third-party service resource requests;

[0009] Based on the pre-configured Nginx three-layer structure detection and replacement mechanism, the requested third-party service resources are unpacked, replaced and packaged in sequence to complete content compliance detection and replacement.

[0010] As a further technical solution, the unpacking further includes:

[0011] Intercept the sub-application entry request configured by the micro-frontend main application and proxy the sub-application entry address;

[0012] Based on the proxy's sub-application entry address, load third-party service resource files and classify them;

[0013] Execute different decompression strategies for resource files that need to be decompressed after classification.

[0014] As a further technical solution, for the image files that do not need to be decompressed after classification, the replacement further includes:

[0015] The uncompressed image files are inspected based on the pre-trained local AI detection model, and when risks are detected, the image resources are replaced with placeholder images of the same size.

[0016] As a further technical solution, when the local AI detection model cannot recognize the image file, the image file is uploaded and subjected to cloud service detection.

[0017] As a further technical solution, for the decompressed document and the request file, the replacement further includes:

[0018] The document and the requested file are tested for content compliance based on a pre-trained local AI detection model combined with keywords, and the resource path in the file is modified to a proxy path after the detection is completed.

[0019] As a further technical solution, for the decompressed JavaScript script file, the replacement further includes:

[0020] The JavaScript script file is tested for content compliance according to a pre-trained local AI detection model combined with keywords, and after the test is completed, the resource path requested in the JavaScript script file is modified to a proxy path, and the request path of the JavaScript script file is modified to a proxy path.

[0021] As a further technical solution, when the local AI detection model detects high risk, the corresponding high-risk files are uploaded and tested on the cloud service.

[0022] As a further technical solution, the package further includes:

[0023] Record risk points for high-risk files after content compliance testing;

[0024] Obfuscate the replaced resources;

[0025] Compress the obfuscated files.

[0026] Furthermore, the risk point recording and resource obfuscation are both implemented using Lua extensions.

[0027] According to one aspect of the present disclosure, a content compliance detection and replacement system under a micro-frontend architecture is provided, including:

[0028] The acquisition module is used to obtain third-party service resource requests;

[0029] The content compliance detection and replacement module is used to unpack, replace and encapsulate the JavaScript scripts of the third-party service resources in sequence based on the pre-configured Nginx three-layer structure detection and replacement mechanism to complete the content compliance detection and replacement.

[0030] According to one aspect of the present specification, there is provided a computer storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the content compliance detection replacement method under the micro-frontend architecture are implemented.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] 1. High efficiency. The present invention is based on the method of keywords + local AI + cloud services. Compared with traditional content compliance review, its efficiency is greatly improved. It can also mark risks, dynamically replace risky content, and realize content compliance detection without human intervention.

[0033] 2. Good performance. The present invention utilizes a lightweight Nginx+Lua extended script solution, which can avoid the high server resource occupation of the traditional headless browser solution, and does not need to enable background services separately. It has good concurrency and greatly improves execution performance.

[0034] 3. Complete content. Compared with traditional compliance detection that only targets ordinary HTML documents or rendered HTML documents, the present invention intercepts all third-party service resources, and detects and replaces pictures, JavaScript scripts, and request data except HTML documents.

[0035] 4. Low cost. The server of the present invention only needs a single-core 2G configuration to run smoothly, with low operation and maintenance pressure and low overhead cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 The present invention is a flowchart of a content compliance detection replacement method under a micro-frontend architecture according to an embodiment of the present invention.

[0037] Figure 2 Schematic diagram of an unpacking process according to an embodiment of the present invention.

[0038] Figure 3 The figure is a schematic diagram of a picture compliance detection replacement process according to an embodiment of the present invention.

[0039] Figure 4 A schematic diagram of a document compliance detection replacement process according to an embodiment of the present invention.

[0040] Figure 5 The figure is a schematic diagram of a JavaScript script compliance detection replacement process according to an embodiment of the present invention.

[0041] Figure 6 Schematic diagram of a packet packaging process according to an embodiment of the present invention.

[0042] Figure 7 Schematic diagram of a content compliance detection and replacement system under a micro-frontend architecture according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] In view of the drawbacks of the traditional content compliance detection method based on legal contract + manual review, the present invention provides a content compliance detection and replacement method under a micro-frontend architecture. The method designs a three-layer detection and replacement mechanism based on lightweight Nginx, which can start from the source file of the third-party service, unpack, replace and package the JavaScript scripts therein, ensure that all files loaded through the third party can be transmitted through the Nginx of the micro-frontend main application, and implement compliance detection through keywords + AI, and finally send it to the user's browser through packaging.

[0044] The whole process of the present invention can ensure that all resources are audited by its own service through unpacking and injection. There is no need to run a headless browser on the server side (that is, it is equivalent to installing a browser on the server side), nor is there a need to run an independent backend service separately. It only relies on lightweight Nginx and Lua extensions, with superior and reliable performance. At the same time, mature keywords + AI capabilities are used to ensure the accuracy of content compliance detection.

[0045] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations on the technical solution of the present invention. The embodiments of the present invention and the technical features in the embodiments may be combined with each other unless there is a conflict. Example 1

[0046] This embodiment introduces a content compliance detection and replacement method under a micro-frontend architecture. The method is based on a three-layer detection and replacement mechanism to ensure that all files loaded through a third party can be transmitted through the Nginx of the micro-frontend main application and implement content compliance detection.

[0047] like Figure 1 As shown, the method includes:

[0048] Step 1: Get a third-party service resource request.

[0049] Step 2: Based on the pre-configured Nginx three-layer structure detection and replacement mechanism, the requested third-party service resources are unpacked, replaced, and packaged in sequence to complete content compliance detection and replacement.

[0050] In this embodiment, Figure 2 As shown, the unpacking process is divided into three steps: interception, classification, and decompression.

[0051] In the micro-frontend architecture, a service entry address of a sub-application will be configured in the main application. This embodiment uses Nginx's Proxy_pass proxy method to intercept and proxy this address, rather than directly applying this address to the micro-frontend main application as in the traditional micro-frontend.

[0052] After intercepting the sub-application entry request, two more steps need to be done: first classify the resource files, and then implement different decompression strategies for different resource files. It should be noted that after intercepting the sub-application entry request, the sub-application entry will load different resource types. There are two common types: static files and script files (JavaScript files).

[0053] This embodiment implements different decompression strategies for different file types, as shown below:

[0054] 1. Static files of resource image type do not need to be decompressed;

[0055] 2.Html, PHP and other files are intercepted and then decompressed by GZIP;

[0056] 3. After the JavaScript file is intercepted, it is decompressed by GZIP and formatted by JavaScript.

[0057] After three steps of interception, classification and decompression, the unpacking process is completed.

[0058] After the unpacking process, different files will be generated, including uncompressed image files, decompressed document files (such as HTML, PHP files), and decompressed and formatted JavaScript files. In the replacement step, these different products need to be detected and replaced.

[0059] For image files, such as Figure 3 As shown, this embodiment deploys a lightweight local AI detection model in the background. Through the Lua (a programming language based on Nginx for secondary development) program in Nginx, the image resources are detected by the local AI model. If there is no risk, they are released directly; if a risk is detected, the image resource is replaced with a placeholder image of the same size; if the local AI model cannot predict, the image is uploaded to the image content compliance detection service of the cloud service.

[0060] For documents and request files such as HTML, PHP, etc., such as Figure 4 As shown, two steps are required. The first is the compliance detection of document content. This type of document content is tested using the "keyword + local AI model" method. If only a small number of keywords are hit, use "*" to replace them and return the replaced document; if the keyword density is higher than 30%, the entire document is tested again using Lua and the local AI model. Second, all resource reference paths in HTML, PHP and other documents need to be modified to the proxy path in the unpacking step, including images, JavaScript resources, etc. This operation ensures that no third-party resources can skip the interception process in the unpacking step.

[0061] Furthermore, after the entire document is re-checked using Lua and the local AI model, if a high risk is found, the high-risk file is uploaded to the commercial cloud for cloud service detection; if there is no risk, "*" is used to replace the keyword and the replaced document is returned.

[0062] Furthermore, when the cloud service detects a high risk, a blank document is returned; when the cloud service detects no risk, "*" is used to replace the keyword and the replaced document is returned.

[0063] For JavaScript script files, such as Figure 5As shown, three steps are required: first, compliance detection of file content, which is the same as document processing strategies such as HTML and PHP; second, modification of resource reference paths, which requires replacing all resource paths requested in JavaScript with the proxy paths in step 1; third, modification of request paths in JavaScript, which also needs to be replaced with the proxy paths in step 1 to ensure compliance detection of requests. Replacement process of JavaScript files.

[0064] The file generated after the above replacement process will be relatively large, and it is an unpacked file. Direct transmission will have certain security risks. Therefore, this embodiment sets a packaging process in the final stage, such as Figure 6 As shown, it is mainly divided into three steps: risk point recording; document and script file obfuscation; and document and script file compression.

[0065] First, for any type of file, after compliance testing, if it is a high-risk file, Lua will be used on the Nginx side to make a record and notify the website operation and maintenance personnel of the corresponding data, which is the risk point record.

[0066] Secondly, because the source files are unpacked and replaced before, in order to ensure security, Lua will be used to obfuscate the documents, scripts, requests, pictures and other resources after the replacement process.

[0067] Finally, the obfuscated file is compressed using GZIP and then sent to the user’s client.

[0068] The above is a detailed process of the detection and replacement method of micro-frontend content compliance. It should be noted that content compliance detection of cloud services is optional. In actual applications, enterprises can train their own unique detection models for their fields, and can achieve good detection results only by using keywords and local AI models. If the requirements for content compliance are extremely high, you can purchase the corresponding enterprise-level content compliance cloud service, but due to factors such as cloud service bandwidth, the response speed of the enterprise information platform will be reduced to a certain extent.

[0069] After practical testing, the three-layer detection device of this embodiment and the method of keywords plus local AI can achieve a success rate of about 80%, and the commercial compliance detection cloud service can reach 90%. Compared with traditional content compliance audits, it can greatly improve efficiency and perfectly match the advantages of fast access, fast launch, and consistent experience of micro-frontends, realizing a content compliance detection replacement solution with good performance, high efficiency, and low cost under the micro-frontend architecture. Example 2

[0070] Based on the same inventive concept as Example 1, this example introduces a content compliance detection and replacement system under a micro-front-end architecture. The system obtains user requests through an enterprise information platform and implements content compliance detection and replacement through the Nginx three-layer structure deployed in the server.

[0071] like Figure 7 As shown, a user logs in to the enterprise information platform and requests a third-party service resource through the enterprise information platform. The requested third-party service resource is detected and replaced by the Nginx three-layer structure detection and replacement mechanism deployed in the server and then sent to the user's browser.

[0072] The system comprises:

[0073] The acquisition module is used to obtain third-party service resource requests;

[0074] The content compliance detection and replacement module is used to unpack, replace and encapsulate the JavaScript scripts of the third-party service resources in sequence based on the pre-configured Nginx three-layer structure detection and replacement mechanism to complete the content compliance detection and replacement.

[0075] The specific functions of the above modules are implemented by referring to the relevant contents of the method in Example 1, which will not be described in detail. It is particularly pointed out that:

[0076] The unpacking further comprises:

[0077] Intercept the sub-application entry request configured by the micro-frontend main application and proxy the sub-application entry address;

[0078] Based on the proxy's sub-application entry address, load third-party service resource files and classify them;

[0079] Execute different decompression strategies for resource files that need to be decompressed after classification.

[0080] For the image files that do not need to be decompressed after classification, the replacement further includes:

[0081] The uncompressed image files are inspected based on the pre-trained local AI detection model, and when risks are detected, the image resources are replaced with placeholder images of the same size.

[0082] For the decompressed document and the request file, the replacement further includes:

[0083] The document and the requested file are tested for content compliance based on a pre-trained local AI detection model combined with keywords, and the resource path in the file is modified to a proxy path after the detection is completed.

[0084] For the decompressed JavaScript script file, the replacement further includes:

[0085] The JavaScript script file is tested for content compliance according to a pre-trained local AI detection model combined with keywords, and after the test is completed, the resource path requested in the JavaScript script file is modified to a proxy path, and the request path of the JavaScript script file is modified to a proxy path.

[0086] The packet further comprises:

[0087] Record risk points for high-risk files after content compliance testing;

[0088] Obfuscate the replaced resources;

[0089] Compress the obfuscated files. Example 3

[0090] Based on the same inventive concept as other embodiments, this embodiment introduces a computer storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the content compliance detection replacement method under the micro-frontend architecture are implemented.

[0091] In summary, the present invention intercepts all documents, pictures, requests, JavaScript scripts and other contents of third-party resources by dynamically replacing the contents, and performs compliance checks on different types of resources respectively, thereby realizing compliance checks of all types.

[0092] The present invention innovatively deploys three Nginx on one server, giving full play to the high performance of Nginx and the flexibility of Lua language, and realizing the functions that need to be realized in headless browser or independent background service. It should be noted that, when the resource consumption is relatively small, one Nginx can also complete the unpacking, replacement and packaging process. The deployment of three Nginx is for the later service distribution deployment. Some Nginx instances with relatively large resource consumption can dynamically expand multiple Nginx instances. For example, one Nginx instance for unpacking, three Nginx instances for replacement, and one Nginx instance for packaging.

[0093] The present invention proposes a detection method of local keywords + local AI model + cloud service. Through the method of local keywords + local AI model + cloud service, the trinity is integrated to ensure the accuracy of detection and, to a certain extent, the operating efficiency.

[0094] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0095] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0096] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0097] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0098] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

Claims

1. The content compliance detection and replacement method under the micro-frontend architecture is characterized by: include: Get third-party service resource requests; Based on the pre-configured Nginx three-layer structure detection and replacement mechanism, the requested third-party service resources are unpacked, replaced, and packaged in sequence to complete content compliance detection and replacement; The unpacking further includes: intercepting the sub-application entry request configured by the micro-frontend main application, and proxying the sub-application entry address; based on the proxy sub-application entry address, loading and classifying the third-party service resource files; executing different decompression strategies for the resource files that need to be decompressed after classification; For image files that do not need to be decompressed after classification, the replacement further includes: detecting the uncompressed image files according to a pre-trained local AI detection model, and replacing the image resources with placeholder images of the same size when risks are detected; when the local AI detection model cannot recognize the image file, uploading the image file and performing cloud service detection; For the decompressed document and request file, the replacement further includes: performing content compliance detection on the document and request file according to a pre-trained local AI detection model combined with keywords, and modifying the resource path in the file to a proxy path after the detection is completed; For the decompressed JavaScript script file, the replacement further includes: performing content compliance detection on the JavaScript script file according to a pre-trained local AI detection model combined with keywords, and after the detection is completed, modifying the resource path requested in the JavaScript script file to a proxy path, and modifying the request path of the JavaScript script file to a proxy path; when the local AI detection model detects a high risk, uploading the corresponding high-risk file and performing cloud service detection; The packaging further includes: recording risk points of high-risk files after content compliance detection; obfuscating resources after replacement processing; and compressing the obfuscated files.

2. The content compliance detection and replacement system under the micro-frontend architecture is characterized by: include: The acquisition module is used to obtain third-party service resource requests; A content compliance detection and replacement module is used to sequentially unpack, replace and encapsulate the JavaScript scripts of the third-party service resources based on a pre-configured Nginx three-layer structure detection and replacement mechanism to complete content compliance detection and replacement; The unpacking further includes: intercepting the sub-application entry request configured by the micro-frontend main application, and proxying the sub-application entry address; based on the proxy sub-application entry address, loading and classifying the third-party service resource files; executing different decompression strategies for the resource files that need to be decompressed after classification; For image files that do not need to be decompressed after classification, the replacement further includes: detecting the uncompressed image files according to a pre-trained local AI detection model, and replacing the image resources with placeholder images of the same size when risks are detected; when the local AI detection model cannot recognize the image file, uploading the image file and performing cloud service detection; For the decompressed document and request file, the replacement further includes: performing content compliance detection on the document and request file according to a pre-trained local AI detection model combined with keywords, and modifying the resource path in the file to a proxy path after the detection is completed; For the decompressed JavaScript script file, the replacement further includes: performing content compliance detection on the JavaScript script file according to a pre-trained local AI detection model combined with keywords, and after the detection is completed, modifying the resource path requested in the JavaScript script file to a proxy path, and modifying the request path of the JavaScript script file to a proxy path; when the local AI detection model detects a high risk, uploading the corresponding high-risk file and performing cloud service detection; The packaging further includes: recording risk points of high-risk files after content compliance detection; obfuscating resources after replacement processing; and compressing the obfuscated files.

3. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the content compliance detection replacement method under the micro-frontend architecture as described in claim 1 are implemented.