RPA automatic live broadcast system and method based on embedded WebView module of mobile equipment

Abstract

The invention discloses an RPA automatic live broadcast system based on a WebView module embedded in a mobile device, the RPA automatic live broadcast system comprises the mobile device and a Web end, the mobile device comprises a pushing end and a watching end, and the WebView module is embedded in the pushing end; the WebView module is used for loading a customized JS script and calling a required JS method, can receive various data and instructions transmitted by the pushing end, and feeds back an execution result of the Web end to the pushing end; the Web end is provided with an RPA function module and has a live broadcast platform function; and the RPA function module can realize automatic simulation operation of account authorization, commodity popup and bullet screen sending. The invention also discloses a method of the live broadcast system. The method comprises the following steps: S1, initializing; s2, authorizing login; s3, live broadcast operation (including commodity pushing and bullet screen replying) is carried out; and S4, ending live broadcast. The method improves the live broadcast efficiency, reduces manual errors, improves the live broadcast quality, enhances the live broadcast flexibility, can be customized and developed according to different platforms, and is wide in application range.

Description

Technical Field

[0001] This invention relates to the field of automated live streaming technology, and in particular to an RPA automated live streaming system and method based on a WebView module embedded in a mobile device. Background Technology

[0002] In current mobile live streaming scenarios, many operations during the live stream (such as account authorization, product display, and sending interactive comments) largely rely on manual completion. This not only increases the operational burden on the streamer and makes them prone to errors, but also results in low efficiency, making it difficult to meet the diverse and efficient demands brought about by the rapid development of live streaming.

[0003] Although some automation technologies exist, their integration and application in mobile live streaming scenarios are limited. Traditional automation solutions often fail to fully consider the real-time, mobility, and compatibility with live streaming platforms of mobile live streaming, and thus cannot well adapt to the complex business logic of mobile live streaming. Summary of the Invention

[0004] This invention aims to provide a robotic process automation (RPA) automated live streaming system based on WebView embedded in mobile devices, in order to solve the problems of cumbersome and inefficient operation of existing mobile device live streaming, and to automate key operations such as account authorization, product pop-ups, and sending bullet comments during the live streaming process, thereby improving the efficiency and quality of live streaming.

[0005] The technical solution of the present invention is as follows: An RPA-automated live streaming system based on a WebView module embedded in a mobile device includes a mobile device and a web client. The mobile device includes a push client and a viewing client. The push client embeds a WebView module. The WebView module is used to load customized JS scripts and call the required JS methods. It can receive various data and instructions transmitted by the push client and feed back the execution results of the web client to the push client. The web client is equipped with an RPA function module and has live streaming platform functionality. The RPA function module is designed based on the JS script code of the web client to automate the operation of JS methods, including simulated operations such as account authorization, product pop-ups, and sending bullet comments.

[0006] Preferably, the functions of the web client include: loading the live broadcast homepage, detecting user login status, confirming verification code information, loading the user information page, querying and loading product information, and loading the user comment section page.

[0007] Preferably, the RPA function module is implemented based on the JS script loaded by the WebView module, and can operate the loaded Web page based on the JS script, including: simulating the user's login operation on the account authorization page, simulating clicking the "Showcase" button on the product management page, and simulating inputting reply content in the bullet screen input area.

[0008] This invention also provides an RPA-automated live streaming method based on a WebView module embedded in a mobile device, employing the aforementioned RPA-automated live streaming system, comprising the following steps: S1. Initialization: S11. Start the live streaming function on the mobile device, initialize the embedded WebView module, and load the customized JS script. S12. Start WebSocket on the mobile device to establish a communication connection with the web client and server, in preparation for subsequent data interaction; S2, Authorized Login: S21. The user loads the live streaming platform's account authorization and login page through the WebView module of the mobile device; S22. The RPA function module on the web side is based on a customized JS script, which realizes user authorization login by simulating operations; S3, Live Streaming Operation: S31. After the user authorizes and logs in, the online live broadcast will start in the live broadcast room on the push end, and the viewing end can receive the live broadcast content in real time. S32, Product Pop-up: When a user explains a product on the push end during the live stream, the RPA function module captures the product information and sends it to the JS script of the WebView module. Then, it sends the product pop-up command to the Web end. The Web end then executes the automated process of the product pop-up, so that the user on the viewing end can see the corresponding product pop-up interface in the live stream. S4. End of live stream: Automatically release program resources when the live stream ends.

[0009] Preferably, in step S11, the operation of initializing the WebView module includes: S111. Initialize the WebView module: Embed the WebView module in the mobile device and initialize the WebView controller; S112. Load the target live stream page: Load the account's live stream homepage in the WebView module of the mobile device; S113. Injecting custom JS scripts: Load custom JS scripts in the WebView module; S114. Achieve bidirectional data interaction: Enable bidirectional data communication between mobile devices and the Web client; S115. Release runtime resources: When the live stream is closed, automatically release the resources occupied by the WebView module.

[0010] Preferably, step S12 further includes initializing the long link in the live stream comment section, specifically as follows: S121. Obtain WSS address: Obtain the WSS address of the long link in the comment section of the live broadcast room through the RPA function module or API function module; S122. Establish a long connection: Create a WebSocket instance to enable real-time bidirectional communication between the mobile device and the web client; S123. Maintain connection status: Build business logic for successful connections and reconnection mechanisms for connection failures to maintain a real-time long-term connection status. S124. Sending and receiving data: Sending and receiving messages to and from the server through the WebSocket instance, and processing them according to the business scenario; S125. Close connection: When the live stream ends, close the established long connection.

[0011] Preferably, in step S21, the RPA function module identifies page elements, simulates user operations, completes user authorization, and feeds back the authorization result to the mobile device.

[0012] Preferably, in step S22, after initialization, the live streaming homepage URL of the platform's personal account is automatically loaded on the push end. The login status of the page is monitored on the web end. If the user is already logged in, there is no need to go through the authorization process. The user information is sent back to the push end via JsBridge, and the push end displays the user information page according to the business scenario. If the user is not logged in, the QR code data on the page is sent back to the push end. The push end parses the data and displays the QR code. The user logs in after authorizing by scanning the code.

[0013] Preferably, in step S32, the automated process of executing the product pop-up on the Web end is as follows: first, the product name is searched; if it is not found, a message is sent back to inform the push end of the reason for the error, and the push end handles it according to the business scenario; otherwise, if the product is found, the RPA function module simulates clicking the explanation button in the product bar to achieve the purpose of the product pop-up.

[0014] Preferably, step S3 further includes step S33, sending bullet comments: obtaining the WSS address of the long link in the live room comment section through the WebSocket interface, establishing the long link, and having the RPA function module listen to the content in the live room comment section. If a keyword is matched, the content to be replied to is automatically matched, and then the bullet comment instruction is sent to the Web end. Then the Web end executes the automated process of sending bullet comments, so that the user on the viewing end can see the reply content in the live room comment section.

[0015] Furthermore, in step S33, the automated process of sending bullet comments on the Web client is as follows: first, the input box for reply content is searched; if it is not found, a message is sent back to inform the push client of the reason for the error, and the push client handles it according to the business scenario; otherwise, if the input box is found, the input box is simulated to be clicked, and the received reply content is pasted into the input box, and then it is simulated to be sent to the comment section of the live broadcast room to achieve the effect of automatic reply.

[0016] The beneficial effects of this invention are as follows: 1. Improve live streaming efficiency: By automating account authorization, product pop-ups, and sending comments, the time and effort required for manual operation by the streamer are greatly reduced, allowing the streamer to focus more on the live streaming content itself and improving the overall efficiency of the live streaming.

[0017] 2. Improve live streaming quality: Reduce the probability of human error and ensure the accuracy and stability of various operations during the live stream, thereby improving the quality of the live stream and the viewer experience.

[0018] 3. Enhanced live streaming flexibility: The design based on WebView and custom JS scripts allows the system to be easily customized according to the characteristics and needs of different live streaming platforms, enhancing the system's adaptability and flexibility in different live streaming scenarios. Attached Figure Description

[0019] Figure 1 This is an architecture diagram of an RPA automated live streaming system based on a WebView module embedded in a mobile device, according to the present invention. Figure 2 This is an initialization diagram of an RPA automated live streaming system based on a WebView module embedded in a mobile device, according to the present invention. Figure 3 This is a flowchart illustrating the account authorization process of an RPA automated live streaming system based on a WebView module embedded in a mobile device, according to the present invention. Figure 4 This is a flowchart of a product pop-up window in an RPA automated live streaming system based on a WebView module embedded in a mobile device, according to the present invention. Figure 5This is a flowchart illustrating the bullet screen sending process of an RPA automated live streaming system based on a WebView module embedded in a mobile device, according to the present invention. Detailed Implementation

[0020] The present invention will be further described below with reference to the accompanying drawings and embodiments: Current mobile live streaming relies heavily on manual operations such as account authorization, product pop-ups, and comment sending, which is inefficient and prone to errors. Existing automation technologies struggle to adapt to its complex business logic. Therefore, this invention provides an RPA-automated live streaming system based on a WebView module embedded in a mobile device. This system pushes live streams from the mobile device, embeds a WebView module, and starts a WebSocket. Customized JavaScript scripts are loaded through the WebView module, and the web client implements RPA functionalities based on these scripts, covering automated operations such as account authorization, product pop-ups, and comment sending.

[0021] like Figure 1 As shown, this automated live streaming system includes a mobile device and a web client. The mobile device can communicate with the web client and the server in real time. The mobile device includes a push client and a viewing client. The push client is responsible for pushing the live stream, and the viewing client is used to watch the live stream. The push client embeds an invisible WebView module, which can communicate with the web client, various servers, or other related clients by starting a WebSocket port. The mobile device, as the initiator and basic carrier of the live stream, provides the operating environment for the entire automated live streaming process.

[0022] The WebView module is used to load customized JS scripts and call the required JS methods. This script is the core carrier for implementing RPA automation functions. Through interaction with the push client and the web client, the WebView module receives various data and instructions transmitted by the push client, including product information and bullet screen replies, and feeds back the execution results from the web client to the push client.

[0023] Specifically, the JavaScript code implements the following functionality for product information: When a notification is sent to the push notification server that a product needs to be listed, the product name / product ID is automatically filled into the input box. The DOM update event of the list is checked; if the list returns the corresponding search result, the user is simulated clicking the "List" button for the corresponding product; otherwise, no action is taken. The code implements the following functionality for handling bullet screen replies: When a notification is sent to the push notification server regarding the bullet screen comment to be replied to, the user is simulated to use JavaScript to fill the corresponding text into the corresponding input box, and after a 500ms delay, the reply button is clicked.

[0024] The web client is equipped with an RPA function module, which can realize the functions of the live streaming platform, including: loading the live streaming homepage, detecting user login status, confirming verification code information, loading user information page, querying and loading product information, loading user comment section page, and implementing product pop-up and bullet screen sending functions based on RPA operations.

[0025] The RPA module, implemented using JavaScript loaded by the WebView module, is used to automate specific functions such as account authorization, product pop-ups, and sending comments. This module simulates operations on relevant pages using JavaScript, such as simulating user login on the account authorization page, clicking the "Add to Showcase" button on the product management page, and simulating inputting replies in the comment input area.

[0026] In this invention, the WebView module is embedded in the push terminal to load custom JS files. The RPA function module on the Web terminal is designed with JS script code for functions such as displaying products and sending bullet comments to automate the operation of JS methods. The push terminal and the Web terminal work together to complete the automated operation effect. The push terminal can load the JS scripts on the Web terminal and call the required JS methods (such as displaying products and sending bullet comments).

[0027] This invention also provides an RPA-automated live streaming method based on a WebView module embedded in a mobile device. Using the aforementioned RPA-automated live streaming system, after initialization, each functional module executes tasks according to a specific process. The authorization module uses JavaScript to simulate user account authorization, the product pop-up module transmits information via the mobile device to display product pop-ups, and the bullet comment module automatically sends bullet comments based on keywords. The specific steps are as follows: S1. Initialization: S11. Start the live streaming function on the mobile device (push end, viewing end), initialize the embedded WebView module, and load the customized JS script; S12. Start WebSocket on the mobile device (push end, viewing end) to establish a communication connection with the web end, server or other necessary ends to prepare for subsequent data interaction; like Figure 2 As shown, in step S11, the operation of initializing the WebView module includes: S111. Initialize the WebView module: Embed the WebView module in the mobile device and initialize the WebView controller; S112. Load the target live stream page: Load the account's live stream homepage in the WebView module of the mobile device; S113. Injecting custom JS scripts: Load custom JS scripts into the WebView module. This JS file contains the core code of the RPA function module. The required RPA function is implemented through custom code. S114. Achieve bidirectional data interaction: Enable bidirectional data communication between mobile devices and the Web client; S115. Release runtime resources: When the live stream is closed, automatically release the resources occupied by the WebView module.

[0028] like Figure 2 As shown, step S12 also includes initializing the long links in the live stream comment section. The specific steps are as follows: S121. Obtain WSS address: Obtain the WSS address of the long link in the comment section of the live broadcast room through the RPA function module or API function module; S122. Establish a long connection: Create a WebSocket instance to enable real-time bidirectional communication between the mobile device and the web client; S123. Maintain connection status: Build business logic for successful connections and reconnection mechanisms for connection failures to maintain a real-time long-term connection status. S124. Sending and receiving data: Sending and receiving messages to and from the server through the WebSocket instance, and processing them according to the business scenario (such as matching the reply content according to the built-in rules and informing the RPA function module to send the reply content). S125. Close connection: When the live stream ends, close the established long connection.

[0029] S2, Authorized Login: S21. The user loads the live streaming platform's account authorization and login page through the WebView module of the mobile device; S22. The RPA function module on the web side is based on a customized JS script. By recognizing page elements, it simulates user operations to complete user authorization, such as automatically filling in account passwords, clicking the login button, etc., and then sends the authorization result back to the mobile device.

[0030] Specifically, in step S22, after initialization, the live streaming homepage URL of the platform's personal account is automatically loaded on the push end. The login status of the page is monitored on the web end. If the user is already logged in, no authorization process is required; JsBridge will send the user information back to the push end, which will then process it according to the business scenario, such as displaying user information. If the user is not logged in, the QR code data on the page is sent back to the push end. The push end parses the data and displays the QR code, which the user scans to authorize. During this process, account risk control may require a verification code. The specific authorization process is as follows: Figure 3 As shown.

[0031] S3, Live Streaming Operation: S31. After the user authorizes and logs in, the live broadcast begins in the live broadcast room on the push terminal, while the viewing terminal receives the live broadcast content. During the live broadcast, the device can recommend products based on the live broadcast content and automatically reply by listening to the content in the live broadcast room's comment section in real time. S32. Product pop-up window: such as Figure 4 As shown, when a user explains a product on the push end during the live stream, the RPA module retrieves the product information (product ID or product name) and sends it to the JS script in the WebView module. The received product information is then sent to the Web end via the corresponding JS method: window.zxhy.up through JsBridge. Here, window.zxhy.up is a method agreed upon by the Web end and the push end. Then, the Web end executes the automated process of the product pop-up, so that the user on the viewing end can see the corresponding product pop-up interface in the live stream. Specifically, in step S32, the automated process of executing the product pop-up on the Web end is as follows: first, the product name is searched; if it is not found, a message is sent back to inform the push end of the reason for the error, and the push end handles it according to the business scenario; otherwise, if the product is found, the RPA function module simulates clicking the explanation button in the product bar to achieve the purpose of the product pop-up.

[0032] S33, Sending bullet comments: such as Figure 5 As shown, the WSS address of the long link in the live stream comment section is obtained through the WebSocket interface. After the long link is established, the RPA function module listens to the content of the live stream comment section. If a matching keyword is hit, the content to be replied to is automatically matched according to the built-in rules. After obtaining the reply content, the corresponding JS method, window.zxhy.up, is executed through JsBridge to send the bullet screen command to the Web client. Then, the Web client executes the automated process of sending bullet screens, so that the viewers can see the corresponding reply content in the live stream comment section.

[0033] Specifically, in step S33, the built-in rules include: 1) When a new user enters the live stream, a welcome message will be automatically sent, such as "Welcome new user"; 2) When a user follows, a thank-you message will be automatically sent, such as "Thank you for your attention"; 3) When the host receives a gift, a thank-you message will be automatically sent, such as "Thank you for your gift"; 4) When built-in keywords are triggered in the live stream chat, such as "rules", "size", "shipping insurance" and other commonly used words by live stream users, the host can automatically reply with pre-configured content; the automatic matching method obtains the WSS address of the live stream through the API function module and then establishes a long connection with the push terminal to capture the live stream chat content in real time and perform parsing and matching.

[0034] Specifically, in step S33, the automated process of sending bullet comments on the Web client is as follows: first, the input box for reply content is searched; if it is not found, a message is sent back to inform the push client of the reason for the error, and the push client handles it according to the business scenario; otherwise, if the input box is found, the input box is simulated to be clicked, and the received reply content is pasted into the input box, and then it is simulated to be sent to the comment section of the live broadcast room to achieve the effect of automatic reply.

[0035] S4. End of live stream: When the live stream stops, the running resources of the WebView module and RPA function module are automatically released.

[0036] It should be noted that the above embodiments are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Equivalent modifications made based on the above embodiments are all within the scope of protection of the present invention.

Claims

1. An RPA automation live streaming system based on a mobile device embedded WebView module, characterized in that, The mobile device and the web end, the mobile device includes a push end and a viewing end, the push end is embedded in a WebView module; the WebView module is used for loading customized JS scripts and calling required JS methods, can receive various data and instructions transmitted by the push end, and feeds back the execution results of the web end to the push end; the web end sets an RPA function module, and has a live broadcast platform function; the RPA function module can realize the automatic simulation operation of account authorization, product pop-up window and sending of bullet screen.

2. The RPA automation live broadcast system based on the embedded WebView module of a mobile device according to claim 1, characterized in that: The functions of the web end include: loading a live broadcast home page, user login state detection, confirming verification code information, loading a user information page, querying and loading product information and loading a user comment area page.

3. The RPA automation live broadcast system based on a mobile device embedded WebView module according to claim 1, characterized in that: The RPA function module is realized according to the JS scripts loaded by the WebView module, can operate the loaded web page based on the JS scripts, including: simulating the login operation of a user on an account authorization page, simulating the click on the upper window button on a product management page, and simulating the input of reply content in a bullet screen input area.

4. A method for live broadcasting RPA automation based on a mobile device live broadcasting embedded WebView module, using the RPA automation live broadcasting system of claim 1, characterized in that, The method comprises the following steps: S1, initialization: S11, starting the live broadcast push function of the mobile device, initializing the embedded WebView module and loading customized JS scripts simultaneously; S12, starting WebSocket of the mobile device, establishing communication connection with the web end and the server, and preparing for subsequent data interaction; S2, authorized login: S21, the user loads the account authorization and login page of the live broadcast platform through the WebView module of the mobile device; S22, the RPA function module of the web end realizes the authorized login of the user through simulation operation based on the customized JS scripts; S3, live broadcast running: S31, after the user is authorized and logged in, starting online live broadcast in the live broadcast room of the push end, and the viewing end can receive live broadcast content in real time; S32, product pop-up window: when the user explains the product in the push end during live broadcast, the RPA function module sends the product information to the JS scripts of the WebView module, and then sends the product pop-up instruction to the web end, and then the web end executes the automatic process of the product pop-up window, so that the user of the viewing end can see the corresponding product pop-up window interface appear in the live broadcast room; S4, live broadcast end: when stopping live broadcast, automatically releasing the running resources of the program.

5. The RPA automation live method based on a mobile device live embedded WebView module according to claim 4, characterized in that: In the step S11, the operation of initializing the WebView module comprises: S111, initializing the WebView module: embedding the WebView module in the mobile device and initializing the WebView controller; S112, loading the target live broadcast page: loading the live broadcast home page of the account in the WebView module of the mobile device; S113, injecting custom JS scripts: loading custom JS scripts in the WebView module; S114, realizing bidirectional data interaction: realizing bidirectional data communication between the mobile device and the web end; S115, releasing running resource occupation: when the live broadcast is closed, automatically releasing the resources occupied by the WebView module.

6. The RPA automation live method based on a mobile device live embedded WebView module according to claim 4, characterized in that: The step S12 further includes a long link initialization operation on the live room comment area, and the specific steps are as follows: S121, obtaining a WSS address: obtaining the WSS address of the long link of the live room comment area through an RPA function module or an API function module; S122, establishing a long link: creating a WebSocket instance to realize real-time bidirectional communication between a mobile device and a web end; S123, maintaining a connection state: constructing a business logic of successful connection and a reconnection mechanism of connection failure to maintain a real-time long link state; S124, sending and receiving data: sending and receiving messages to a server through the WebSocket instance, and processing according to a business scenario; S125, closing a connection: when the live broadcast is closed, the established long link state is closed.

7. The RPA automation live method based on a mobile device live embedded WebView module according to claim 4, characterized in that: In the step S22, after the initialization is completed, the live broadcast home page URL of the personal account of the platform is automatically loaded on the push end, and the login state of a page is monitored on the web end; if the login state is a logged-in state, the user information is returned to the push end through JsBridge, and the push end displays the user information page according to a business scenario; if the login state is a logged-out state, the two-dimensional code data on the page is returned to the push end, the push end analyzes the data and displays the two-dimensional code, and the user logs in after authorization through the two-dimensional code.

8. The RPA automation live method based on a mobile device live embedded WebView module according to claim 4, characterized in that: In the step S32, the web end executes an automatic process of a product pop-up window as follows: first, the received product name is searched, if the product is not found, a message is returned to the push end to inform an abnormal reason, the push end processes according to a business scenario; otherwise, the product is found, the RPA function module simulates clicking on the explanation button in the product column to achieve the purpose of the product pop-up window.

9. The RPA automation live method based on a mobile device live embedded WebView module according to claim 4, characterized in that: In the step S3, the step S33, sending a barrage: the WSS address of the long link of the live room comment area is obtained through a WebSocket interface, the RPA function module listens to the content of the live room comment area after the long link is established, if a keyword is hit, the content to be replied is automatically matched, a barrage instruction is sent to the web end, then the web end executes an automatic process of sending a barrage, so that the user of the viewing end can see the reply content in the live room comment area.

10. The RPA automation live method based on a mobile device live embedded WebView module according to claim 9, characterized in that: In the step S33, the web end executes an automatic process of sending a barrage as follows: first, an input box of the reply content is searched, if the input box is not found, a message is returned to the push end to inform an abnormal reason, the push end processes according to a business scenario; otherwise, the input box is found, the input box is simulated to be clicked, the received reply content is pasted into the input box, and then the input box is simulated to be sent to the live room comment area, so as to achieve the effect of automatic reply.

Images