Business data processing method and device, storage medium and electronic equipment

Abstract

The invention discloses a business data processing method and device, a storage medium and electronic equipment. The method comprises the following steps: executing a target interaction operation in response to a first account, skipping to a first application, and acquiring a first image currently displayed by the first application and a business process template associated with the first application, the first image being used for indicating that the first account is acquiring business data associated with a target business, and the business process template being associated with the first application; the business process template is used for indicating an image feature corresponding to a process node of the target business; whether a target template exists in the business process template or not and whether the first application runs in a system foreground or not are inquired based on the first image, and image features of the target template and image features of the first image meet preset matching conditions; and under the condition that the target template exists and the first application always runs in the system foreground, executing a character recognition operation on the first image, and determining business data. The technical problem that the processing accuracy of the business data is relatively low is solved.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular to a method and device for processing business data, a storage medium, and an electronic device. Background Art

[0002] In the prior art, although screen sharing is used to obtain business data, it also faces the technical problem of low accuracy in business data processing. For example, when a user displays his business data in a third-party application through screen sharing, the background system needs to identify and analyze the shared screen content. However, traditional recognition technology may cause text recognition errors due to interference factors in the screenshot, such as user gestures, pop-ups of other applications, or changes in screen brightness, thereby affecting the accuracy of business data.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present application provide a method and device for processing business data, a storage medium, and an electronic device to at least solve the technical problem of low accuracy in processing business data.

[0005] According to one aspect of an embodiment of the present application, a method for processing business data is provided, including: in response to a first account executing a target interactive operation, jumping to a first application, obtaining a first image currently displayed by the first application and a business process template associated with the first application, wherein the first image is used to indicate that the first account is obtaining business data associated with a target business, and the business process template is used to indicate image features corresponding to process nodes of the target business; based on the first image, querying whether there is a target template in the business process template and whether the first application is running in the system foreground, wherein a preset matching condition is satisfied between the target template and the image features of the first image; and when the target template exists and the first application is always running in the system foreground, performing a text recognition operation on the first image to determine the business data.

[0006] According to another aspect of an embodiment of the present application, a business data processing device is also provided, including: an acquisition module, used to jump to a first application in response to a first account executing a target interactive operation, and acquire a first image currently displayed by the first application and a business process template associated with the first application, wherein the first image is used to indicate that the first account is acquiring business data associated with a target business, and the business process template is used to indicate image features corresponding to process nodes of the target business; a query module, used to query whether a target template exists in the business process template based on the first image and whether the first application is running in the system foreground, wherein a preset matching condition is satisfied between the target template and the image features of the first image; a determination module, used to perform a text recognition operation on the first image to determine the business data when the target template exists and the first application is always running in the system foreground.

[0007] Optionally, the device is also used to: in response to the first account executing the target interactive operation and jumping to the first application, record the start time of launching the first application; detect the running status of the first application; and when the running status indicates that the execution of the target business is completed, record the end time of exiting the first application.

[0008] Optionally, the device is also used to: determine the recording time period according to the start time and the end time; obtain the display time period of the target screen, wherein the target screen is used to display the first image, and the display time period is determined by the time when the target screen starts to display the first frame image of executing the target business and the time when the target screen ends to display the last frame image of executing the target business; when the target template exists, the first application runs in the system foreground, and the display time period and the recording time period meet the preset time period conditions, generate a first prompt message, wherein the first prompt message indicates that the business data can be used to indicate the business status of the target business corresponding to the first account.

[0009] Optionally, the device is used to query whether a target template exists in the business process template and whether the first application is running in the system foreground based on the first image in the following manner, including at least one of the following: when the target template does not exist, generating a second prompt message, wherein the second prompt message is used to indicate that the business data needs to be reacquired; when the first application switches from running in the system foreground to the system background, generating a third prompt message, wherein the third prompt message is used to indicate that the business data needs to be reacquired.

[0010] Optionally, the device is also used to: when generating at least one of the second prompt information and the third prompt information, add a feedback tag to the first image, wherein the feedback tag is used to indicate that the first image does not include the business data; and store the first image in the target database.

[0011] Optionally, the device is used to obtain the first image currently displayed by the first application and the business process template associated with the first application in the following manner: upon obtaining application authorization for the first application by the first account, receiving a video stream pushed by the first application in real time, wherein the video stream is composed of individual image frames displayed by the first application; and taking at least one of a timed screenshot operation or a frame-by-frame screenshot operation from the video stream to obtain the first image.

[0012] Optionally, the device is also used to: start the first application; when a target acquisition operation is triggered in the first application, cause a target screen to display the first image, wherein the target screen is used to display the first image, and the target acquisition operation is used to indicate acquisition of the business data; and acquire the first image and the business process template.

[0013] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned method for processing business data when running.

[0014] According to another aspect of the embodiment of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above method for processing business data.

[0015] According to another aspect of an embodiment of the present application, there is also provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the business data processing method through the computer program.

[0016] In the embodiment of the present application, in response to the first account executing the target interactive operation, jumping to the first application, obtaining the first image currently displayed by the first application and the business process template associated with the first application, wherein the first image is used to indicate that the first account is obtaining business data associated with the target business, and the business process template is used to indicate the image features corresponding to the process node of the target business; based on the first image, querying whether there is a target template in the business process template and whether the first application is running in the system foreground, wherein the image features of the target template and the first image meet the preset matching conditions; in the case where the target template exists and the first application is always running in the system foreground, performing a text recognition operation on the first image to determine the business data, combined with real-time screen sharing, image feature matching and application running status monitoring, the purpose of significantly improving the accuracy of business data processing is achieved, and data deviations that may be caused by user misoperation, switching applications or tampering with information are effectively avoided. In this way, the technical effect of greatly improving business processing efficiency and decision-making quality on the basis of ensuring the authenticity and compliance of user information is achieved, thereby solving the technical problem of low accuracy of business data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 is a schematic diagram of an application environment of an optional method for processing business data according to an embodiment of the present application;

[0019] Figure 2 is a flowchart of an optional method for processing business data according to an embodiment of the present application;

[0020] Figure 3 is a schematic diagram of an optional method for processing business data according to an embodiment of the present application;

[0021] Figure 4 is a schematic diagram of the structure of an optional service data processing device according to an embodiment of the present application;

[0022] Figure 5 is a schematic diagram of the structure of an optional business data processing product according to an embodiment of the present application;

[0023] Figure 6 It is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] The present application is described below in conjunction with embodiments:

[0027] According to one aspect of an embodiment of the present application, a method for processing business data is provided. Optionally, in this embodiment, the method for processing business data can be applied to: Figure 1 In the hardware environment composed of the server 101 and the terminal device 103 shown in FIG. Figure 1As shown, the server 101 is connected to the terminal device 103 via a network, and can be used to provide services for the terminal device or an application 107 installed on the terminal device. The application can be a video application, an instant messaging application, a browser application, an educational application, a game application, etc. A database 105 may be set up on the server or independently of the server to provide data storage services for the server 101, for example, a game data storage server. The above-mentioned network may include, but is not limited to, a wired network and a wireless network, wherein the wired network includes, a local area network, a metropolitan area network and a wide area network; the wireless network includes, Bluetooth, WIFI and other networks that implement wireless communication; the terminal device 103 may be a terminal configured with an application, and may include, but is not limited to, at least one of the following: a mobile phone (such as an Android phone, an iOS phone, etc.), a laptop computer, a tablet computer, a PDA, a MID (Mobile Internet Devices), a PAD, a desktop computer, a smart TV, an intelligent voice interaction device, a smart home appliance, a vehicle-mounted terminal, an aircraft, a virtual reality (VR) terminal, an augmented reality (AR) terminal, a mixed reality (MR) terminal and other computer devices; the above-mentioned server may be a single server, a server cluster consisting of multiple servers, or a cloud server.

[0028] Combination Figure 1 As shown, the above-mentioned method for processing business data can be executed by an electronic device, which can be a terminal device or a server. The above-mentioned method for processing business data can be implemented by the terminal device or the server respectively, or by the terminal device and the server together.

[0029] The above is only an example and is not specifically limited in this embodiment.

[0030] Optionally, as an optional implementation, as Figure 2 As shown, the method for processing the above business data includes:

[0031] S202, in response to the first account performing the target interactive operation, jumping to the first application, obtaining a first image currently displayed by the first application and a business process template associated with the first application, wherein the first image is used to indicate that the first account is obtaining business data associated with the target business, and the business process template is used to indicate image features corresponding to process nodes of the target business;

[0032] Optionally, in the embodiment of the present application, the above target interactive operation refers to any operation initiated by the user at the client foreground, which is intended to use the first application to obtain or upload business data, including but not limited to clicking a specific button on the screen, performing a sliding operation, entering a search keyword, etc. The above business process template covers all key steps from the start to the completion of the target business, ensuring that the system can accurately identify the business process position of the user.

[0033] It should be noted that the acquisition of the first image and the matching of the business process template can be achieved in a variety of ways. For example, the first image can be captured in real time through screen sharing technology, and the matching of the business process template can be completed through an image recognition algorithm or a template matching algorithm. This application is not limited to this, and any technical means that can achieve the above goals are within the scope of protection of the present invention.

[0034] For example, when the first account performs a target interactive operation, the system background detects and records this operation, and then automatically jumps to the first application; during the operation of the first application, the first image displayed by the first application is continuously obtained and compared with the preset business process template in real time; by matching the image features, the system background can determine whether the first account is obtaining business data according to the expected process node, thereby ensuring the accuracy and completeness of the information obtained. This process is not only applicable to a single business scenario, but can also be flexibly expanded to a variety of different business types.

[0035] S204, querying whether there is a target template in the business process template based on the first image and whether the first application is running in the system foreground, wherein the target template and the image features of the first image meet a preset matching condition;

[0036] Optionally, in an embodiment of the present application, the above-mentioned target template refers to the expected feature template of the first image used for comparison and identification in the business process template, which includes but is not limited to color distribution, element layout, text style, and specific interface elements, etc., which is used to determine whether the user is in the correct business process node.

[0037] It should be noted that the matching conditions between the target template and the first image can be implemented in a variety of ways, for example, based on pixel-level difference analysis, feature point matching algorithm, classification prediction of machine learning models, etc. At the same time, there are also various methods for detecting whether the first application is running in the foreground of the system. It can be by using the API provided by the operating system to monitor the activity status of the application, or by periodically detecting the hierarchical position of the application window. This application does not limit the specific matching algorithms and detection methods. Any technical solution that can effectively achieve matching between the target template and the first image, and determine whether the first application is running in the foreground is covered here.

[0038] For example, the system background queries whether there is a target template that meets the preset matching conditions in the business process template based on the first image acquired in real time. At the same time, the system also needs to continuously detect whether the first application is running in the system foreground to ensure the continuity and authenticity of the user's operation. If the matching conditions between the first image and the target template are met, and the first application is indeed running in the foreground, the system will assume that the user is acquiring business data in accordance with the prescribed process, so that subsequent data processing operations can be further performed.

[0039] On the contrary, if the matching fails or it is detected that the first application is running in the background, the system will determine that the user operation is abnormal, which may involve cheating or misoperation, and then take corresponding processing strategies, such as prompting the user to re-execute the target interaction operation.

[0040] S206: When the target template exists and the first application is always running in the system foreground, a text recognition operation is performed on the first image to determine business data.

[0041] Optionally, in an embodiment of the present application, the above-mentioned text recognition operation includes but is not limited to the process of parsing and identifying the text content in the first image that successfully matches the target template using OCR (optical character recognition) technology or other image text recognition algorithms, and extracting business data related to the target business from the first image, including but not limited to numbers, letters, tabular data, text descriptions, etc.

[0042] It should be noted that there can be many specific ways to determine business data. For example, the system may need to perform further logical checks on the recognized text to confirm whether the format of the data is correct; it may also need to combine the context of the user's operation to determine whether the text content meets the expectations of the business scenario; it can also determine the accuracy of the newly acquired data by comparing it with the existing business data in the database. At the same time, the execution frequency and timing of the text recognition operation can also be flexibly adjusted, such as executing it immediately after the user completes a specific operation, or uniformly recognizing and processing the collected images at the end of the entire business process. This application does not limit the specific method and timing of determining business data, and any strategy that can ensure the accuracy and completeness of business data is applicable to this technical solution.

[0043] For example, when the target template is confirmed to match the first image correctly, and it is detected that the first application is always running in the system foreground during the user's target interaction operation, the system will perform a text recognition operation on the first image. This process first parses the text information in the image through a preset algorithm, and then compares the recognized business data with the expected data format or context logic in the business process template to ensure the accuracy of the data.

[0044] In an exemplary embodiment, Figure 3 is a schematic diagram of an optional method for processing business data according to an embodiment of the present application. The implementation process of the present application is as follows Figure 3 shown.

[0045] Through the embodiment of the present application, in response to the first account executing the target interactive operation, jumping to the first application, obtaining the first image currently displayed by the first application and the business process template associated with the first application, wherein the first image is used to indicate that the first account is obtaining business data associated with the target business, and the business process template is used to indicate the image features corresponding to the process node of the target business; based on the first image, querying whether there is a target template in the business process template and whether the first application is running in the system foreground, wherein the image features of the target template and the first image meet the preset matching conditions; in the case where the target template exists and the first application is always running in the system foreground, performing a text recognition operation on the first image, determining the business data mode, combining real-time screen sharing, image feature matching and application running status monitoring, achieving the purpose of significantly improving the accuracy of business data processing, and effectively avoiding data deviations that may be caused by user misoperation, switching applications or tampering with information. In this way, the technical effect of greatly improving business processing efficiency and decision-making quality on the basis of ensuring the authenticity and compliance of user information is achieved, thereby solving the technical problem of low accuracy of business data processing.

[0046] As an optional solution, in response to the first account executing the target interactive operation and jumping to the first application, the method also includes: recording the start time of launching the first application; detecting the running status of the first application; and recording the end time of exiting the first application when the running status indicates that the execution of the target business has ended.

[0047] Optionally, in an embodiment of the present application, the above-mentioned running status refers to the system's monitoring result of the activity status of the first application, which includes but is not limited to foreground running, background running, paused, exited and other states, and is used to determine whether the target business is being executed or has been completed. The above-mentioned starting time refers to the moment corresponding to the key frame corresponding to the business data of the above-mentioned target business, rather than the moment of entering the first application. For example, the user enters the first application and clicks the "Business Data Query" button after a period of time. The start time is recorded at this time. The above-mentioned end time refers to the moment when the business query ends and the first application is exited. If the user ends the query but does not exit the first application, then the above-mentioned end time will be determined as the key frame moment corresponding to the end of the business query.

[0048] It should be noted that there are many ways to record the start time and end time of the first application. For example, the timestamp can be captured through the operating system API; when it is detected that the first application exits or switches to the background, the timestamp is also recorded to obtain a complete business execution time window. In addition, the running status of the first application can also be detected with the help of various means, including but not limited to monitoring the life cycle events of the application, querying the hierarchical status of the application window, sending heartbeat packets to detect application responses, etc. This application does not limit the specific time recording method and running status detection strategy. Any technical solution that can accurately define the target business execution time period and application status is covered here.

[0049] In an exemplary embodiment, taking the application scenario of the fund A query service as an example, when the first account performs the target interactive operation in the client foreground, that is, clicks the "Fund A Query" button on the screen, the system responds to this operation and jumps to the first application, that is, the Fund A Query official application. At this time, the user enters the Fund A Query official application, and the Fund A Query official application displays an image including the prompt message "Do you want to query Fund A?". The moment when the user clicks to confirm the query and enters the query interface of Fund A is determined as the above-mentioned starting time, and then the running status of the first application is continuously monitored. When the user completes the query operation, that is, sees the query results and is ready to return to the client, the system detects that the running status of the first application indicates that the target business execution has ended, and then records the end time of exiting the first application, thereby fully recording the time period when the user queries the Fund A information.

[0050] Through the embodiment of the present application, by real-time monitoring of the running status of the first application and recording the business execution time window, effective monitoring of the integrity and compliance of user operations is achieved, thereby achieving the purpose of ensuring that business data is truly obtained within a specified time period.

[0051] As an optional scheme, the above method also includes: determining the recording time period according to the above starting time and the above ending time; obtaining the display time period of the target screen, wherein the above target screen is used to display the above first image, and the above display time period is determined by the time when the above target screen starts to display the first frame image of executing the above target business and the time when the above target screen ends to display the last frame image of executing the above target business; when the above target template exists, the above first application runs in the system foreground, and the above display time period and the above recording time period meet the preset time period conditions, generate a first prompt message, wherein the above first prompt message indicates that the above business data can be used to indicate the business status of the above target business corresponding to the above first account.

[0052] Optionally, in the embodiment of the present application, the above-mentioned recording period refers to the time interval defined by the start time and the end time, which is used to represent the entire time period from the user launching the first application to completing the target business operation. The display period refers to the time period spanned by the target screen starting to display the first frame image of the target business to the end of the display of the last frame image of the target business, which is used to identify the duration of the target business actually presented on the screen.

[0053] In an exemplary embodiment, taking the application scenario of personal credit report query as an example, when the first account performs the target interactive operation in the client foreground, that is, starts the online query of the credit report, the system records the start time of this operation. Subsequently, the system continuously obtains the display period of the target screen, that is, the period from the display of the first frame credit report page to the display of the end of the last frame report. In this process, the system also detects whether the first application (credit report query application) is always running in the system foreground, and whether the target template exists. If the target template exists, the first application runs in the system foreground, and the display period and the recording period meet the preset period conditions (such as the two periods completely overlap, or the overlap ratio exceeds 95%), the system will generate a first prompt message, which indicates that the business data (credit report content) can be used to indicate the credit status corresponding to the first account, that is, the system confirms the authenticity and integrity of the credit report.

[0054] Through the embodiments of the present application, by recording and comparing the display period and the recording period, combined with target template matching and application operation status detection, comprehensive monitoring of the business data collection process is achieved, ensuring the timeliness and authenticity of the data, and achieving the purpose of improving the accuracy and trust of business processing.

[0055] As an optional scheme, the above-mentioned method queries whether a target template exists in the above-mentioned business process template based on the above-mentioned first image and whether the above-mentioned first application is running in the system foreground, and the above-mentioned method also includes at least one of the following: when the above-mentioned target template does not exist, generating a second prompt information, wherein the above-mentioned second prompt information is used to indicate that the above-mentioned business data needs to be re-acquired; when the above-mentioned first application switches from running in the system foreground to the system background, generating a third prompt information, wherein the above-mentioned third prompt information is used to indicate that the above-mentioned business data needs to be re-acquired.

[0056] Optionally, in the embodiment of the present application, the second prompt information and the third prompt information refer to messages generated by the system for feedback to the user, reminding the user that the current operation is invalid or abnormal and needs to be re-executed to obtain business data. These messages can be in the form of text pop-ups, voice prompts, mobile phone vibration feedback, etc., including but not limited to warnings that clearly indicate user operation errors, buttons that provide restart operations, or instructions to guide users to contact customer service for help.

[0057] It should be noted that the situation where the system background detects that the target template does not exist or the first application is switched from the system foreground to the background may be caused by a variety of reasons, including but not limited to user misoperation, third-party application abnormalities, network delays, etc. When generating the second prompt information or the third prompt information, a more specific error cause description or solution steps can be provided to help users correct the error more quickly and reacquire business data. In addition, the presentation method and content of the prompt information can be flexibly adjusted according to the specific application scenario and user interface design, and this application does not limit this.

[0058] In an exemplary embodiment, taking the application scenario of personal income tax information query as an example, when the user performs the target interactive operation in the client foreground, that is, starts to use the first application (personal income tax query application) to query, the system will obtain the first image displayed by the first application in real time and match it with the business process template. If at a certain moment, the system finds that the first image does not match any target template in the business process template, that is, the user may not follow the instructions, then a second prompt message will be generated, indicating that the business data (personal income tax information) currently obtained by the user is invalid and needs to be re-acquired. Similarly, if the system detects that the first application (personal income tax query application) is switched from the system foreground to the system background, that is, the user may switch to other applications during the query process, the system will generate a third prompt message, which is also used to indicate that the business data (personal income tax information) currently obtained by the user is invalid and needs to be restarted.

[0059] Through the embodiments of the present application, a method of real-time detection of the target template matching status and the application foreground status is adopted to achieve abnormal detection and timely feedback of the user operation process, thereby achieving the purpose of preventing user cheating and ensuring the authenticity and integrity of business data. At the same time, it also enhances the user experience and avoids repeated work caused by user operational errors.

[0060] As an optional solution, the method further includes: when generating at least one of the second prompt information and the third prompt information, adding a feedback tag to the first image, wherein the feedback tag is used to indicate that the first image does not include the business data; and storing the first image to the target database.

[0061] Optionally, in the embodiment of the present application, the feedback tag refers to the marking information associated with the first image, which is used to indicate whether the image contains valid business data or its status is abnormal, and may include but is not limited to tags such as "invalid data", "template mismatch", "application switched to background", etc. The target database is a storage area for storing the first image and the feedback tags associated with it, which may be a local database or a database of a cloud server, including but not limited to a relational database, a non-relational database, etc.

[0062] It should be noted that there are many specific implementation methods for storing in the target database. For example, the first image and the feedback label can be stored as a key-value pair, or the image data and label information can be stored in different tables or collections respectively, and associated through a unique identifier. In addition, the timing of the storage operation can also be flexibly set. It can be stored immediately when the second prompt information or the third prompt information is generated, or the marked images can be stored uniformly after the entire business process is completed. This application does not limit the specific storage method and timing, and any technical solution that can effectively store the first image and the feedback label is applicable.

[0063] In an exemplary embodiment, taking the application scenario of provident fund information query as an example, when the system generates a second prompt message (indicating that business data needs to be reacquired because the target template does not exist) or a third prompt message (indicating that business data needs to be reacquired because the first application is switched from the foreground to the background), the system will add a feedback tag to the first image currently acquired. For example, if it is detected that the user operation does not match the template, the system may add a "template does not match" tag; if it is detected that the user switches to other applications during the operation, the system may add a "application switched to the background" tag. Subsequently, the system stores the first image with the feedback tag in the target database, and these labeled images can be used as the data basis for subsequent analysis of user behavior, optimization of business process templates, or adjustment of system monitoring strategies.

[0064] Through the embodiment of the present application, by adding feedback tags to the first image and storing them in the target database, effective recording and classification of abnormal operation images are achieved, achieving the purpose of optimizing business processes and improving the intelligent monitoring capabilities of the system. This mechanism not only helps to analyze user behavior patterns, but also provides data support for the self-learning and optimization of the system, and enhances the ability to handle complex user operation scenarios.

[0065] As an optional solution, the above-mentioned acquisition of the first image currently displayed by the above-mentioned first application and the business process template associated with the above-mentioned first application includes: upon obtaining the application authorization of the above-mentioned first account to the above-mentioned first application, receiving the video stream pushed in real time by the above-mentioned first application, wherein the above-mentioned video stream is composed of each image frame displayed by the above-mentioned first application; taking at least one of the timed screenshot operation or the frame-by-frame screenshot operation from the above-mentioned video stream to obtain the above-mentioned first image.

[0066] Optionally, in an embodiment of the present application, the above-mentioned application authorization refers to the user granting permission to use the first application through the first account, including but not limited to the user turning on screen sharing, screen recording or data access rights in the settings. The above-mentioned video stream is a continuous data stream composed of individual image frames displayed by the first application, which can be transmitted from the client to the server through real-time streaming technology. The above-mentioned timed screenshot operation or frame-by-frame screenshot operation refers to the system capturing images from the received video stream frame by frame at a set time interval or according to the image frame, as the first image for subsequent processing.

[0067] It should be noted that the method of acquiring the video stream may vary depending on the client device and the first application, including but not limited to using the screen recording function at the operating system level, the screenshot mechanism within the first application, or the real-time streaming media capture service provided by a third-party SDK. Similarly, the frequency and timing of the screenshot operation can be adjusted according to business needs and network conditions, which can be either timed screenshots at fixed time intervals or frame-by-frame screenshots of specific image frames. This application does not limit the specific technology for video stream acquisition and the specific algorithm for screenshot operations.

[0068] In an exemplary embodiment, taking the application scenario of online inquiry of personal credit report as an example, when the user authorizes to turn on the screen sharing function through the first account, the system starts to receive the video stream pushed in real time by the first application (credit report inquiry application). As the user performs the inquiry operation of the credit report in the first application, the system can perform a timed screenshot operation from the video stream at a set time interval, for example, once per second, to capture the currently displayed first image. Alternatively, the system can also take screenshots of each image frame in the video stream in a frame-by-frame manner to obtain a more detailed first image. These first images will be used for subsequent business process template matching and information recognition.

[0069] Through the embodiments of the present application, by receiving the video stream in real time and flexibly using the timed screenshot or frame-by-frame screenshot technology, the screen content of the first application operated by the user is accurately captured, thereby achieving the purpose of monitoring the user operation process, identifying business data and preventing data tampering.

[0070] As an optional solution, the above method also includes: starting the above-mentioned first application; when the target acquisition operation is triggered in the above-mentioned first application, causing the target screen to display the above-mentioned first image, wherein the above-mentioned target screen is used to display the above-mentioned first image, and the above-mentioned target acquisition operation is used to indicate the acquisition of the above-mentioned business data; acquiring the above-mentioned first image and the above-mentioned business process template.

[0071] Optionally, in an embodiment of the present application, the target screen refers to the screen for user interactive operations, that is, the display interface of the client device, including but not limited to mobile phone screens, tablet screens, etc. The first image is a screenshot or screenshot of the target screen when performing a target acquisition operation, which is used for subsequent business data identification and template matching. The business process template is a preset image template or operation sequence template used to compare the first image to determine whether the user operates according to the expected process, including but not limited to screenshots of each key step, operation paths, element layouts, etc.

[0072] It should be noted that the target acquisition operation can be triggered in a variety of ways, including but not limited to the user clicking a specific button on the screen, entering a specific instruction, or through a voice command. There are also many possibilities for obtaining the first image. For example, the current screen content can be obtained directly from the target application through the screen recording function or screenshot function of the client device, or by using an application program interface (API). The establishment and update of business process templates can be based on historical data, user feedback, or changes in business needs to adapt to different business scenarios and operating environments. This application does not limit the specific implementation details of the above operations.

[0073] In an exemplary embodiment, taking the application scenario of personal credit report query as an example, when the user selects the "start query" target interaction operation on the client device, the system starts the first application, namely the personal credit report query application. In the first application, when the user triggers the target acquisition operation, such as clicking the "query" button, the system immediately captures the display content of the target screen and generates a first image. Subsequently, the system obtains a business process template, which contains the expected screen layout and elements in the process of querying the credit report, and is used to compare the first image to determine whether the user operation is performed as expected.

[0074] Through the embodiments of the present application, by launching the first application in real time and monitoring the content displayed on the target screen, accurate monitoring of the user operation process and instant acquisition of business data are achieved, thereby achieving the purpose of ensuring compliance of the data collection process, preventing user cheating, and improving business processing efficiency.

[0075] In an exemplary embodiment, taking the financial industry as an example, in the current To C (consumer-oriented) services, it is usually necessary to introduce external customer information to enrich user portraits as an important basis for business handling decisions. The embodiment of the present application can enable users to actively share their desktops and personally conduct corresponding information inquiries and "upload" relevant personal information under the premise of user authorization. In the process of users sharing their screens, real-time prompts are given to users, and technical means are used to ensure that users do not cheat in the process. Specifically:

[0076] 1) The client (mobile phone or other terminal device) foreground jumps to the corresponding function interface of other APP (the first application mentioned above) through an external link, and performs step-by-step operations according to the voice prompts or pop-up window guidance of the client background. If each step of the operation exceeds the time limit, an error will be prompted. In addition, other APPs record whether the user is always running in the system foreground during the operation, and the corresponding start and end time points.

[0077] 2) The client background starts sharing the screen from the jump action, pushes the video stream of the current client shared screen to the server, pushes the video stream of the shared screen to the server through the corresponding screen sharing function, and provides voice or pop-up prompts at each node.

[0078] 3) On the server side, the received video stream is captured periodically / frame by frame and sent to the interface judgment module. If it is detected that the shared screenshot matches the defined template, the current user is determined to be at the corresponding process node and a voice operation prompt is given to the user; if the shared screenshot does not match any defined template, a new screenshot is taken and a voice prompt is given.

[0079] Among them, for the interface judgment function, the template can be defined in advance by recording the existence and relative position of key elements in the interface of the business process. During the judgment process, the match between the screen interface and the template is judged based on the comparison of key elements. After the entire business process is executed, a pop-up window and voice prompt can be used to jump back to the original APP and perform the next step.

[0080] More specifically, on the server side, when the interface is judged as mismatched, it is considered that the user did not operate according to the regulations and may be suspected of cheating. The corresponding pictures are stored as feedback data, which can be used for subsequent model optimization. The user is prompted that the current operation has failed and needs to be re-recorded; after the user's business processing flow is completed, the records and time periods of whether other APPs of the original APP have always been running in the foreground during the business processing are queried. If after comparing the query results, it is judged that the user has switched to the background during the corresponding process, it is considered that the user did not operate according to the regulations and may be suspected of cheating, and the user is prompted that the current operation has failed and needs to be re-recorded; for the judgment of interface matching, the judgment starts from the key frame of the first page in the business process to reduce the configuration workload of the service and configuration templates; for the key screenshots that are identified for text recognition, they are stored after deduplication, that is, the interface matching at the image level is performed first, and then text recognition is performed after passing, which reduces the pressure on the text recognition service.

[0081] It should also be noted that in the embodiments of the present application, when achieving the goals of user screen sharing and information recognition, it is not necessary to use screen recording, because the present application adopts different technical paths, such as real-time screenshots, image recognition, application status monitoring, etc., to achieve the purpose of ensuring the authenticity and integrity of user operations and preventing cheating, without the need to record the video of the user's screen operations throughout the process, so as to reduce the amount of data transmission, reduce the demand for computing resources, improve processing efficiency, and avoid recording unnecessary video content.

[0082] Through the embodiments of the present application, a method of "uploading" data by users sharing the screen is adopted, which avoids the impact on user information during the public-to-public information query process, and is more "friendly" to some customers; it avoids the relevant costs of public-to-public inquiries; it ensures the authenticity of user-reported data on a technical level, and avoids user tampering and cheating; it can be applied to various businesses. For example, after the function of the credit report is developed, only the corresponding template or identification field needs to be changed, and it can be reused in the query of other information such as provident fund and social security.

[0083] It is understandable that in the specific implementation of this application, related data such as user information is involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0084] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0085] According to another aspect of the embodiment of the present application, a device for processing business data for implementing the above-mentioned business data processing method is also provided. Figure 4 As shown, the device comprises:

[0086] The acquisition module 402 is used to jump to the first application in response to the first account performing the target interactive operation, and acquire the first image currently displayed by the first application and the business process template associated with the first application, wherein the first image is used to indicate that the first account is acquiring business data associated with the target business, and the business process template is used to indicate the image features corresponding to the process nodes of the target business;

[0087] A query module 404 is used to query whether a target template exists in the business process template and whether the first application is running in the system foreground based on the first image, wherein the target template meets a preset matching condition with the image features of the first image;

[0088] The determination module 406 is used to perform a text recognition operation on the first image to determine the business data when the target template exists and the first application is always running in the system foreground.

[0089] As an optional solution, the above-mentioned device is also used to: in response to the first account executing the target interactive operation, after jumping to the first application, record the start time of launching the first application; detect the running status of the first application; when the running status indicates that the execution of the target business has ended, record the end time of exiting the first application.

[0090] As an optional solution, the above-mentioned device is also used to: determine the recording time period according to the start time and the end time; obtain the display time period of the target screen, wherein the target screen is used to display the first image, and the display time period is determined by the time when the target screen starts to display the first frame image of executing the target business and the time when the target screen ends to display the last frame image of executing the target business; when the target template exists, the first application runs in the system foreground, and the display time period and the recording time period meet the preset time period conditions, generate a first prompt message, wherein the first prompt message indicates that the business data can be used to indicate the business status of the target business corresponding to the first account.

[0091] As an optional solution, the above-mentioned device is used to query whether there is a target template in the business process template and whether the first application is running in the system foreground based on the first image in the following manner, including at least one of the following: when the target template does not exist, generating a second prompt information, wherein the second prompt information is used to indicate that the business data needs to be reacquired; when the first application switches from running in the system foreground to the system background, generating a third prompt information, wherein the third prompt information is used to indicate that the business data needs to be reacquired.

[0092] As an optional solution, the above-mentioned device is also used to: add a feedback tag to the first image when generating at least one of the second prompt information and the third prompt information, wherein the feedback tag is used to indicate that the first image does not include business data; and store the first image in the target database.

[0093] As an optional solution, the above-mentioned device is used to obtain the first image currently displayed by the first application and the business process template associated with the first application in the following manner: upon obtaining the application authorization of the first account for the first application, receiving the video stream pushed in real time by the first application, wherein the video stream is composed of individual image frames displayed by the first application; taking at least one of a timed screenshot operation or a frame-by-frame screenshot operation from the video stream to obtain the first image.

[0094] As an optional solution, the above-mentioned device is also used to: start a first application; when a target acquisition operation is triggered in the first application, cause the target screen to display a first image, wherein the target screen is used to display the first image, and the target acquisition operation is used to indicate the acquisition of business data; and acquire the first image and a business process template.

[0095] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0096] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0097] According to one aspect of the present application, a computer program product is provided. The computer program product includes a computer program.

[0098] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0099] Figure 5 The structure block diagram of a computer system for implementing an electronic device according to an embodiment of the present application is schematically shown.

[0100] It should be noted that Figure 5 The computer system 500 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0101] like Figure 5 As shown, the computer system 500 includes a central processing unit 501 (CPU), which can perform various appropriate actions and processes according to the program stored in the read-only memory 502 (ROM) or the program loaded from the storage part 508 to the random access memory 503 (RAM). Various programs and data required for system operation are also stored in the random access memory 503. The central processing unit 501, the read-only memory 502 and the random access memory 503 are connected to each other through a bus 504. The input / output interface 505 (Input / Output interface, i.e., I / O interface) is also connected to the bus 504.

[0102] The following components are connected to the input / output interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a local area network card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the input / output interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed so that a computer program read therefrom is installed into the storage section 508 as needed.

[0103] In particular, according to an embodiment of the present application, the process described in each method flow chart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program contains a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the central processor 501, various functions defined in the system of the present application are executed.

[0104] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the central processor 501, various functions provided by the embodiment of the present application are performed.

[0105] According to another aspect of the embodiment of the present application, an electronic device for implementing the above-mentioned business data processing method is also provided. The electronic device may be Figure 1 The terminal device or server shown in the figure. This embodiment is described by taking the electronic device as a terminal device as an example. Figure 6 As shown, the electronic device includes a memory 602 and a processor 604. The memory 602 stores a computer program, and the processor 604 is configured to execute the steps in any of the above method embodiments through the computer program.

[0106] Optionally, in this embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.

[0107] Optionally, in this embodiment, the above-mentioned processor can be configured to execute the methods in each embodiment of the present application through a computer program.

[0108] Alternatively, a person skilled in the art may understand that: Figure 6 The structure shown is for illustration only. Figure 6 The structure of the electronic device is not limited. Figure 6 More or fewer components (such as network interfaces, etc.) as shown in, or with Figure 6 Different configurations shown.

[0109] Among them, the memory 602 can be used to store software programs and modules, such as program instructions / modules corresponding to the business data processing method and device in the embodiments of the present application. The processor 604 executes various functional applications and data processing by running the software programs and modules stored in the memory 602, that is, realizing the above-mentioned business data processing method. The memory 602 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 602 may further include a memory remotely located relative to the processor 604, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Among them, the memory 602 can be specifically, but not limited to, used to store information such as business data. As an example, such as Figure 6 As shown, the memory 602 may include but is not limited to the acquisition module 402, query module 404 and determination module 406 in the business data processing device. In addition, it may also include but is not limited to other module units in the business data processing device, which will not be repeated in this example.

[0110] Optionally, the transmission device 606 is used to receive or send data via a network. Specific examples of the network may include a wired network and a wireless network. In one example, the transmission device 606 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one example, the transmission device 606 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0111] In addition, the electronic device further includes: a display 608 for displaying the business data; and a connection bus 610 for connecting various module components in the electronic device.

[0112] In other embodiments, the terminal device or server may be a node in a distributed system, wherein the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting the multiple nodes through network communication. The nodes may form a peer-to-peer network, and any form of computing device, such as a server, terminal or other electronic device, may become a node in the blockchain system by joining the peer-to-peer network.

[0113] According to one aspect of the present application, a computer-readable storage medium is provided, and a processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the business data processing method provided in various optional implementation methods of the above-mentioned business data processing.

[0114] Optionally, in this embodiment, the above-mentioned computer-readable storage medium can be configured to store data for executing the methods in various embodiments of the present application.

[0115] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0116] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0117] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for one or more electronic devices to execute all or part of the steps of the methods described in each embodiment of the present application.

[0118] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0119] In the several embodiments provided in the present application, it should be understood that the disclosed application can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0120] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0121] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0122] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for processing business data, characterized in that: include: In response to the first account performing a target interactive operation, jumping to the first application, acquiring a first image currently displayed by the first application and a business process template associated with the first application, wherein the first image is used to indicate that the first account is acquiring business data associated with a target business, and the business process template is used to indicate an image feature corresponding to a process node of the target business; Based on the first image, query whether there is a target template in the business process template and whether the first application is running in the system foreground, wherein the target template and the image features of the first image meet a preset matching condition; In the case where the target template exists and the first application is always running in the foreground of the system, a text recognition operation is performed on the first image to determine the business data.

2. The method according to claim 1, characterized in that In response to the first account performing the target interactive operation and jumping to the first application, the method further includes: Recording the start time of starting the first application; Detecting the running status of the first application; When the running status indicates that the execution of the target service is finished, the end time of exiting the first application is recorded.

3. The method according to claim 2, characterized in that The method further comprises: Determine the recording period according to the start time and the end time; Acquire a display period of a target screen, wherein the target screen is used to display the first image, and the display period is determined by the moment when the target screen starts to display a first frame image for executing the target service and the moment when the target screen ends to display a last frame image for executing the target service; When the target template exists, the first application runs in the system foreground, and the display period and the recording period meet the preset period conditions, a first prompt message is generated, wherein the first prompt message indicates that the business data can be used to indicate the business status of the target business corresponding to the first account.

4. The method according to claim 1, characterized in that: The method of querying whether a target template exists in the business process template and whether the first application is running in the system foreground based on the first image further includes at least one of the following: If the target template does not exist, generating second prompt information, wherein the second prompt information is used to indicate that the service data needs to be re-acquired; When the first application switches from running in the system foreground to the system background, a third prompt message is generated, wherein the third prompt message is used to indicate that the business data needs to be re-acquired.

5. The method according to claim 4, characterized in that The method further comprises: When at least one of the second prompt information and the third prompt information is generated, adding a feedback tag to the first image, wherein the feedback tag is used to indicate that the first image does not include the business data; The first image is stored in the target database.

6. The method according to claim 1, characterized in that The acquiring the first image currently displayed by the first application and the business process template associated with the first application includes: In the case of obtaining application authorization of the first account to the first application, receiving a video stream pushed in real time by the first application, wherein the video stream is composed of various image frames displayed by the first application; At least one of a timed screenshot operation and a frame-by-frame screenshot operation is performed from the video stream to obtain the first image.

7. The method according to claim 1, characterized in that The method further comprises: Starting the first application; In a case where a target acquisition operation is triggered in the first application, causing a target screen to display the first image, wherein the target screen is used to display the first image, and the target acquisition operation is used to instruct to acquire the business data; The first image and the business process template are acquired.

8. A device for processing business data, characterized in that: include: an acquisition module, configured to jump to a first application in response to a first account executing a target interactive operation, and acquire a first image currently displayed by the first application and a business process template associated with the first application, wherein the first image is used to indicate that the first account is acquiring business data associated with a target business, and the business process template is used to indicate an image feature corresponding to a process node of the target business; A query module, used for querying whether a target template exists in the business process template and whether the first application is running in the system foreground based on the first image, wherein the target template satisfies a preset matching condition with the image features of the first image; A determination module is used to perform a text recognition operation on the first image to determine the business data when the target template exists and the first application is always running in the foreground of the system.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein the computer program can be executed by an electronic device to perform the method described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.

11. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 7 through the computer program.

Images