Real-time service monitoring and precise abnormality diagnosis system

Abstract

The invention provides a real-time service monitoring and precise abnormality diagnosis system, and relates to the technical field of data processing, and the system comprises an acquisition module which carries out the image collection of an interaction position of a fixed marketing end, and obtains an interaction video; the extraction module is used for extracting a first interaction feature of a first interface with a service corresponding to the screen in the interaction video; the recognition module is used for recognizing the attribute characteristics of the first interaction personnel to obtain a first pre-estimated attribute and recognizing the interaction characteristics of the first interaction personnel to obtain a second interaction characteristic; and the diagnosis module performs combined verification diagnosis based on the first interaction feature, the first pre-estimation attribute and the second interaction feature to obtain an abnormality diagnosis result of the fixed marketing end or the first interaction personnel, so that services can be monitored in real time, multi-dimensional data can be subjected to conjoint analysis, and the accuracy of abnormality diagnosis is improved.

Description

Technical Field

[0001] The present invention relates to data processing technology, and in particular to a real-time business monitoring and precise anomaly diagnosis system. Background Art

[0002] In digital marketing and business interaction scenarios, fixed marketing terminals (such as smart questionnaire terminals and service application devices) are increasingly used. For example, fixed marketing terminals deployed in shopping malls, communities, and other places need to collect user demand data in real time, but they often face multiple challenges in actual operations. Teenagers mistakenly touch adult questionnaires, the elderly mistakenly select options due to vision problems, and device touch failure resulting in unresponsive operations. These anomalies not only affect the accuracy of data collection, but may also lead to business process interruptions or a decline in user experience.

[0003] In existing technologies, traditional monitoring methods can only record operation results, but cannot capture the details of the interaction process in real time. It is difficult to quickly locate the source of the anomaly, resulting in uneven data quality and delayed fault response.

[0004] Therefore, how to monitor business in real time, conduct joint analysis of multi-dimensional data, and improve the accuracy of abnormal diagnosis has become an urgent problem that needs to be solved. Summary of the Invention

[0005] The present invention provides a real-time business monitoring and accurate anomaly diagnosis system, which can monitor business in real time, jointly analyze multi-dimensional data, and improve the accuracy of anomaly diagnosis.

[0006] A first aspect of the present invention provides a real-time business monitoring and accurate anomaly diagnosis system, comprising: The acquisition module collects images of the interaction locations of fixed marketing terminals to obtain interaction videos; An extraction module extracts a first interaction feature of a first interface having a service corresponding to the screen in the interactive video; an identification module that identifies the attribute characteristics of the first interacting person to obtain a first estimated attribute, and identifies the interaction characteristics of the first interacting person to obtain a second interaction characteristic; The diagnosis module performs combined verification diagnosis based on the first interaction feature, the first estimated attribute, and the second interaction feature to obtain an abnormal diagnosis result of a fixed marketing terminal or the first interaction person.

[0007] Optionally, in a possible implementation of the first aspect, capturing an image at an interactive position of the fixed marketing terminal to obtain an interactive video includes: After determining that the fixed marketing terminal is triggered, the image acquisition device acquires images of the fixed marketing terminal from top to bottom according to a preset angle; The screen edge area of ​​the fixed marketing terminal in the image is extracted and captured to obtain the interactive video.

[0008] Optionally, in a possible implementation of the first aspect, extracting a first interaction feature of a first interface having a service corresponding to the screen in the interactive video includes: Extracting text that is a first color feature in the interactive video to obtain a first sub-feature, where the first color is the title color; Extracting text that is a second color feature in the interactive video to obtain a second sub-feature, where the second color is an option color; The combined first interactive feature is obtained by analyzing the feature relationship between the first sub-feature and the second sub-feature.

[0009] Optionally, in a possible implementation of the first aspect, analyzing the feature relationship between the first sub-feature and the second sub-feature to obtain the combined first interaction feature includes: Determining a first recognition direction of the interactive video, where the first recognition direction is a top-to-bottom direction of a fixed marketing terminal; The first sub-feature and the second sub-feature are sequentially identified based on the first recognition direction, and text extraction is performed based on OCR. When it is determined that all the first sub-features and the second sub-features have been identified, or when it is determined that the text in the first sub-feature and the second sub-feature cannot be identified, the recognition is stopped to obtain a feature recognition sequence; The feature recognition sequences are combined and marked to obtain a plurality of first interaction features.

[0010] Optionally, in a possible implementation of the first aspect, combining and marking the feature recognition sequences to obtain a plurality of first interaction features includes: If it is determined that the first feature of the feature recognition sequence is not the first sub-feature, the feature recognition sequence is determined to have failed the correctness verification, and the interactive video is acquired again; If the feature recognition sequence is judged to pass the correctness verification, the first sub-features and the second sub-features of adjacent serial numbers are sequentially extracted and combined to obtain a first interactive feature; Add a separator mark to the first interactive feature of the last sequence number.

[0011] Optionally, in a possible implementation of the first aspect, identifying the attribute characteristics of the first interacting person to obtain the first estimated attribute, and identifying the interaction characteristics of the first interacting person to obtain the second interaction characteristics, include: Recognizing attribute characteristics of the first interacting person based on the distance measuring device at the fixed marketing terminal to obtain a first estimated attribute of the height information; The finger position of the first interacting person is identified to obtain a second interaction feature with the finger position.

[0012] Optionally, in a possible implementation of the first aspect, identifying the finger location of the first interacting person to obtain a second interaction feature having the finger location includes: The first interacting person's finger sleeve is provided with a first identification object, wherein the first identification object is a finger sleeve with a third color characteristic; The position of the first identification object is located and identified to obtain an identification track, a trigger point is generated when it is determined that the first identification object is in contact with the screen, and a corresponding second interaction feature is generated based on the identification track and the trigger point.

[0013] Optionally, in a possible implementation of the first aspect, performing combined verification diagnosis based on the first interaction feature, the first estimated attribute, and the second interaction feature to obtain an abnormality diagnosis result for a fixed marketing terminal or the first interacting person includes: Combining and classifying the first interaction feature, the first estimated attribute, and the second interaction feature to obtain multiple verification groups; The verification logic corresponding to each verification group is called to obtain the corresponding abnormal diagnosis result.

[0014] Optionally, in a possible implementation of the first aspect, the calling of the verification logic corresponding to each verification group to obtain a corresponding abnormality diagnosis result includes: Extracting a verification group corresponding to the first interaction feature and the second interaction feature; Determine the first interaction feature at the trigger point corresponding to the second interaction feature, and obtain first verification information triggered based on the video; The second verification information of the first interactive feature corresponding to the fixed marketing terminal is extracted in real time. If the first verification information and the second verification information do not correspond, an abnormal diagnosis result of the fixed marketing terminal is generated and an option selection is performed based on the second interactive feature.

[0015] Optionally, in a possible implementation of the first aspect, selecting an option based on the second interaction feature includes: The number of times the first identifier touches the option in the second interactive feature is determined, and if the number of touches is greater than a preset number, the corresponding option is used as the target option.

[0016] Optionally, in a possible implementation of the first aspect, the calling of the verification logic corresponding to each verification group to obtain a corresponding abnormality diagnosis result includes: Extracting a verification group corresponding to the first interaction feature, the first estimated attribute, and the second interaction feature; If the first interaction feature and the second interaction feature correspond to each other, extracting the corresponding option results; If the option result does not correspond to the first estimated attribute, an abnormal trigger tag corresponding to the fixed marketing end is generated and fed back to the fixed marketing end.

[0017] According to a second aspect of the present invention, a storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, it is used to implement the method of the first aspect of the present invention and various possible designs of the first aspect.

[0018] The beneficial effects of the present invention are as follows: 1. The present invention can monitor business in real time, jointly analyze multi-dimensional data, and improve the accuracy of abnormal diagnosis. First, after being triggered by a fixed marketing terminal, the present invention can capture interactive video at a preset angle through a camera, and capture the edge area of ​​the screen to generate an interactive video with focused content. The system separates the title and options by color features, and extracts the first and second sub-features in sequence along the top-down recognition direction to obtain a feature recognition sequence. When the sequence starts with a non-title feature, the video is automatically recaptured to ensure data validity.

[0019] 2. The present invention can integrate user attribute characteristics and interactive behavior characteristics to achieve matching and identification of the operating subject and the operating intention, thereby improving the accuracy of abnormality diagnosis. Specifically, the present invention can use ranging equipment to obtain user height information, and combine finger positioning technology to capture finger trigger points and movement trajectories in real time to quickly identify interface adaptation anomalies. By combining and analyzing multi-dimensional data, the user's physical actions are associated with operational behaviors, avoiding misjudgment of single-dimensional diagnosis and improving the accuracy of abnormality diagnosis.

[0020] 3. The present invention can realize accurate positioning and automatic processing of abnormal types based on the combined verification logic of multi-dimensional features. Among them, the present invention can verify the first interactive feature, the first estimated attribute, and the second interactive feature in groups, call the preset logic to eliminate interference factors, and when the first interactive feature has no abnormality but the finger trigger point is misaligned with the option position, that is, the second interactive feature is abnormal, combined with the user's first estimated attribute being normal, it can be determined that the device touch has failed, and the correct option is automatically triggered according to the finger trajectory to maintain the business process. If the option result does not match the user attribute, an abnormal trigger tag is generated and invalid data entry is blocked. Through the combined verification mechanism, not only can the abnormal types such as device failure, interface adaptation, and user misoperation be accurately distinguished, but also manual intervention can be reduced through automatic error correction, thereby improving the efficiency of abnormal response. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A schematic diagram of a real-time business monitoring and accurate anomaly diagnosis system provided by the present invention. DETAILED DESCRIPTION

[0022] The following specific embodiments are used to describe the technical solution of the present invention in detail. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0023] like Figure 1 FIG. 1 is a schematic diagram of a real-time business monitoring and accurate abnormality diagnosis system provided by the present invention, wherein the real-time business monitoring and accurate abnormality diagnosis system comprises: S1, the acquisition module, acquires images of the interaction position of the fixed marketing terminal to obtain the interaction video.

[0024] It should be noted that the personnel objects for data collection at different marketing ends are inconsistent. For example, some fixed marketing ends mainly collect information on the needs of adults, while some fixed marketing ends mainly collect information on the needs of teenagers. When people of the wrong age group fill in the data, it will affect the accuracy of data collection. In addition, some personnel will have operational deviations when filling in data information, such as the data filled in does not correspond to the question, which will reduce the accuracy of data collection. Therefore, the business process of collecting data can be monitored in real time and abnormal phenomena can be diagnosed in time.

[0025] It is understandable that in order to ensure the accuracy of the collected information, the identity of the person will be preliminarily identified. Then, when the person fills in the information, the interactive video of the fixed marketing end and the corresponding interactive position of the person will be collected to realize the identification of the person's identity information and monitor the interactive operations, so as to facilitate timely abnormal diagnosis.

[0026] Among them, the acquisition module is a system processing module for data acquisition, which provides identification information for subsequent monitoring and analysis. The fixed marketing end is a device for data acquisition at a fixed position. The interactive position is the location where the equipment and personnel interact with each other, that is, the location of the fixed marketing end. The interactive video is the video data collected by the acquisition device at the corresponding position of the fixed marketing end. For example, it can be video data of the fixed marketing end screen and personnel captured by a camera.

[0027] In some embodiments, a specific implementation of step S1 (capturing an image at the interaction position of the fixed marketing terminal to obtain an interaction video) includes: S11, after determining that the fixed marketing terminal is triggered, the image acquisition device acquires images of the fixed marketing terminal from top to bottom according to a preset angle.

[0028] It can be understood that when a signal is received that a person clicks on a fixed marketing terminal, the corresponding image acquisition device will capture an image of the fixed marketing terminal.

[0029] The image acquisition device is a device for performing image acquisition, such as a camera, and the preset angle is an angle for performing image acquisition in advance, which is pre-set.

[0030] S12, extracting and capturing the screen edge area of ​​the fixed marketing terminal in the image to obtain an interactive video.

[0031] It is understandable that when the collected data is subsequently identified and verified, only the content filled in by the personnel needs to be identified. Therefore, the final interactive video only needs to fix the screen area of ​​the marketing end, and the remaining redundant areas can be deleted, such as the blank area next to the screen. Then, the intercepted interactive video can be obtained, which is convenient for subsequent information identification.

[0032] Among them, the screen edge area is the boundary area of ​​the screen corresponding to the fixed marketing end.

[0033] S2, an extraction module extracts a first interaction feature of a first interface having a business corresponding to the screen in the interactive video.

[0034] It is understandable that the interactive interfaces of fixed marketing terminals, such as questionnaires and service applications, usually use color to distinguish between questions and options, such as black titles and blue options. By extracting text with different color features, the unstructured visual information in the video can be converted into structured "question-option" data, which makes it easier for the system to identify the specific content of user interaction. That is, by separating the question and option information through color features, structured interaction feature data is constructed to facilitate subsequent abnormality diagnosis.

[0035] The extraction module is a system processing module for extracting the first interaction feature, the first interface is a video frame interface with business information in the interactive video, and the first interaction feature is feature information in the first interface.

[0036] In some embodiments, a specific implementation of step S2 (extracting a first interactive feature of a first interface having a service corresponding to the screen in the interactive video) includes: S21, extracting text with a first color feature in the interactive video to obtain a first sub-feature, where the first color is the title color.

[0037] It is understandable that since the interactive video is a questionnaire information displayed on a fixed marketing end, the questions and options in the questionnaire have different colors to facilitate personnel to distinguish them. At the same time, when identifying the interactive video, information can also be identified based on color to facilitate determining whether there are any abnormalities in the current interactive video.

[0038] The first color is the color corresponding to the information title in the first interface, which may be pre-set, such as black, and the first sub-feature is the text title corresponding to the first color.

[0039] S22: extract text with a second color feature in the interactive video to obtain a second sub-feature, where the second color is an option color.

[0040] It is understandable that the second color is the color corresponding to the information option in the first interface, which can be pre-set, such as blue, and the second sub-feature is the text option corresponding to the first color.

[0041] S23, analyzing the feature relationship between the first sub-feature and the second sub-feature to obtain a combined first interaction feature.

[0042] It is understandable that since a question must have corresponding options, the first interaction feature can be determined based on the relationship between the identified first sub-feature and the second sub-feature, so as to integrate the first sub-feature and the second sub-feature of the same group into a complete interaction unit, match the user's actual operation scenario, and facilitate subsequent improvement of the efficiency of diagnosis.

[0043] It is not difficult to understand that questions and options have a fixed hierarchical relationship in the interface, such as questions on the top and options on the bottom. Therefore, the corresponding feature relationship can be determined according to the corresponding positions of different features.

[0044] In some embodiments, a specific implementation of step S23 (analyzing the feature relationship between the first sub-feature and the second sub-feature to obtain the combined first interaction feature) includes: S231, determining a first identification direction of the interactive video, where the first identification direction is a top-to-bottom direction of a fixed marketing terminal.

[0045] It is understandable that the interactive interface of a fixed marketing terminal usually follows the reading order of "from top to bottom, from left to right", such as the questions are on the top and the options are on the bottom. By defining the first recognition direction as "from top to bottom", the human visual browsing path can be simulated, and the logical units of "questions to options" can be captured in sequence, so as to subsequently verify whether the user operations comply with the interface logic. In addition, the arrangement of elements of the interactive interface has spatial regularity. For example, the questions on the questionnaire page are arranged in sequence in the vertical direction. Determining the first recognition direction of "from top to bottom" can enable the system to extract features according to the actual layout order of the interface, avoiding mismatching of questions and options due to direction confusion, such as mistakenly associating the options of the lower question with the upper question.

[0046] Among them, the first recognition direction is the recognition direction of identifying the text content of the interactive video, that is, the direction from top to bottom.

[0047] Through the above implementation, the present invention can clearly identify the direction to match the interface layout rules, ensure the order of feature extraction, and facilitate subsequent information recognition of the interactive video.

[0048] S232, sequentially identifying the first sub-feature and the second sub-feature based on the first recognition direction, and performing text extraction based on OCR. When it is determined that all the first sub-features and the second sub-features have been recognized, or when it is determined that the text in the first sub-feature and the second sub-feature cannot be recognized, the recognition is stopped to obtain a feature recognition sequence.

[0049] It should be noted that since the image acquisition device is at a certain distance from the screen, when a person fills in information at a fixed sales end, the person will block the fixed end screen. Therefore, when identifying features, some text may not be recognized, and the recognition will be stopped, thereby obtaining a feature recognition sequence.

[0050] It can be understood that the video frames are scanned in sequence from top to bottom, the first sub-feature (question) is identified first, and then the second sub-feature (option) below is identified, to ensure that each question and its corresponding option are extracted in pairs, and OCR technology is used to extract text from the first interface. When all visible features are identified, such as the end of the interface or when unrecognizable text is encountered, such as blur or occlusion, recognition is stopped to avoid invalid data from being mixed into the sequence.

[0051] Among them, OCR is an existing text recognition technology, and the feature recognition sequence is a feature sequence obtained by sequentially arranging the recognized features.

[0052] Through the above implementation, the present invention can achieve automatic recognition, reduce manual intervention, and improve feature extraction efficiency.

[0053] S233: Combine and mark the feature recognition sequences to obtain a plurality of first interaction features.

[0054] It is understandable that feature recognition sequences, such as "question 1, option, question 2, option", need to be split and combined according to "question-option group". For example, "question 1+option" is a first interactive feature, that is, the ordered extracted feature sequences are combined into independent interactive units, and marks are added to distinguish different logical groups for subsequent feature analysis.

[0055] Through the above implementation, the present invention can split a long sequence into independent units, which is convenient for abnormal diagnosis in modules.

[0056] In some embodiments, a specific implementation of step S233 (combining and marking the feature recognition sequences to obtain a plurality of first interaction features) includes: S2331: If it is determined that the first feature of the feature recognition sequence is not the first sub-feature, it is determined that the feature recognition sequence has failed the correctness verification, and the interactive video is acquired again.

[0057] It is understandable that when it is determined that the first feature in the feature recognition sequence is not the first sub-feature extracted in advance, it may mean that the person slid or performed other operations on the screen, resulting in the first feature recognized not being the title, that is, there was a problem with the recognition, and therefore, the interactive video needs to be acquired again.

[0058] Through the above-mentioned implementation, the present invention can prevent invalid or disordered feature sequences from entering subsequent processes, reduce diagnostic misjudgments, discover data problems in a timely manner in the feature combination stage, and avoid errors from being transmitted to the diagnosis module.

[0059] S2332: If it is determined that the feature recognition sequence passes the correctness verification, first sub-features and second sub-features with adjacent serial numbers are sequentially extracted and combined to obtain a first interactive feature.

[0060] It can be understood that when in the feature sequence that passes the initial verification, the adjacent first sub-features (questions) and the subsequent continuous second sub-features (options) constitute a complete interactive unit (such as "question + optional answer"). Through this adjacent combination rule, it can be ensured that each question and its corresponding option are correctly associated to avoid option mismatches across questions. For example, it is prevented that people mistake the options of the second question for the options of the first question and choose to answer, thereby missing the second question.

[0061] It should be noted that the feature type needs to be dynamically tracked during the combination process. When a new first sub-feature is detected, the combination of the previous interaction unit is automatically terminated and the construction of the new unit begins.

[0062] S2333, adding a separator mark to the first interactive feature of the last sequence number.

[0063] It is understandable that after the feature sequence combination is completed (such as all "question-option groups" have been split), a special separator mark is added to the last interactive feature to clearly identify the end position of the sequence. When there is page turning in the interface, the separator mark can distinguish the interactive features of different pages. If the video acquisition is interrupted due to equipment failure, the mark can help the system identify the last unit that has been fully parsed to avoid incomplete data from participating in the diagnosis.

[0064] The separation mark is a mark symbol that separates the combination and is used for combination analysis, such as "END".

[0065] Through the above implementation, the present invention can add a separator after the last sequence number to avoid parsing ambiguity caused by the ambiguity of the sequence end, and support accurate positioning during breakpoint resumption or abnormal recovery.

[0066] S3, a recognition module recognizes the attribute characteristics of the first interacting person to obtain a first estimated attribute, and recognizes the interaction characteristics of the first interacting person to obtain a second interaction characteristic.

[0067] It is understandable that the attribute characteristics of the interacting person (such as height) and the interaction characteristics (such as the finger click position) contain implicit clues to the rationality of the operation. Through multimodal data collection, the physical attributes and operation behavior characteristics of the interacting person are extracted for subsequent abnormality diagnosis.

[0068] Among them, the identification module is a processing module for identifying features, the first interacting person is a person who interacts with the fixed marketing end for information, the attribute feature is the attribute feature information of the person's corresponding identity, such as height, etc., the first estimated attribute is the estimated value of the attribute feature of the first interacting person, the interaction feature is the feature information of the interaction between the person and the fixed marketing end, and the second interaction feature is the trigger feature information for the information interaction between the person, such as the interaction trigger position.

[0069] In some embodiments, a specific implementation of step S3 (identifying the attribute characteristics of the first interacting person to obtain the first estimated attribute, and identifying the interaction characteristics of the first interacting person to obtain the second interaction characteristics) includes: S31 , identifying attribute features of the first interacting person based on the distance measuring device at the fixed marketing terminal, and obtaining a first estimated attribute of the height information.

[0070] It is understandable that fixed marketing terminals are usually designed for specific groups of people, such as touch screens at adult height, and height data can be used to determine whether the user belongs to the target group.

[0071] For example, if the ranging device detects that the user is 1.2 meters tall (child), and the device interface does not provide a child adaptation mode, it may lead to operational deviations. Through height information (first estimated attribute), the system can pre-adjust the diagnostic rules, such as increasing the tolerance for accidental touches by children or prompting interface adaptation issues, and using the ranging device to obtain the height data of the interacting person in order to obtain the first estimated attribute of the height information.

[0072] The distance measuring device is a device for measuring the height distance of a person, such as a binocular vision camera.

[0073] S32: Identify the finger location of the first interacting person to obtain a second interaction feature with finger location.

[0074] It is understandable that fingers are the main interactive tool, and their positioning data (such as click location and movement trajectory) directly reflects the operation intention. Finger positioning technology is used to capture the interaction trajectory and trigger points to determine the user's operation behavior.

[0075] Among them, finger positioning can be combined with color marking, such as users wearing finger sleeves of specific colors, to improve recognition accuracy in complex backgrounds.

[0076] Through the above implementation, the present invention can convert blurred video actions into quantifiable coordinate data, and by comparing the finger trigger point with the actual position of the option, it is convenient to promptly determine the operation abnormality and quickly locate abnormal types such as invalid clicks and accidental touches.

[0077] In some embodiments, a specific implementation of step S32 (recognizing the finger location of the first interacting person to obtain a second interaction feature with finger location) includes: S321: The finger sleeve of the first interacting person is provided with a first identification object, where the first identification object is a finger sleeve with a third color characteristic.

[0078] It is understandable that in a natural environment, there may be color confusion between fingers and the background. For example, if the skin color of the user's hand is similar to the screen background, it will lead to visual positioning errors. By wearing finger sleeves with a third color feature (such as bright red), the finger area can be quickly separated through color filtering algorithms (such as HSV color space segmentation) to eliminate other interference factors.

[0079] The first identification object is an object for positioning identification, such as a finger sleeve with a third preset color, where the third color is a preset color for distinguishing the first two colors.

[0080] It is worth mentioning that the fixed marketing end has a finger sleeve fixed with a traction rope. When personnel fill out the questionnaire, they need to wear the inherent finger sleeve to interact and trigger the screen of the fixed marketing end.

[0081] S322: Locate and identify the position of the first identification object to obtain an identification track, generate a trigger point when it is determined that the first identification object is in contact with the screen, and generate a corresponding second interaction feature based on the identification track and the trigger point.

[0082] It is understandable that since the person wears the fingertip on the finger and moves it to the screen to trigger the operation, the process has a motion trajectory. Therefore, the recognition trajectory can be obtained based on the position positioning and identification of the first identification object, and when the recognition trajectory is determined to be triggered on the screen, the trigger point is generated, thereby obtaining the second interaction feature, that is, the interaction information of the trigger point with the fingertip positioning.

[0083] The recognition track is the position movement track corresponding to the first recognition object, and the trigger point is the position point where the first recognition object triggers the screen.

[0084] S4, a diagnosis module, performs combined verification diagnosis based on the first interaction feature, the first estimated attribute, and the second interaction feature to obtain an abnormal diagnosis result of a fixed marketing terminal or the first interaction person.

[0085] It is understandable that single-dimensional data is difficult to comprehensively determine the type of anomaly. Therefore, multiple feature data can be combined for verification and diagnosis to determine the abnormal diagnosis results of a fixed marketing terminal or the first interactive person, so that prompts or corrections can be made in a timely manner later.

[0086] The abnormal diagnosis result is the diagnosis result of monitoring abnormal information.

[0087] For example, if the interface options are displayed normally (no abnormality in the first interactive feature) but the user clicks on invalid areas multiple times (an abnormality in the second interactive feature), it may indicate a device touch failure. If the user's height is lower than the device design standard (an abnormality in the first estimated attribute) and the operation trajectory is chaotic, it may be a user adaptability issue.

[0088] Through the above-mentioned embodiments, the present invention can combine multi-dimensional data to perform abnormality analysis in order to improve the accuracy of diagnosis.

[0089] In some embodiments, a specific implementation of step S4 (performing a combined verification diagnosis based on the first interaction feature, the first estimated attribute, and the second interaction feature to obtain an abnormality diagnosis result for a fixed marketing terminal or the first interacting person) includes: S41 , combining and classifying the first interaction feature, the first estimated attribute, and the second interaction feature to obtain a plurality of verification groups.

[0090] It can be understood that the multiple first interaction features, first estimated attributes and second interaction features obtained are grouped to obtain verification groups, that is, different verification groups have different first interaction features, first estimated attributes and second interaction feature information, so that their feature information is cross-combined, so that the classified multiple verification groups can be verified subsequently, thereby improving the accuracy of the abnormal diagnosis results.

[0091] The verification group is a feature combination for information verification.

[0092] S42: Call the verification logic corresponding to each verification group to obtain the corresponding abnormality diagnosis result.

[0093] It can be understood that through the predefined verification logic, logical deduction is performed on each verification group to obtain abnormal diagnosis results, and the abnormal type and cause are output.

[0094] Through the above-mentioned embodiments, the present invention can eliminate the interference of a single factor through combination rules and improve the accuracy of abnormality diagnosis.

[0095] In some embodiments, the specific implementation of step S42 (calling the verification logic corresponding to each verification group to obtain the corresponding abnormality diagnosis result) includes: S421: Extract verification groups corresponding to the first interaction feature and the second interaction feature.

[0096] It is understandable that in order to subsequently verify the matching between user click behavior and interface elements, the verification group corresponding to the first interaction feature and the second interaction feature can be extracted, so as to subsequently conduct targeted analysis of the consistency between the interface display and the actual response.

[0097] S422: Determine the first interaction feature of the trigger point corresponding to the second interaction feature, and obtain first verification information triggered based on the video.

[0098] It can be understood that by searching for the corresponding element in the interface layout of the first interactive feature through the trigger point in the second interactive feature (such as the location coordinates of the finger click), the user's "intended click target" is determined.

[0099] The first verification information is a first interactive feature triggered by a person, such as content that can be triggered by a person clicking.

[0100] S423, extracting the second verification information of the first interactive feature corresponding to the fixed marketing terminal in real time, and if the first verification information and the second verification information do not correspond, generating an abnormal diagnosis result of the fixed marketing terminal and performing option selection based on the second interactive feature.

[0101] It can be understood that the second verification information is the first interactive feature fed back by the fixed marketing end. When the first verification information and the second verification information do not correspond, it means that the finger has been triggering, but the fixed marketing end cannot be selected, which means that there is an abnormality in the fixed marketing end. Therefore, the option confirmation selection can be automatically made according to the second interactive feature to achieve abnormal diagnosis and automatic error correction.

[0102] For example, if the first verification information (video analysis shows that option A was clicked) is inconsistent with the second verification information (the device records that option B was selected), it indicates that there is an interaction anomaly, that is, the touch layer offset causes the physical click position and the logical response position to be misaligned.

[0103] It is not difficult to understand that, according to the triggering track of the second interactive feature, the system can actively select the option actually clicked by the user (rather than the option mistakenly recorded by the device) to ensure that the business process continues to execute. For example, if the user clicks the "Confirm" button visible in the video, but the device does not respond, the system can automatically trigger the confirmation operation and mark the abnormality to be repaired.

[0104] In some embodiments, a specific implementation of step S423 (selecting an option based on the second interaction feature) includes: S4231, determining the number of times the first identifier in the second interactive feature touches the option, and if the number of touches is greater than a preset number, taking the corresponding option as the target option.

[0105] It is understandable that by counting the number of times the first identification object (finger sleeve) touches the option area, valid operations can be distinguished from invalid operations. For example, if the preset number of times is 2, and the user touches "Option A" three times in a row, it is determined that the user's true intention is to select this option, even if the device records that other options have been selected.

[0106] The number of touches is the number of clicks on the first identification object triggering the option, the preset number is a pre-set number, and the target option is the option with a touch number greater than the preset number.

[0107] In other embodiments, the specific implementation of step S42 (the step of calling the verification logic corresponding to each verification group to obtain the corresponding abnormality diagnosis result) further includes: S424: Extract the verification group corresponding to the first interaction feature, the first estimated attribute, and the second interaction feature.

[0108] It can be understood that the first interaction feature, the first estimated attribute, and the verification group corresponding to the second interaction feature are extracted so as to subsequently compare the first interaction feature with the second interaction feature.

[0109] S425: If the first interaction feature and the second interaction feature correspond to each other, extract corresponding option results.

[0110] It is understandable that the first interaction feature corresponds to the second interaction feature, which can indicate that the user operation conforms to the answering logic of the interface questionnaire, and the corresponding option results can be extracted for subsequent comparison with the first estimated attribute.

[0111] S426: If the option result does not correspond to the first estimated attribute, an abnormal trigger tag corresponding to the fixed marketing end is generated and fed back to the fixed marketing end.

[0112] It is understandable that when the option result does not correspond to the first estimated attribute, it means that the corresponding person does not match the service object of the fixed marketing end, that is, the interacting person may have accidentally touched the fixed marketing end, and thus, an abnormal trigger label can be generated and displayed on the fixed marketing end.

[0113] Among them, the abnormal trigger tag is the tag prompt information that accidentally triggers the fixed marketing end to achieve business monitoring and avoid entering and analyzing the corresponding reported information, which affects the accuracy of data collection.

[0114] The present invention also provides a storage medium, in which a computer program is stored. When the computer program is executed by a processor, it is used to implement the methods provided in the various embodiments described above.

[0115] The storage medium may be a computer storage medium or a communication medium. A communication medium includes any medium that facilitates the transfer of a computer program from one location to another. A computer storage medium may be any available medium that can be accessed by a general-purpose or special-purpose computer. For example, a storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium may also be an integral part of the processor. The processor and the storage medium may be located in an application-specific integrated circuit (ASIC). In addition, the ASIC may be located in a user device. Of course, the processor and the storage medium may also exist as discrete components in a communication device. The storage medium may be a read-only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0116] The present invention also provides a program product, which includes execution instructions stored in a storage medium. At least one processor of a device can read the execution instructions from the storage medium, and at least one processor executes the execution instructions so that the device implements the methods provided in the various embodiments described above.

[0117] In the above-mentioned terminal or server embodiments, it should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASICs), etc. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the present invention may be directly executed by a hardware processor or by a combination of hardware and software modules within the processor.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. Real-time business monitoring and accurate abnormality diagnosis system, characterized by: include: The acquisition module collects images of the interaction locations of fixed marketing terminals to obtain interaction videos; An extraction module extracts a first interaction feature of a first interface having a service corresponding to the screen in the interactive video; an identification module that identifies the attribute characteristics of the first interacting person to obtain a first estimated attribute, and identifies the interaction characteristics of the first interacting person to obtain a second interaction characteristic; The diagnosis module performs combined verification diagnosis based on the first interaction feature, the first estimated attribute, and the second interaction feature to obtain an abnormal diagnosis result of a fixed marketing terminal or the first interaction person.

2. The system according to claim 1, wherein: The fixed interactive position of the marketing terminal is used to collect images to obtain interactive videos, including: After determining that the fixed marketing terminal is triggered, the image acquisition device acquires images of the fixed marketing terminal from top to bottom according to a preset angle; The screen edge area of ​​the fixed marketing terminal in the image is extracted and captured to obtain the interactive video.

3. The system according to claim 1, wherein: The extracting the first interactive feature of the first interface having the service corresponding to the screen in the interactive video includes: Extracting text that is a first color feature in the interactive video to obtain a first sub-feature, where the first color is the title color; Extracting text that is a second color feature in the interactive video to obtain a second sub-feature, where the second color is an option color; The combined first interactive feature is obtained by analyzing the feature relationship between the first sub-feature and the second sub-feature.

4. The system according to claim 3, characterized in that The analyzing the feature relationship between the first sub-feature and the second sub-feature to obtain the combined first interaction feature includes: Determining a first recognition direction of the interactive video, where the first recognition direction is a top-to-bottom direction of a fixed marketing terminal; The first sub-feature and the second sub-feature are sequentially identified based on the first recognition direction, and text extraction is performed based on OCR. When it is determined that all the first sub-features and the second sub-features have been identified, or when it is determined that the text in the first sub-feature and the second sub-feature cannot be identified, the recognition is stopped to obtain a feature recognition sequence; The feature recognition sequences are combined and marked to obtain a plurality of first interaction features.

5. The system according to claim 4, characterized in that The feature recognition sequence is combined and marked to obtain a plurality of first interaction features, including: If it is determined that the first feature of the feature recognition sequence is not the first sub-feature, the feature recognition sequence is determined to have failed the correctness verification, and the interactive video is acquired again; If the feature recognition sequence is judged to pass the correctness verification, the first sub-features and the second sub-features of adjacent serial numbers are sequentially extracted and combined to obtain a first interactive feature; Add a separator mark to the first interactive feature of the last sequence number.

6. The system according to claim 1, wherein: The identifying of the attribute characteristics of the first interacting person to obtain the first estimated attribute and identifying the interaction characteristics of the first interacting person to obtain the second interaction characteristics include: Recognizing attribute characteristics of the first interacting person based on the distance measuring device at the fixed marketing terminal to obtain a first estimated attribute of the height information; The finger position of the first interacting person is identified to obtain a second interaction feature with the finger position.

7. The system according to claim 6, characterized in that The identifying the finger position of the first interacting person to obtain a second interaction feature having the finger position includes: The first interacting person's finger sleeve is provided with a first identification object, wherein the first identification object is a finger sleeve with a third color characteristic; The position of the first identification object is located and identified to obtain an identification track, a trigger point is generated when it is determined that the first identification object is in contact with the screen, and a corresponding second interaction feature is generated based on the identification track and the trigger point.

8. The system according to claim 1, wherein: The combined verification diagnosis based on the first interaction feature, the first estimated attribute, and the second interaction feature to obtain an abnormality diagnosis result of a fixed marketing terminal or the first interaction person includes: Combining and classifying the first interaction feature, the first estimated attribute, and the second interaction feature to obtain multiple verification groups; The verification logic corresponding to each verification group is called to obtain the corresponding abnormality diagnosis result.

9. The system according to claim 8, characterized in that The calling of the verification logic corresponding to each verification group to obtain the corresponding abnormality diagnosis result includes: Extracting a verification group corresponding to the first interaction feature and the second interaction feature; Determine the first interaction feature at the trigger point corresponding to the second interaction feature, and obtain first verification information triggered based on the video; The second verification information of the first interactive feature corresponding to the fixed marketing terminal is extracted in real time. If the first verification information and the second verification information do not correspond, an abnormal diagnosis result of the fixed marketing terminal is generated and an option selection is performed based on the second interactive feature.

10. The system according to claim 9, characterized in that The selecting an option based on the second interaction feature includes: The number of times the first identifier touches the option in the second interactive feature is determined, and if the number of touches is greater than a preset number, the corresponding option is used as the target option.

11. The system according to claim 8, wherein: The calling of the verification logic corresponding to each verification group to obtain the corresponding abnormality diagnosis result includes: Extracting a verification group corresponding to the first interaction feature, the first estimated attribute, and the second interaction feature; If the first interaction feature and the second interaction feature correspond to each other, extracting the corresponding option results; If the option result does not correspond to the first estimated attribute, an abnormal trigger tag corresponding to the fixed marketing end is generated and fed back to the fixed marketing end.

Images