Real-time service monitoring and accurate exception diagnosis system

Abstract

The application provides a real-time service monitoring and accurate abnormality diagnosis system, relates to the technical field of data processing, and comprises a collection module, which collects images of interactive positions of a fixed marketing end to obtain interactive videos; an extraction module, which extracts first interactive features of a first interface of a service corresponding to a screen in the interactive videos; an identification module, which identifies attribute features of first interactive personnel to obtain first estimated attributes and identifies interactive features of the first interactive personnel to obtain second interactive features; and a diagnosis module, which performs combined verification diagnosis based on the first interactive features, the first estimated attributes and the second interactive features to obtain an abnormality diagnosis result of the fixed marketing end or the first interactive personnel, can monitor services in real time, jointly analyzes multidimensional data, and improves the accuracy of abnormality diagnosis.

Description

Technical Field

[0001] This invention relates to data processing technology, and more particularly to a real-time business monitoring and accurate anomaly diagnosis system. Background Technology

[0002] In digital marketing and business interaction scenarios, the application of fixed marketing terminals (such as smart questionnaire terminals and service application devices) is becoming increasingly widespread. For example, fixed marketing terminals deployed in shopping malls, communities and other places need to collect user demand data in real time. However, in actual operation, they often face multiple challenges, such as teenagers accidentally touching adult questionnaires, elderly people misselecting options due to vision problems, and device touch failure causing unresponsive operation. These anomalies not only affect the accuracy of data collection, but may also lead to business process interruption or a decline in user experience.

[0003] In existing technologies, traditional monitoring methods can only record the results of operations, but cannot capture the details of the interaction process in real time, making it difficult to quickly locate the source of anomalies, resulting in inconsistent data quality and delayed fault response.

[0004] Therefore, how to monitor business operations in real time, perform joint analysis of multi-dimensional data, and improve the accuracy of anomaly diagnosis has become an urgent problem to be solved. Summary of the Invention

[0005] This invention provides a real-time business monitoring and accurate anomaly diagnosis system, which can monitor business in real time, perform joint analysis of multi-dimensional data, and improve the accuracy of anomaly diagnosis.

[0006] A first aspect of the present invention provides a real-time business monitoring and accurate anomaly diagnosis system, comprising:

[0007] The acquisition module captures images from the interactive locations on a fixed marketing terminal to obtain interactive videos;

[0008] The extraction module extracts the first interactive features of the first interface of the business corresponding to the screen in the interactive video;

[0009] The identification module identifies the attribute features of the first interacting person to obtain the first estimated attribute, and identifies the interaction features of the first interacting person to obtain the second interaction feature.

[0010] The diagnostic module performs combined verification and diagnosis based on the first interaction feature, the first predicted attribute, and the second interaction feature to obtain abnormal diagnostic results for fixed marketing personnel or the first interaction personnel.

[0011] Optionally, in one possible implementation of the first aspect, the interactive video is obtained by capturing images at the interaction location of the fixed marketing terminal, including:

[0012] After determining that the fixed marketing terminal has been triggered, the image acquisition device captures images of the fixed marketing terminal from top to bottom at a preset angle.

[0013] Extract and crop the screen edge area of ​​the fixed marketing end in the image to obtain the interactive video.

[0014] Optionally, in one possible implementation of the first aspect, extracting the first interactive features of the first interface of the service corresponding to the screen from the interactive video includes:

[0015] The first sub-feature is obtained by extracting the text with the first color feature from the interactive video, where the first color is the title color;

[0016] The second sub-feature is obtained by extracting the text with the second color feature from the interactive video, where the second color is the option color;

[0017] The first interactive feature is obtained by analyzing the feature relationship between the first sub-feature and the second sub-feature.

[0018] Optionally, in one possible implementation of the first aspect, the step of analyzing the feature relationship between the first sub-feature and the second sub-feature to obtain the combined first interactive feature includes:

[0019] Determine the first recognition direction of the interactive video, which is the top-to-bottom direction of the fixed marketing end;

[0020] Based on the first recognition direction, the first sub-feature and the second sub-feature are recognized sequentially, and text extraction is performed based on OCR. When it is determined that all the first sub-features and the second sub-features have been recognized, or the text in the first sub-features and the second sub-features cannot be recognized, the recognition is stopped and the feature recognition sequence is obtained.

[0021] By combining and labeling the feature recognition sequences, multiple first interactive features are obtained.

[0022] Optionally, in one possible implementation of the first aspect, the combination and labeling of the feature recognition sequences to obtain a plurality of first interactive features includes:

[0023] If the first feature of the feature recognition sequence is not the first sub-feature, the feature recognition sequence is deemed to have failed the correctness check, and the interactive video is reacquired.

[0024] If the feature recognition sequence passes the correctness check, then the first sub-feature and the second sub-feature of adjacent numbers are extracted and processed to obtain a first interactive feature.

[0025] Add a separator mark to the first interactive feature of the last sequence number.

[0026] Optionally, in one possible implementation of the first aspect, the identification of the attribute features of the first interacting person to obtain a first estimated attribute, and the identification of the interaction features of the first interacting person to obtain a second interaction feature, includes:

[0027] Based on the attribute feature recognition of the first person interacting with the device at the fixed marketing terminal, the first estimated attribute of height information is obtained.

[0028] By locating and recognizing the finger of the first person interacting, a second interaction feature with finger location is obtained.

[0029] Optionally, in one possible implementation of the first aspect, the identification of the finger location of the first interacting person to obtain a second interaction feature with finger location includes:

[0030] The finger sleeve of the first interactive person is equipped with a first identification object, which is a finger sleeve with a third color feature;

[0031] The location of the first object is identified to obtain the identification trajectory. When it is determined that the first object is in contact with the screen, a trigger point is generated. Based on the identification trajectory and the trigger point, a corresponding second interaction feature is generated.

[0032] Optionally, in one possible implementation of the first aspect, the combined verification and diagnosis based on the first interaction feature, the first predicted attribute, and the second interaction feature to obtain the abnormal diagnosis result for the fixed marketing end or the first interaction personnel includes:

[0033] Multiple validation groups are obtained by combining and classifying the first interaction feature, the first predicted attribute, and the second interaction feature respectively.

[0034] The corresponding verification logic for each verification group is invoked to obtain the corresponding anomaly diagnosis result.

[0035] Optionally, in one possible implementation of the first aspect, the invocation of the verification logic corresponding to each verification group to obtain a corresponding anomaly diagnosis result includes:

[0036] Extract the verification groups corresponding to the first and second interaction features;

[0037] Determine the first interaction feature corresponding to the trigger point of the second interaction feature, and obtain the first verification information triggered by the video;

[0038] Extract the second verification information corresponding to the first interaction feature of the fixed marketing terminal in real time. If the first verification information and the second verification information do not correspond, generate the abnormal diagnosis result of the fixed marketing terminal and select options based on the second interaction feature.

[0039] Optionally, in one possible implementation of the first aspect, the option selection based on the second interaction feature includes:

[0040] Determine the number of touches between the first identification object and the option in the second interaction feature. If the number of touches is greater than the preset number, then the corresponding option is taken as the target option.

[0041] Optionally, in one possible implementation of the first aspect, the invocation of the verification logic corresponding to each verification group to obtain a corresponding anomaly diagnosis result includes:

[0042] Extract the validation group corresponding to the first interaction feature, the first predicted attribute, and the second interaction feature;

[0043] If the first interaction feature corresponds to the second interaction feature, then the corresponding option result is extracted;

[0044] If the option result does not correspond to the first predicted attribute, an exception trigger tag corresponding to the fixed marketing end will be generated and fed back to the fixed marketing end.

[0045] A second aspect of the present invention provides a storage medium storing a computer program, which, when executed by a processor, is used to implement the methods of the first aspect of the present invention and various possible designs of the first aspect.

[0046] The beneficial effects of this invention are as follows:

[0047] 1. This invention enables real-time monitoring of business operations and joint analysis of multi-dimensional data to improve the accuracy of anomaly diagnosis. Firstly, after triggering at a fixed marketing terminal, this invention can capture interactive video at a preset angle using a camera and extract the screen edge area to generate a focused interactive video. The system separates the questions and options using color features, and sequentially extracts the first and second sub-features along a top-to-bottom recognition direction to obtain a feature recognition sequence. If the sequence begins without question features, the video is automatically re-captured to ensure data validity.

[0048] 2. This invention integrates user attribute features and interaction behavior features to achieve matching and identification of the operating subject and operating intent, thereby improving the accuracy of anomaly diagnosis. Specifically, this invention can use a ranging device to obtain user height information and combine it with finger positioning technology to capture finger trigger points and movement trajectories in real time, enabling rapid identification of interface adaptation anomalies. By combining and analyzing data from multiple dimensions, it correlates user physical actions with operating behaviors, avoiding misjudgments from single-dimensional diagnoses and improving the accuracy of anomaly diagnosis.

[0049] 3. This invention can achieve accurate location and automatic processing of anomalies based on the combined verification logic of multi-dimensional features. Specifically, this invention can group and verify the first interaction feature, the first estimated attribute, and the second interaction feature, and call preset logic to eliminate interference factors. When the first interaction feature is normal but the finger trigger point is misaligned with the option position, i.e., the second interaction feature is abnormal, combined with the user's first estimated attribute being normal, it can be determined that the device touch is malfunctioning, and the correct option is automatically triggered according to the finger trajectory to maintain the business process. If the option result does not match the user attribute, an anomaly trigger tag is generated and invalid data entry is blocked. Through the combined verification mechanism, not only can the anomaly types such as device failure, interface adaptation, and user misoperation be accurately distinguished, but also manual intervention can be reduced through automatic error correction, thereby improving the efficiency of anomaly response. Attached Figure Description

[0050] Figure 1 This is a schematic diagram of a real-time business monitoring and accurate anomaly diagnosis system provided by the present invention. Detailed Implementation

[0051] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0052] like Figure 1 The diagram shown illustrates a real-time business monitoring and accurate anomaly diagnosis system provided by the present invention. This system includes:

[0053] S1, the acquisition module, acquires images from the interactive locations on the fixed marketing terminal to obtain interactive videos.

[0054] It should be noted that the personnel collecting data on different marketing platforms are not the same. For example, some fixed marketing platforms mainly collect information on the needs of adults, while others mainly collect information on the needs of teenagers. When personnel of the wrong age group fill in the data, it will affect the accuracy of data collection. In addition, some personnel may make operational errors when filling in the data, such as the data they fill in does not match the questions, which will reduce the accuracy of data collection. Therefore, it is possible to monitor the data collection process in real time and diagnose anomalies in a timely manner.

[0055] Understandably, in order to ensure the accuracy of the collected information, the identity of the personnel will be initially identified. Then, when the personnel fill in the information, interactive video of the interaction between the fixed marketing terminal and the corresponding interaction position will be collected to identify the personnel's identity information and monitor the interaction operation, so as to facilitate timely diagnosis of anomalies.

[0056] The acquisition module is a system processing module that collects data and provides identification information for subsequent monitoring and analysis. The fixed marketing terminal is a data acquisition device at a fixed location. The interaction location is the location where the device and personnel interact with each other, i.e., the location of the fixed marketing terminal. The interactive video is the video data collected by the acquisition device at the corresponding location of the fixed marketing terminal. For example, it can be video data with the fixed marketing terminal screen and personnel captured by a camera.

[0057] In some embodiments, the specific implementation of step S1 (obtaining interactive video by capturing images at the interaction location of the fixed marketing terminal) includes:

[0058] S11, after determining that the fixed marketing terminal has been triggered, the image acquisition device acquires images of the fixed marketing terminal from top to bottom at a preset angle.

[0059] It is understandable that when a user clicks on a fixed marketing terminal, the corresponding image acquisition device will capture an image of the fixed marketing terminal.

[0060] Among them, the image acquisition device is the device that performs image acquisition, such as a camera, and the preset angle is the angle at which the image is acquired in advance, which is preset.

[0061] S12, extract and crop the screen edge area of ​​the fixed marketing end in the image to obtain the interactive video.

[0062] Understandably, when verifying the collected data later, only the content filled in by the personnel needs to be identified. Therefore, the final interactive video only needs to fix the screen area of ​​the marketing end, and other extra areas can be deleted, such as the blank area next to the screen. In this way, the extracted interactive video can be obtained, which is convenient for subsequent information recognition.

[0063] Among them, the screen edge area is the boundary area of ​​the screen corresponding to the fixed marketing terminal.

[0064] S2, the extraction module, extracts the first interactive features of the first interface of the business corresponding to the screen in the interactive video.

[0065] Understandably, interactive interfaces on fixed marketing platforms, such as questionnaires and service applications, typically use colors to distinguish between questions and options, such as black titles and blue options. By extracting text with different color features, unstructured visual information in videos can be transformed into structured "question-option" data, making it easier for the system to identify the specific content of user interactions. In other words, by separating question and option information through color features, structured interactive feature data can be constructed, which is convenient for subsequent anomaly diagnosis.

[0066] The extraction module is a system processing module for extracting the first interactive feature, the first interface is a video frame interface with business information in the interactive video, and the first interactive feature is the feature information in the first interface.

[0067] In some embodiments, the specific implementation of step S2 (extracting the first interactive features of the first interface of the service corresponding to the screen in the interactive video) includes:

[0068] S21, extract the text with the first color feature from the interactive video to obtain the first sub-feature, where the first color is the title color.

[0069] Understandably, since the interactive video displays questionnaire information on a fixed marketing end, the questions and options in the questionnaire are in different colors to help people distinguish them. At the same time, when identifying the interactive video, information can also be identified based on the colors, making it easier to determine whether there are any abnormalities in the current interactive video.

[0070] The first color is the color corresponding to the information title in the first interface, which can be preset, such as black. The first sub-feature is the text title corresponding to the first color.

[0071] S22, extract the text in the interactive video that has the second color feature to obtain the second sub-feature, where the second color is the option color.

[0072] Understandably, the second color is the color corresponding to the information option in the first interface, which can be preset, such as blue, and the second sub-feature is the text option corresponding to the first color.

[0073] S23, based on the feature relationship between the first sub-feature and the second sub-feature, the combined first interactive feature is obtained.

[0074] Understandably, since a question must have corresponding options, the first interaction feature can be determined based on the relationship between the first and second sub-features. This allows the first and second sub-features in the same group to be integrated into a complete interaction unit, matching the user's actual operation scenario and facilitating improved diagnostic efficiency in the future.

[0075] It's easy to understand that questions and options have a fixed hierarchical relationship in the interface, such as questions on top and options below. Therefore, the corresponding feature relationship can be determined based on the position of different features.

[0076] In some embodiments, the specific implementation of step S23 (the step of analyzing the feature relationship between the first sub-feature and the second sub-feature to obtain the combined first interactive feature) includes:

[0077] S231, determine the first recognition direction of the interactive video, wherein the first recognition direction is the top-to-bottom direction of the fixed marketing end.

[0078] Understandably, the interactive interface of a fixed marketing terminal usually follows a reading order of "from top to bottom and from left to right," such as questions at the top and options at the bottom. By defining the first recognition direction as "from top to bottom," the human visual browsing path can be simulated, capturing the logical units of "question to option" in sequence. This allows for subsequent verification of whether the user's operation conforms to the interface logic. Furthermore, the arrangement of elements in the interactive interface has spatial regularity. For example, the questions on a questionnaire page are arranged vertically. Determining the first recognition direction as "from top to bottom" allows the system to extract features according to the actual layout order of the interface, avoiding mismatches between questions and options due to directional confusion, such as mistakenly associating the options of the lower questions with the upper questions.

[0079] The first recognition direction is the direction of recognition for the text content in the interactive video, that is, the direction from top to bottom.

[0080] Through the above implementation methods, the present invention can clearly identify the direction to match the interface layout rules, ensure the sequentiality of feature extraction, and facilitate subsequent information recognition of interactive videos.

[0081] S232, based on the first recognition direction, the first sub-feature and the second sub-feature are recognized sequentially, and text extraction is performed based on OCR. When it is determined that all the first sub-features and the second sub-features have been recognized, or the text in the first sub-features and the second sub-features cannot be recognized, the recognition is stopped and the feature recognition sequence is obtained.

[0082] It should be noted that, because the image acquisition device is a certain distance from the screen, when a person is filling out information at the fixed sales end, the person will block the screen at the fixed end. Therefore, when recognizing features, some characters may not be recognized, so the recognition will stop, thus obtaining the feature recognition sequence.

[0083] Understandably, the video frames are scanned sequentially from top to bottom. First, the first sub-feature (question) is identified, and then the second sub-feature (option) is identified below. This ensures that each question and its corresponding option are extracted in pairs. OCR technology is used to extract text from the first interface. When all visible features are identified, such as at the end of the interface or when unrecognizable text is encountered, such as blurred or obscured text, recognition stops to avoid invalid data from being mixed into the sequence.

[0084] Among them, OCR is an existing text recognition technology, and the feature recognition sequence is a feature sequence obtained by arranging the recognized features in sequence.

[0085] Through the above-described embodiments, the present invention can achieve automatic identification, reduce manual intervention, and improve feature extraction efficiency.

[0086] S233, combine and label the feature recognition sequences to obtain multiple first interactive features.

[0087] Understandably, feature recognition sequences, such as "Question 1, Option, Question 2, Option", need to be split and combined according to "Question-Option Groups". For example, "Question 1 + Option" is a first interactive feature. That is, the ordered extracted feature sequences are combined into independent interactive units and marked to distinguish different logical groups for subsequent feature analysis.

[0088] Through the above implementation methods, the present invention can split long sequences into independent units, which facilitates abnormal diagnosis by module.

[0089] In some embodiments, the specific implementation of step S233 (combining and labeling the feature recognition sequences to obtain multiple first interactive features) includes:

[0090] S2331, if the first feature of the feature recognition sequence is not the first sub-feature, then the feature recognition sequence has failed the correctness check, and the interactive video is reacquired.

[0091] Understandably, if it is determined that the first feature in the feature recognition sequence is not the first sub-feature extracted in advance, it means that the person swiped or performed other operations on the screen, causing the first feature to be recognized to be not the question, that is, there is a problem with the recognition. Therefore, it is necessary to re-acquire the interactive video.

[0092] Through the above implementation methods, the present invention can prevent invalid or disordered feature sequences from entering subsequent processes, reduce diagnostic misjudgments, promptly detect data problems in the feature combination stage, and prevent errors from being transmitted to the diagnostic module.

[0093] S2332, if the feature recognition sequence passes the correctness check, then the first sub-feature and the second sub-feature of the adjacent sequence number are extracted and processed in sequence to obtain a first interactive feature.

[0094] It is understandable that when the first adjacent sub-feature (question) and the subsequent consecutive second sub-feature (option) in the feature sequence that passes the initial verification form a complete interaction unit (such as "question + optional answer"), this adjacent combination rule can ensure that each question and its corresponding option are correctly associated, avoiding cross-question option mismatch. For example, it can prevent people from mistaking the option of the second question as the option of the first question and making a selection answer, thus missing the second question.

[0095] It should be noted that the feature type needs to be dynamically tracked during the combination process. When a new first sub-feature is detected, the combination of the previous interaction unit is automatically terminated and the construction of the new unit begins.

[0096] S2333, add a separator mark to the first interactive feature of the last sequence number.

[0097] Understandably, after the feature sequence is combined (such as when all "question-option groups" have been split), adding a special separator mark to the last interactive feature can clearly identify the end position of the sequence. When there are page turns on the interface, the separator mark can distinguish the interactive features of different pages. If video capture is interrupted due to equipment failure, the mark can help the system identify the last unit that has been completely parsed, avoiding incomplete data from participating in the diagnosis.

[0098] Among them, the separator is used to separate combinations and the symbol is used for combination analysis, such as "END".

[0099] Through the above implementation methods, the present invention can add a separator mark after the last serial number to avoid parsing ambiguity caused by the blurring of the sequence end, and support accurate positioning when resuming interrupted transmissions or recovering from abnormalities.

[0100] S3, the recognition module, identifies the attribute features of the first interacting person to obtain the first estimated attribute, and identifies the interaction features of the first interacting person to obtain the second interaction feature.

[0101] It is understandable that the attributes (such as height) and interaction characteristics (such as finger click location) of the interacting personnel contain clues to the rationality of the operation. By collecting multimodal data, the physical attributes and operational behavior characteristics of the interacting personnel can be extracted for subsequent anomaly diagnosis.

[0102] The identification module is a feature processing module. The first interactive person is the person who interacts with the fixed marketing terminal. The attribute feature is the attribute feature information of the person's corresponding identity, such as height. The first estimated attribute is the value estimated by the attribute feature of the first interactive person. The interaction feature is the feature information of the person interacting with the fixed marketing terminal. The second interaction feature is the trigger feature information of the person's information interaction, such as the interaction trigger location.

[0103] In some embodiments, the specific implementation of step S3 (obtaining the first estimated attribute by identifying the attribute features of the first interacting person, and obtaining the second interaction feature by identifying the interaction features of the first interacting person) includes:

[0104] S31, based on the attribute feature recognition of the first interactive person by the ranging device at the fixed marketing terminal, the first estimated attribute of height information is obtained.

[0105] Understandably, fixed marketing terminals are usually designed for specific groups of people, such as touchscreens for adults, where height data can be used to determine whether a user belongs to the target group.

[0106] For example, if the ranging device detects that the user's height is 1.2 meters (child), but the device interface does not provide a child-friendly mode, it may lead to operational deviations. By using height information (the first estimated attribute), the system can adjust the diagnostic rules in advance, such as increasing the tolerance for accidental touches by children or prompting interface adaptation issues. The system can use the ranging device to obtain the height data of the interacting person in order to obtain the first estimated attribute of the height information.

[0107] Among them, the ranging device is a device for measuring the height and distance of a person, such as a binocular vision camera.

[0108] S32, locate and identify the finger of the first interactive person to obtain a second interactive feature with finger location.

[0109] It is understandable that, as the primary interaction tool, the location data of the finger (such as click location and movement trajectory) directly reflects the operation intention. By capturing the interaction trajectory and trigger point through finger positioning technology, the user's operation behavior can be determined.

[0110] Among these methods, finger positioning can be combined with color markings, such as having users wear specific colored finger sleeves, to improve recognition accuracy in complex backgrounds.

[0111] Through the above implementation methods, the present invention can convert blurry video motion into quantifiable coordinate data. By comparing the finger trigger point with the actual position of the option, it is easier to identify operational anomalies in a timely manner and quickly locate abnormal types such as invalid clicks and accidental touches.

[0112] In some embodiments, a specific implementation of step S32 (identifying the finger location of the first interactive user to obtain a second interactive feature with finger location) includes:

[0113] S321, the finger sleeve of the first interactive person is provided with a first identification object, the first identification object being a finger sleeve with a third color feature.

[0114] It is understandable that in natural environments, the color of fingers and background may be confused. For example, if the user's skin color is similar to that of the screen background, it can lead to visual positioning errors. By wearing finger sleeves with a third color feature (such as bright red), the finger area can be quickly separated through color filtering algorithms (such as HSV color space segmentation) to eliminate other interfering factors.

[0115] The first identification object is the object used for location identification, such as a finger glove with a third preset color, which is a pre-set color that distinguishes the first two.

[0116] It is worth mentioning that the fixed marketing terminal has a finger cot fixed by a tether. When people fill out the questionnaire, they need to wear the fixed finger cot and interact with the screen of the fixed marketing terminal.

[0117] S322, the position of the first object is located and identified to obtain the identification trajectory. When it is determined that the first object is in contact with the screen, a trigger point is generated. Based on the identification trajectory and the trigger point, a corresponding second interactive feature is generated.

[0118] It is understandable that since a person wears a finger cot on their finger and moves it to the screen to trigger the operation, there is a motion trajectory in the process. Therefore, the identification trajectory can be obtained by locating and identifying the position of the first identification object. When the identification trajectory is determined to be triggered on the screen, a trigger point is generated, thereby obtaining the second interactive feature, that is, the interactive information of the trigger point with the finger cot positioning.

[0119] The recognition trajectory is the position movement trajectory corresponding to the first recognition object, and the trigger point is the position point where the first recognition object performs a trigger operation on the screen.

[0120] S4, the diagnostic module, performs combined verification and diagnosis based on the first interaction feature, the first predicted attribute, and the second interaction feature to obtain abnormal diagnostic results for the fixed marketing end or the first interaction personnel.

[0121] It is understandable that single-dimensional data is insufficient to comprehensively determine the type of anomaly. Therefore, multiple feature data can be combined for verification and diagnosis to identify the anomaly diagnosis results of the fixed marketing end or the first person to interact with, so that timely prompts or corrections can be made in the future.

[0122] Among them, the abnormal diagnosis result is the diagnosis result of monitoring abnormal information.

[0123] For example, if the interface options are displayed normally (no abnormality in the first interaction feature), but the user clicks on invalid areas multiple times (abnormality in the second interaction feature), it may indicate that the device's touch control is malfunctioning. If the user's height is below the device's design standard (abnormality in the first estimated attribute) and the operation trajectory is chaotic, it may be a user compatibility issue.

[0124] Through the above-described embodiments, the present invention can combine multi-dimensional data for anomaly analysis in order to improve the accuracy of diagnosis.

[0125] In some embodiments, the specific implementation of step S4 (the combined verification and diagnosis based on the first interaction feature, the first predicted attribute, and the second interaction feature to obtain the abnormal diagnosis result of the fixed marketing end or the first interaction personnel) includes:

[0126] S41, the first interaction feature, the first predicted attribute, and the second interaction feature are combined and classified to obtain multiple validation groups.

[0127] It is understandable that the obtained first interaction features, first predicted attributes, and second interaction features are grouped to obtain validation groups. That is, different validation groups have different information on first interaction features, first predicted attributes, and second interaction features. The feature information is cross-combined so that the multiple validation groups of the classification can be verified in the future, thereby improving the accuracy of the abnormal diagnosis results.

[0128] The verification group is a combination of features used for information verification.

[0129] S42 calls the verification logic corresponding to each verification group to obtain the corresponding anomaly diagnosis result.

[0130] Understandably, by using predefined verification logic, logical deduction is performed on each verification group to obtain anomaly diagnosis results and output the anomaly type and cause.

[0131] Through the above implementation methods, the present invention can eliminate interference from a single factor by combining rules, thereby improving the accuracy of abnormal diagnosis.

[0132] In some embodiments, the specific implementation of step S42 (where the call to the verification logic corresponding to each verification group yields the corresponding anomaly diagnosis result) includes:

[0133] S421, extract the verification groups corresponding to the first interaction feature and the second interaction feature.

[0134] Understandably, in order to perform matching verification between user click behavior and interface elements, the verification groups corresponding to the first and second interaction features can be extracted for subsequent targeted analysis of the consistency between the interface display and the actual response.

[0135] S422, determine the first interaction feature of the trigger point corresponding to the second interaction feature, and obtain the first verification information triggered by the video.

[0136] It is understandable that by using the trigger points in the second interaction feature (such as the coordinates of the finger click position), the corresponding element in the interface layout of the first interaction feature is searched, thereby determining the user's "intended click target".

[0137] The first verification information is the first interaction feature triggered by a person, such as content that can be triggered by a person clicking.

[0138] S423, extract the second verification information of the first interaction feature corresponding to the fixed marketing terminal in real time. If the first verification information and the second verification information do not correspond, generate the abnormal diagnosis result of the fixed marketing terminal and select the option based on the second interaction feature.

[0139] Understandably, the second verification information is the first interaction feature fed back by the fixed marketing terminal. When the first verification information and the second verification information do not correspond, it means that the finger is constantly being triggered, but the fixed marketing terminal cannot make a selection, which indicates that there is an anomaly on the fixed marketing terminal. Therefore, the option confirmation selection can be automatically made based on the second interaction feature to achieve anomaly diagnosis and automatic error correction.

[0140] For example, if the first verification information (video analysis shows that option A was clicked) is inconsistent with the second verification information (the device records that option B was selected), it indicates that there is an interaction anomaly, that is, the touch layer offset causes the physical click position and the logical response position to be misaligned.

[0141] It is easy to understand that, based on the trigger trajectory of the second interaction feature, the system can actively select the option that the user actually clicks (rather than the option that the device mistakenly records) to ensure that the business process continues to execute. For example, if the user clicks the "Confirm" button visible in the video, but the device does not respond, the system can automatically trigger a confirmation operation and mark the abnormality for repair.

[0142] In some embodiments, a specific implementation of step S423 (the option selection based on the second interaction feature) includes:

[0143] S4231, determine the number of touches between the first identification object and the option in the second interaction feature. If the number of touches is greater than the preset number, then the corresponding option is taken as the target option.

[0144] Understandably, by counting the number of times the first identification object (finger cot) touches the option area, valid and invalid operations can be distinguished. For example, if the preset number of times is 2, and the user touches "Option A" 3 times in a row, it is determined that the user's true intention is to select that option, even if the device records that other options are selected.

[0145] Among them, the number of touches is the number of times the first identification object triggers the option, the preset number of touches is the number of touches set in advance, and the target option is the option whose number of touches is greater than the preset number of touches.

[0146] In some other embodiments, the specific implementation of step S42 (where the call to the verification logic corresponding to each verification group yields the corresponding anomaly diagnosis result) further includes:

[0147] S424, extract the first interaction feature, the first predicted attribute, and the verification group corresponding to the second interaction feature.

[0148] It is understandable that by extracting the first interaction feature, the first predicted attribute, and the verification group corresponding to the second interaction feature, the first interaction feature and the second interaction feature can be compared subsequently.

[0149] S425, if the first interaction feature corresponds to the second interaction feature, then extract the corresponding option result.

[0150] Understandably, the correspondence between the first and second interaction features indicates that the user's actions conform to the answering logic of the questionnaire. The corresponding option results can be extracted for subsequent comparison with the first predicted attribute.

[0151] S426 If the option result does not correspond to the first predicted attribute, an exception trigger tag corresponding to the fixed marketing end is generated and fed back to the fixed marketing end.

[0152] It is understandable that when the option result does not correspond to the first predicted attribute, it means that the corresponding personnel do not match the service target of the fixed marketing terminal. That is, the interaction personnel may have accidentally touched the fixed marketing terminal, thus generating an abnormal trigger label to be displayed on the fixed marketing terminal.

[0153] Among them, the abnormal trigger tag is a tag prompt message on the fixed marketing terminal that is unexpectedly triggered, so as to realize business monitoring and avoid entering and analyzing the corresponding information, which would affect the accuracy of data collection.

[0154] The present invention also provides a storage medium storing a computer program, which, when executed by a processor, is used to implement the methods provided in the various embodiments described above.

[0155] The storage medium can be a computer storage medium or a communication medium. A communication medium includes any medium that facilitates the transfer of computer programs from one location to another. A computer storage medium can be any available medium accessible to a general-purpose or special-purpose computer. For example, the storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be a component of the processor. The processor and storage medium can reside in an Application Specific Integrated Circuit (ASIC). This ASIC can also be located within a user device. Alternatively, the processor and storage medium can exist as discrete components in a communication device. Storage media can be read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage devices, etc.

[0156] The present invention also provides a program product including execution instructions stored in a storage medium. At least one processor of the device can read the execution instructions from the storage medium, and the execution instructions by the at least one processor cause the device to implement the methods provided in the various embodiments described above.

[0157] In the above-described terminal or server embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0158] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A real-time business monitoring and precise anomaly diagnosis system, characterized in that, include: The acquisition module captures images from the interactive locations on a fixed marketing terminal to obtain interactive videos; The extraction module extracts the first interactive features of the first interface of the business corresponding to the screen in the interactive video; The identification module identifies the attribute features of the first interacting person to obtain the first estimated attribute, and identifies the interaction features of the first interacting person to obtain the second interaction feature. The diagnostic module performs combined verification and diagnostics based on the first interaction feature, the first predicted attribute, and the second interaction feature to obtain abnormal diagnostic results for a fixed marketing end or the first interaction personnel, including: Multiple validation groups are obtained by combining and classifying the first interaction feature, the first predicted attribute, and the second interaction feature respectively. The corresponding verification logic for each verification group is invoked to obtain the corresponding anomaly diagnosis results, including: Extract the verification groups corresponding to the first and second interaction features; Determine the first interaction feature of the trigger point corresponding to the second interaction feature, and obtain the first verification information triggered by the video; Extract the second verification information corresponding to the first interaction feature of the fixed marketing terminal in real time. If the first verification information and the second verification information do not correspond, generate the abnormal diagnosis result of the fixed marketing terminal and select options based on the second interaction feature.

2. The system according to claim 1, characterized in that, The interactive video is obtained by capturing images at the interactive location of the fixed marketing terminal, including: After determining that the fixed marketing terminal has been triggered, the image acquisition device captures images of the fixed marketing terminal from top to bottom at a preset angle. Extract and crop the screen edge area of ​​the fixed marketing end in the image to obtain the interactive video.

3. The system according to claim 1, characterized in that, The extraction of the first interactive features of the first interface of the service corresponding to the screen from the interactive video includes: The first sub-feature is obtained by extracting the text with the first color feature from the interactive video, where the first color is the title color; The second sub-feature is obtained by extracting the text with the second color feature from the interactive video, where the second color is the option color; The first interactive feature is obtained by analyzing the feature relationship between the first sub-feature and the second sub-feature.

4. The system according to claim 3, characterized in that, The first interactive feature obtained by analyzing the feature relationship between the first sub-feature and the second sub-feature includes: Determine the first recognition direction of the interactive video, which is the top-to-bottom direction of the fixed marketing end; Based on the first recognition direction, the first sub-feature and the second sub-feature are recognized sequentially, and text extraction is performed based on OCR. When it is determined that all the first sub-features and the second sub-features have been recognized, or the text in the first sub-features and the second sub-features cannot be recognized, the recognition is stopped and the feature recognition sequence is obtained. By combining and labeling the feature recognition sequences, multiple first interactive features are obtained.

5. The system according to claim 4, characterized in that, The combination and labeling of the feature recognition sequences yields multiple first interactive features, including: If the first feature of the feature recognition sequence is not the first sub-feature, the feature recognition sequence is deemed to have failed the correctness check, and the interactive video is reacquired. If the feature recognition sequence passes the correctness check, then the first sub-feature and the second sub-feature of adjacent numbers are extracted and processed to obtain a first interactive feature. Add a separator mark to the first interactive feature of the last sequence number.

6. The system according to claim 1, characterized in that, The process of identifying the attribute features of the first interacting person to obtain a first estimated attribute, and identifying the interaction features of the first interacting person to obtain a second interaction feature, includes: Based on the attribute feature recognition of the first person interacting with the device at the fixed marketing terminal, the first estimated attribute of height information is obtained. By locating and recognizing the finger of the first person interacting, a second interaction feature with finger location is obtained.

7. The system according to claim 6, characterized in that, The finger location and recognition of the first interacting person yields a second interaction feature with finger location, including: The finger sleeve of the first interactive person is equipped with a first identification object, which is a finger sleeve with a third color feature; The location of the first object is identified to obtain the identification trajectory. When it is determined that the first object is in contact with the screen, a trigger point is generated. Based on the identification trajectory and the trigger point, a corresponding second interaction feature is generated.

8. The system according to claim 1, characterized in that, The option selection based on the second interaction feature includes: Determine the number of touches between the first identification object and the option in the second interaction feature. If the number of touches is greater than the preset number, then the corresponding option is taken as the target option.

9. The system according to claim 1, characterized in that, The call to the verification logic corresponding to each verification group yields corresponding anomaly diagnosis results, including: Extract the validation group corresponding to the first interaction feature, the first predicted attribute, and the second interaction feature; If the first interaction feature corresponds to the second interaction feature, then the corresponding option result is extracted; If the option result does not correspond to the first predicted attribute, an exception trigger tag corresponding to the fixed marketing end will be generated and fed back to the fixed marketing end.

Images