A system that detects problems in video streaming.

Abstract

This invention relates to a system (1) that detects problems such as freezing, mosaic, black screen and pixelation on devices (2) broadcasting over the internet using artificial intelligence and reports the detected errors. The AI-powered invention can obtain accurate and precise results by analyzing screenshots of the content on optimized target audience devices (2) that are transmitted to the system (1) at certain intervals with very little delay from the live broadcast data. This system (1) uses computer vision-based artificial intelligence to detect problems such as freezing, mosaic, black screen and pixelation experienced by users (A) watching broadcasts over the internet on their tablets, phones, computers and similar devices (2).

Description

1 TARIFF A system that detects problems in video streaming. Technical Area This invention addresses freezing, pixelation, and black screen issues on devices used for streaming over the internet. AI-powered detection of issues such as screen glitches and pixelation, and detection 5 It is related to a system that reports the errors it makes. State of the Art In current practices, all live broadcasts are blocked by broadcasting organizations' wall systems. Video on Demand (VoD) content is monitored in real-time by the operator. However, before being broadcast, it is checked for errors and also subjected to censorship. It is prepared by filtering according to the rules. Transfer of content to distribution networks, Problems that may occur during processing and delivery to the user's device It cannot be detected by this method. Text messages received by call centers via voice message or chatbot complaints are handled by the operator or these complaints are handled by various artificial intelligence 15 There are systems where reports are generated by organizing them with models. In this case, the user... Problems that may occur individually with the device or connection quality may be related to the published content. It might give the impression that a related problem has occurred, but this problem is not a general issue specific to the content. It might not be a problem. Performing operations by analyzing video or image on a live broadcast and 20 a system whose results can be considered accurately proven It is not available. Patent application number TR2023 / 004526, which is included in the prior art. in the document, artificial intelligence model in internet content and live TV broadcasting A system that identifies problems and suggests solutions is described. 25 The application document in question refers to video or image analysis over a streaming broadcast. a system that detects problems such as mosaic effects, black screens, and pixelation by doing so Not offered. 2 In conclusion, solutions that address the needs described above are relevant to the subject. Due to its shortcomings, it has become necessary to make improvements in the relevant technical field. Brief Description of Find The invention was created by drawing inspiration from existing situations and overcoming the aforementioned drawbacks. It aims to solve. 5 The purpose of this invention is to eliminate freezing, pixelation, and other issues on devices used for streaming over the internet. It detects problems such as black screen and pixelation using artificial intelligence, and It involves establishing a system that reports the errors it detects. The AI-powered invention is optimized with minimal latency from live streaming data. Screen 10 shows content from the target audience's devices being periodically uploaded to the system. By analyzing the images, precise and accurate results can be obtained. Thanks to the system described in the invention;  Assumptions based on packet loss, especially at the network layer It is wrong considering the activities,  Without analyzing the image content in the existing structures, relying solely on network data 15 problem identification based on  Using the entire video content for analysis,  Using high bandwidth in the analysis process,  High operator power usage and increased labor costs The fundamental problems mentioned above are being overcome. 20 The structural and characteristic features and all the advantages of the invention are given in the figure below. Thanks to the detailed explanation written with references to the diagram, it becomes clearer. This will be understood, and therefore the evaluation should also take this detailed explanation into account. It needs to be done by taking precautions. Figures That Will Help Understand the Discovery 25 Figure 1 is a schematic representation of the system that is the subject of the invention. Explanation of Part References 1. System 3 2. Device 3. Application 4. Complaint database 5. Image database 6. Analysis server 5 7. Central server 8. Primary artificial intelligence model 9. Secondary artificial intelligence model A. User B. Operator 10 Detailed Description of Find In this detailed description, the preferred configurations of the system (1) that is the subject of the invention are: This explanation is provided solely to facilitate a better understanding of the subject. This invention addresses freezing, pixelation, black screen issues (2) on devices broadcasting over the internet. AI-powered detection of issues such as screen glitches and pixelation. It is related to a system (1) that reports the errors it has made. The AI-powered invention is optimized with minimal latency from live streaming data. Screens of content from the (2) target audience devices are transferred to the system (1) at certain intervals By analyzing the images, precise and accurate results can be obtained. Thanks to the system (1) which is the subject of the invention; 20  Assumptions based on packet loss, especially at the network layer It is wrong considering the activities,  without analyzing the image content in existing configurations, relying solely on network data problem identification based on  Using the entire video content for analysis, 25  Using high bandwidth in the analysis process, 4  Use of high operator (B) power and increased labor costs These fundamental problems are being overcome. The system that is the subject of the invention, the schematic representation of which is given in Figure 1 (1);  The screen captures that are uploaded to the user's (A) device (2) at certain intervals field application (3), 5  complaints that store instant user (A) complaint records transmitted from the application (3) database (4),  Screenshots from the application (3) with specific tags (e.g. Internet Service Provider, location, platform, version, content, time the content was played, It stores information such as the duration of the content, mobile network, and location together, and this data is processed as follows: 10 image database listing in catalog form (5),  Screenshots transmitted from the application (3) at specific intervals to the cloud environment taking from and saving to the image database (5), image database (5) AI algorithms such as CNN, RNN, and LSTM are used on the recorded images. using the analysis, as a result of the analysis, the target audience devices (2) 15 analytics that optimizes content from live streaming data with very little latency. server (6) and  User (A) who came to the complaint unit and was recorded in the complaint database (4) combining complaints with analysis results, complaints and analysis results If they match, create an action plan and share it with user (A), user (A) 20 Freezing, mosaic, black screen and pixelation detected on the device (2) reporting broadcasting problems such as those to the operator (B) and thus remotely central server (7) which provides intervention and analysis capabilities It includes. This system (1) allows users (A) to watch broadcasts over the internet using tablets, phones, 25 Freezing, pixelation, black screen and other issues experienced on computers and similar devices (2) Artificial intelligence based on computer vision to detect problems like pixelation. uses. The system (1) consists of two main components. The first of these is in the end device (2) It is a working application (3). This application (3) is installed on the user's (A) device (2) or 30 The application (3) that he currently uses to watch broadcasts can be customized and used. The application in question (3) takes screenshots at regular intervals. For example, once every second. or more frequent screenshots can be taken. This frequency can be set by operator (B). The second most important component of the system (1) is the analysis server (6). The analysis server (6), received at certain intervals by application (3) running on user (A) device (2) The screenshots are analyzed by artificial intelligence algorithms, 5  whether there is a high degree of similarity between consecutive images identification and thus detection of freezing or mosaic-like problems. being done,  Determining whether it contains a high percentage of black pixels and thus identifying black Identifying screen problems, 10  whether there are groups of pixels that show consistent similarity in certain regions identification and thus detection of pixelation problems, It is configured to perform its operations. System (1); taking screenshots, transmitting images, artificial intelligence analysis, complaints It consists of analysis and matching, and reporting processes. 15 In the process of taking a screenshot, the application (3) running on the user’s (A) device (2), The employee takes screenshots at regular intervals determined by operator (B). The action of taking and sending screenshots is not performed on each device (2) and the target is Rule-based definitions are provided for the masses. Each platform (Android TV, iPhone, Android Phone, Windows etc.), each mobile technology (3G, 4G, 4.5G, 5G, Wi-Fi 2.4 Ghz, 20 GHz, etc.), each internet service provider (Türk Telekom, Vodafone, Turkcell, etc.), Sampling can be done based on location (province, district, neighborhood, street, etc.). In the image transmission process, the captured screenshots are analyzed via the cloud. The data is transmitted to the server (6). Information about the data (content, time the content is played, content (duration information, platform, mobile network, location etc.) are entered into the relevant tables (5) in the image database. It is recorded. In the AI ​​analysis process, the analysis server (6) analyzes the incoming screenshots. The analysis server (6) contains two different artificial intelligence models. The primary artificial intelligence Model (8) determines the similarity ratio by analyzing successive screenshots. For example, On the device (A) of the user (2) who is watching a football match, there is a scene where the ball is moving 30 The successive images will look very similar to each other. The primary artificial intelligence model (8) if this 6 If the similarity ratio is higher than it should be, freezing or mosaicking occurs. It assumes that the primary artificial intelligence model (8) is usually CNN (Convolutional Neural It is created using the Network) algorithm and with "Siamese Networks" methods. It performs visual analyses. The secondary artificial intelligence model (9) detects black screen and pixelation. It analyzes screenshots to identify problems. If there are 5 consecutive screens... If most of the images are black, the secondary AI model (9) black screen It determines that there is a problem. The secondary artificial intelligence model (9) is for pixelation detection, It examines the similarity rate in specific regions of the images. Continuous in a fixed region. High similarity can indicate the presence of pixelation. In the process of analyzing and matching complaints, the central server (7), analysis 10 It receives and evaluates the analysis results from the server (6). The central server (7) provides high-level analysis results. Similarity rate, freezing or mosaic effect, high black pixel count, black screen, The consistent similarity in certain areas is interpreted as a pixelation problem. Central Server (7) receives complaints from the call centre and the conversations are transcribed, other 15 from the chatbot (3) in the application or from the complaint pages The complaints are recorded in the complaint database (3) by deduplication. Central server (7), There is an overlap between the image analysis results and customer complaints for the relevant time period. If it sees this, it runs the ready-made action list, informs user (A), and The operator (B) returns the relevant answer. In the reporting process, the central server (7) analyzes the detected problems according to the results. It clusters the data and reports it to operator (B). Based on this information, operator (B) identifies the source of the problem. They can try to identify and find a solution. The working method of the system (1) can be explained with an example scenario. A user (A) internet He's watching a football match. There's a brief freeze during the film. The application (3) on the user's (A) device (2) regularly displays a 25 every second while watching a movie. He's taking screenshots for the umpteenth time. Screenshots taken during freezing are very different from before and after images. They will be similar. The analysis server (6) detects this similarity and produces a high similarity rate. The central server (7) interprets this high similarity rate as freezing and this content 30 It checks if there are any similar complaints for the user (A). If there are no similar complaints, the user (A) 7 inferring that there may be a problem with his own device (2) or network configuration User (A) can be automatically provided with a message and a list of instructions in this regard. Another If a complaint has been made, it is assumed that there may be a problem with the content and this will be communicated to operator (B). Information can be provided in this direction. Thanks to the system (1) which is the subject of the invention, operators (B) can improve the broadcast experience of users (A) 5 They can identify and address the problems affecting the region more quickly and effectively.

Claims

8 REQUESTS 1. Freezing, pixelation, black screen on devices broadcasting over the internet (2) and detects problems like pixelation in an AI-powered way. It is a system (1) that reports the errors it makes, and its feature is; 5  The device (A) of the user (2) is loaded and the screen is displayed at certain intervals Application that captures images (3),  Stores the instant user (A) complaint records transmitted from the application (3) complaint database (4),  Screenshots from the application (3) with specific tags (e.g. 10 Internet Service Provider, location, platform, version, content, content's When played, information such as the duration of the content, mobile network, and location is displayed together. image database (5), which stores and lists these data in a catalog,  Screenshots transmitted from the application (3) at specific intervals to the cloud environment receiving it from and saving it to the image database (5), image database 15 (5) Artificial intelligence such as CNN, RNN, LSTM on the images it records performs analysis using algorithms, and targets the result of the analysis. content on mass devices (2) with very little delay from live broadcast data optimizing analysis server (6) and  User (A) 20 who came to the complaint unit and was recorded in the complaint database (4) combining complaints with analysis results, complaints and analysis results If they match, create an action plan and share it with user (A). Freezing, mosaic, black screen and detected on the device (2) reporting broadcast problems such as pixelation to operator (B) and thus Central server (7) 25 that provides remote intervention and analysis capabilities It includes.

2. The system mentioned in accordance with claim 1 is (1), and its feature is; on user (A) device (2) The screen captures taken at certain intervals by the running application (3) are artificial. analyzed by intelligence algorithms, 9  whether there is a high degree of similarity between consecutive images identification and thus detection of freezing or mosaic-like problems. being done,  Determining whether it contains a high percentage of black pixels and thus Identifying black screen issues, 5  groups of pixels that show consistent similarity in certain regions determining that it is not present and thus identifying pixelation problems. to be done The analysis server (6) is configured to perform its operations. It includes. 10 3. The system mentioned in accordance with claim 1 is (1), and its feature is; sequential screen images by analyzing and determining the similarity rate, if this similarity rate exists CNN assumes that freezing or pixelation occurs if the value is higher than it should be. Created using the (Convolutional Neural Network) algorithm and called "Siamese Primary artificial intelligence model (8) 15 that performs visual analyses with "Networks" methods It includes the analysis server (6).

4. The system mentioned in accordance with claim 1 is (1), and its feature is black screen and pixelation. Analyzing screenshots to identify problems, if consecutive screens If a large portion of the images are black, it indicates a black screen issue. For pixelation detection, the similarity ratio in specific regions of the images is set to 20. The observer found that pixelation occurs if there is consistently high similarity in a fixed region. It includes an analysis server (6) with a secondary artificial intelligence model (9) that understands.

5. The system mentioned in accordance with claim 1 is (1), and its feature is; from the analysis server (6) Receiving and evaluating the analysis results, freezing the high similarity rate or Mosaic, high percentage of black pixels, black screen, consistently 25 in certain areas Those who interpreted the similarity as a pixelation problem, and who came to the call center... Complaints that conversations are transcribed by the chatbot (3) in the application (chatbot) or other complaints coming from complaint pages are stored in the complaint database. (3) recording by singularizing, between the image analysis result and customer complaints If it detects an overlap for the relevant time period, it runs the ready-made action list, 30 central server (7) that informs user (A) and returns the relevant answer to operator (B) It includes.

Images