Method and system for determining content boundaries

Abstract

This provides a method for determining content boundaries. [Solution] The method includes: acquiring display data including a first content object; determining a set of bounding boxes based on the display data using an artificial intelligence detection model that detects content objects based on the display data; determining a set of contours based on the display data using an image processing system that includes one or more computer vision filters or functions that transform the display data; determining correspondence scores for the first bounding boxes and the first contours; determining that the first bounding boxes and the first contours correspond to each other based on the correspondence scores; and determining the content boundary of the first content object based on the first contours.

Description

[Background technology]

[0001] Display data can represent various content objects, such as text and images, that will be displayed using a display device. For example, display data can correspond to a web page that will be rendered using a display device, and the web page may contain one or more images and text blocks.

[0002] Determining the boundaries of content objects is important for, for example, adjusting the display settings of a display device, saving or optimizing the power consumption of a display device, improving the rendering of display data, or a combination of these. Inaccurate content boundaries can result in undesirable and distracting visual artifacts in display rendering, obscuring parts of one or more content objects and degrading the performance of the display device (e.g., increased power consumption, reduced display quality).

[0003] Various artificial intelligence (AI) detection models exist for detecting content objects of one or more content types (e.g., image types) and locating the detected content objects on a display, for example, using bounding boxes. However, these AI detection models often return inaccurate content boundaries. For example, an AI detection method that uses a rectangular bounding box to locate content data cannot fit content objects with non-rectangular boundaries. Therefore, there is a need to accurately determine the boundaries of content that is or will be rendered on a display. [Overview of the project]

[0004] This summary is provided to introduce a selection of concepts that will be further explained in the detailed description below. This summary is not intended to identify any significant or essential features of the claimed subject matter, nor is it intended to be used to help limit the scope of the claimed subject matter.

[0005] In general, in one embodiment, the embodiment relates to a method for determining content boundaries. The method includes: acquiring display data including a first content object; determining a set of bounding boxes based on the display data using an artificial intelligence detection model configured to detect content objects based on the display data; the set of bounding boxes includes a first bounding box associated with the first content object; determining a set of contours based on the display data using an image processing system including one or more computer vision filters or functions that transform the display data; the set of contours includes a first contour; calculating a correspondence score between the first bounding boxes and the first contour; and determining that the first bounding boxes and the first contour correspond to each other based on the correspondence score; further including determining the content boundary of the first content object based on the first contour; further including determining the display settings of a display device based on the content boundary of the first content object; and adjusting the display settings of the display device to the determined display settings.

[0006] In general, in one embodiment, the embodiment relates to a computer system for content boundary determination. The computer system includes an artificial intelligence detection model configured to receive display data, detect content objects in the display data, and output a set of bounding boxes. The computer system further includes an image processing system configured to receive display data and output a set of contours, the image processing system includes one or more computer vision filters or functions for transforming the display data. The computer system further includes a correspondence system configured to determine one or more correspondences between the set of bounding boxes and the set of contours. The computer system is configured to acquire display data containing a first content object. The computer system is further configured to determine a set of bounding boxes based on the display data using the artificial intelligence detection model, the set of bounding boxes including a first bounding box associated with the first content object. The computer system is further configured to determine a set of contours based on the display data using the image processing system, the set of contours including a first contour. The computer system is further configured to calculate a correspondence score for a first bounding box and a first contour, and to determine, using a correspondence system, that the first bounding box and the first contour correspond to each other based on the correspondence score. The computer system is further configured to determine the content boundary of a first content object based on the first contour. The computer system is further configured to determine the display settings of a display device based on the content boundary of a first content object, and to adjust the display settings of the display device to the determined display settings. [Brief explanation of the drawing]

[0007] [Figure 1] This figure shows a display rendering display data including content objects according to one or more embodiments of the present disclosure.

[0008] [Figure 2] A diagram showing the content boundary and bounding box of a content object according to one or more embodiments of the present disclosure.

[0009] [Figure 3] A diagram showing a content boundary determination system according to one or more embodiments of the present disclosure.

[0010] [Figure 4] A diagram showing display data processed by a content boundary determination system according to one or more embodiments of the present disclosure.

[0011] [Figure 5] A diagram showing the use of an artificial intelligence detection model according to one or more embodiments of the present disclosure.

[0012] [Figure 6] A diagram showing a comparison of a bounding box and a content boundary according to one or more embodiments of the present disclosure.

[0013] [Figure 7] A diagram showing an image processing system according to one or more embodiments of the present disclosure.

[0014] [Figure 8] A diagram showing the use of an image processing system according to one or more embodiments of the present disclosure.

[0015] [Figure 9] A diagram showing a flowchart according to one or more embodiments of the present disclosure.

[0016] [Figure 10] A diagram showing the calculation of a correspondence score between a set of bounding boxes and a set of contours according to one or more embodiments of the present disclosure.

[0017] [Figure 11] This figure shows the content boundaries determined by one or more embodiments of the present disclosure.

[0018] [Figure 12] This figure shows a method for determining the content boundaries of content objects in display data according to one or more embodiments of the present disclosure. [Modes for carrying out the invention]

[0019] Specific embodiments of this disclosure will be described in detail below with reference to the drawings. Similar elements in various drawings are indicated by the same reference numerals for consistency.

[0020] The following detailed description of embodiments of this disclosure includes many specific details to provide a complete understanding of the invention. However, it will be apparent to those skilled in the art that the invention can be carried out without these specific details. In other examples, well-known features are not described in detail to avoid unnecessarily complicating the description.

[0021] Throughout this application, ordinal numbers (e.g., 1st, 2nd, 3rd) may be used as adjectives for elements (e.g., any noun in this application). The use of ordinal numbers is not intended to imply or create a particular order of elements, nor to limit any element to only a single element, unless explicitly disclosed, such as by using the terms “before,” “after,” “single,” and other such terms. Rather, the use of ordinal numbers is for distinguishing between elements. For example, unlike the 2nd element, the 1st element may encompass two or more elements, and may follow (or precede) the 2nd element in the order of elements.

[0022] The embodiments disclosed herein relate, in general, to a content boundary determination system capable of accurately and quickly (e.g., in real time) detecting the boundaries of content objects that will be displayed, rendered, or displayed using a display.

[0023] Figure 1 shows an exemplary display (100). The display (100) may be part of a display device (not shown), such as a tablet, laptop, monitor, touchscreen, or other device. In Figure 1, the display (100) shows a menu (102) containing information for the user, such as the current time and battery percentage of the display device. The display (100) is further used to render display data. The display data may include various content objects. Furthermore, in some implementations, content objects are categorized according to their content type. Examples of content types include, but are not limited to, image type and text type. Additional content types may include icons, navigation buttons, hyperlinks, etc. Two content objects, namely a first content object (104) and a second content object (106), are shown in Figure 1. The first content object (104) is an image and therefore has a content type of image type (or simply image). The second content object is text and has a content type of text type (or simply text). Note that the text depicted in the second content object (106) in Figure 1 is placeholder text used to indicate the presence of text, but has no meaning whatsoever.

[0024] Detecting content objects on a display and their locations can be important. For example, a display device, including a display, may adjust the display settings based on content objects. Adjusting display settings based on content objects may be beneficial for one or more reasons, such as selectively increasing the display resolution based on the location of the content objects, reducing the power consumption of the display (e.g., to maintain the battery life of the display device), selectively changing the bit depth of pixels, and reducing latency. For example, an area of ​​the display associated with an image may be adjusted to have a higher resolution than an area of ​​the display associated with text. Similarly, an area of ​​the display may be set to color, grayscale, black and white, and movie mode based on the content objects it contains. Furthermore, the bit depth of pixels within an area of ​​the display may be changed based on the content objects within the area.

[0025] As an example, the display settings of a display device, including the display (100) in Figure 1, can be adjusted based on rendered content objects or display data. In this example, in response to the detection that the type of the first content object (104) is an image, the area of ​​the display (100) associated with the first content object (104) is set to high resolution, high bit depth, and color mode. Furthermore, in response to the detection that the type of the second content object (106) is text, the area of ​​the display (100) associated with the second content object (106) is set to low resolution, low bit depth, and non-color mode.

[0026] "Low" resolution and "low" bit depth are described in contrast to "high" resolution and "high" bit depth. In one or more embodiments, a color mode indicates that each pixel, or each effective pixel, or discretized portion of the display has three or four channels that are visualized as a color when viewed as a whole. For example, three channels could correspond to red, green, and blue. Furthermore, in one or more embodiments, a "high" bit depth is 8 bits, so that each pixel, effective pixel, or channel of a pixel can take one of 256 values, while a "low" bit depth is 1 bit, corresponding to two possible values ​​(e.g., 0 or 1, black or white).

[0027] As shown in the example in Figure 1, the display settings are adjusted based on the content objects, more specifically, the type and position of each content object. In a given example, the display area associated with the first content object (104) has one or more relatively high resolutions and bit depths (e.g., 8 bits) and is set to color mode. The display area associated with the second content object has one or more relatively low resolutions and bit depths (e.g., 1 bit) and is set to monochrome mode. In some cases, the mode, such as monochrome or color, may be entirely specified by the bit depth. Using these display settings, the quality of the image contained by the first content object (104) can be maintained while reducing the power consumption of the display device by not using higher resolutions, bit depths, and colors than required to render the text of the second content object (106).

[0028] Other adjustments to the display settings of a display device can be made based on content objects of the display data being rendered or to be rendered, without departing from the scope of this disclosure. For example, the area of ​​the display related to a detected content object (e.g., an image) can be enhanced using super-resolution techniques or methods.

[0029] Figure 2 shows an exemplary content object (200) having an image type. The exemplary content object (200) can be the first content object (104) in Figure 1. The actual boundaries (202) of the exemplary content object (200) on the display are shown as solid lines in Figure 2. Various artificial intelligence detection models exist for detecting content objects (or display data) in a display and outputting the location or area associated with the content object. A brief description of the artificial intelligence models is provided later in this disclosure. An artificial intelligence detection model can detect images in the display data and return a bounding box as output for each detected image. The bounding box encloses or segments the area of ​​the display corresponding to the detected content object (e.g., an image). The bounding box does not have to be strictly a “box” or rectangular shape. In some cases, the artificial intelligence detection model is configured to generate regular or irregular polygons of a specified shape or type (e.g., an irregular quadrilateral). The bounding box can be represented in a variety of ways. For example, in the case of a rectangular bounding box, the bounding box can be represented by specifying the in-display positions of two opposing corners (opposing in both the first and second directions), such as the top-left and bottom-right corners, or by specifying the center of the bounding box along with its width and height.

[0030] Generally, artificial intelligence (AI) detection models can rapidly process display data to detect content objects that are rendered or will be rendered on a display according to one or more content types, and return a representation of the display area associated with the detected content object (e.g., a bounding box). However, the area representation of detected content objects returned by AI detection models is inaccurate. That is, the bounding box returned by the AI ​​detection model, whether rectangular or other polygonal, does not precisely match the boundary of the associated detected content object. Figure 2 shows an example where an AI detection model detects an exemplary content object (200) and generates a rectangular bounding box (204) representing the display area corresponding to the exemplary content object (200) according to the AI ​​detection method. As shown in the figure, the bounding box (204) does not precisely match the actual boundary (202) of the exemplary content object (200). In the example in Figure 2, the bounding box (204) extends beyond the actual boundary (202) to the right of the exemplary content object (200), resulting in a margin (206). In addition, the bounding box (204) does not extend to the full vertical extent of the exemplary content object (200), resulting in deletion (208), cutting, or cropping of the exemplary content object (200) at its bottom. Furthermore, the actual boundary (202) of the exemplary content object (200) has rounded edges, and the rectangular bounding box (204) has square corners, resulting in a shifted corner area (210) that is included by the bounding box (204) but is not part of the exemplary content object (200).

[0031] Inaccurate boundaries of content objects can cause artifacts and imperfections within the display. For example, display settings may be adjusted to enhance the area of ​​the display associated with content objects having an image type (e.g., increased resolution, increased bit depth, etc.) (i.e., enhance the portion of the display containing the image). Such enhancements applied to the exemplary content object (200) in Figure 2 based on a bounding box (204) generated by an artificial intelligence detection model may result in unnecessary enhancements to the margins (206) and corner areas (210). These unnecessary enhancements increase the power consumption of the display device, including the display, and can cause one or more artifacts in the display, such as a “halo effect” (bright or contrasting borders) that distracts the user's attention. Similarly, if deletion (208) does not completely cover the bottom of the exemplary content object (200), a portion of the exemplary content object (200) will not be enhanced, reducing its quality or visibility compared to the enhanced portion included by the bounding box (204). Therefore, the embodiments disclosed herein relate, as a whole, to content boundary determination systems that accurately and quickly (e.g., in real time) determine the actual boundaries of content objects in cases where the boundaries of content objects cannot be accurately determined by the use of artificial intelligence detection models alone.

[0032] Figure 3 shows a block diagram of a content boundary determination system (300) according to one or more embodiments. The content boundary determination system (300) comprises an artificial intelligence detection model (310), an image processing system (320), and a correspondence system (330). As will be described in detail below, the artificial intelligence detection model (310) and the image processing system (320) each independently process display data (i.e., what has been or will be rendered on the display) and return area representations (i.e., areas on the display) that are thought to correspond to content objects. Specifically, the artificial intelligence detection model (310) returns a set of bounding boxes, each bounding box in the set of bounding boxes relating to a content object detected by the artificial intelligence detection model (310), and the image processing system (320) returns a set of contours. The correspondence system (330) processes the set of bounding boxes and the set of contours to determine one or more content boundaries ("content boundaries"), which represent the actual areas or boundaries of the detected content objects.

[0033] According to one or more embodiments, the content boundary determination system (300) receives display data for or on a display and returns content boundaries. Figure 3 shows the reception of display data (305) and the transmission of determined content boundaries (340). In one or more embodiments, the content boundary determination system (300) transmits the content boundaries to another system or a display device including a display (340), for example, to adjust the display settings of the display device based on the content boundaries. The other system may be a computer system. The computer system may be external to the content boundary determination system (300) or may include the content boundary determination system (300). In some embodiments, the content boundary determination system (300) is used, for example, with a computer system that operates a display device. The content boundary determination system (300) can be associated with a computer system by being included in the computer system or by communicating electrically with the computer system. Thus, determined content boundaries or a set of determined content boundaries can be translated into commands for the computer system. For example, the commands can adjust the display settings of the display device.

[0034] The computer systems referenced herein are intended to encompass any computer device, including servers, desktop computers, laptop computers, smartphones, personal data assistants (PDAs), tablet computer devices, one or more processors in these devices, or any other suitable processing devices, including physical or virtual instances (or both) of computer devices. A computer system may include, for example, one or more auxiliary devices for receiving input and processing or displaying output. Auxiliary devices may include keypads, keyboards, touchscreens, or other input devices capable of accepting user information (e.g., joysticks). Auxiliary devices may further include displays or other output devices that transmit information relating to the operation of the computer system, including digital data, visual or auditory information (or a combination of information), or graphical user interfaces. Therefore, in some cases, a computer system includes a display device.

[0035] A computer system includes one or more computer processors and data storage devices such as one or more non-persistent storage devices (e.g., volatile memory such as random access memory (RAM) and cache memory) and persistent storage devices (e.g., optical drives such as hard disks, compact disc (CD) drives or digital versatile disc (DVD) drives, and flash memory). A processor may be part or all of an integrated circuit for processing instructions. For example, a processor may be one or more cores or microcores, or may include them. A computer system may further include a communication interface which may include integrated circuits for connecting to a network (e.g., a local area network (LAN), a wide area network (WAN) such as the Internet, a mobile network, or any other type of network) and / or another device.

[0036] In some embodiments, the content boundary determination system (300) or its elements are stored in a non-temporary machine-readable medium, and the processes or steps of the content boundary determination system (300) are executed using one or more computer processors. The non-temporary machine-readable medium includes, or may include, a data storage device of a computer system. That is, if the content boundary determination system (300) of Figure 3 is used in or with a computer system such as a display device, the content boundary determination system (300) may be incorporated into the computer system in terms of hardware and / or functionality.

[0037] Figure 4 shows in detail a content boundary determination system (300) according to one or more embodiments. As shown in Figure 4, the content boundary determination system (300) receives and processes display data (410), which includes content objects that are rendered or will be rendered on a display. The display data is processed independently by both an artificial intelligence detection model (310) and an image processing system (320).

[0038] The artificial intelligence detection model (310) is configured to detect content objects in the display and return a set of bounding boxes (420) containing the bounding box for each detected content object. In one or more embodiments, the artificial intelligence detection model (310) is further configured to detect content objects and return the associated bounding boxes for one or more given content types (e.g., image types). That is, the artificial intelligence detection model (310) can be configured according to the content type (415). For example, the artificial intelligence detection model (310) can be configured to detect images that are rendered or will be rendered on the display (detection of image-type content objects). In this example, for each image detected in the display data, a bounding box is returned by the artificial intelligence detection model (310) (in the set of bounding boxes (420)).

[0039] Figure 5 shows an example of an artificial intelligence detection model (310) that processes display data (e.g., display data A(510)) and returns a set of bounding boxes (e.g., set of bounding boxes A(520)). In particular, Figure 5 shows an artificial intelligence detection model (310) processing an exemplary instance of display data referred to as display data A(510). Furthermore, in the example of Figure 5, the artificial intelligence detection model (310) is identified as being configured to detect an image (515) (i.e., content type (415) is image). As can be seen from the figure, display data A(510) is data from a web page of a news site and includes two images, a first image (512) and a second image (514). Here, the cross-hatching used on images (512, 514) of display data A(510) is used to indicate that the display data is in color or contains color portions (e.g., the first and second images (512, 514) are color images). The artificial intelligence detection model (310) processes display data (e.g., display data A(510)), detects content objects (e.g., images), and returns a set of bounding boxes (e.g., set of bounding boxes A(520)). As can be seen from the example in Figure 5, set of bounding boxes A(520) includes two bounding boxes, namely a first bounding box (522) and a second bounding box (524). Each bounding box in set of bounding boxes (420) can be said to be associated with a content object in the display data (410). For example, in Figure 5, the first bounding box (522) is associated with the first image (512), and the second bounding box (524) is associated with the second image (514). In other words, the bounding boxes provide an approximate location, region, or area representation of the content object in the display data.According to one or more embodiments, an artificial intelligence detection model (310) detects content objects and provides an approximate area representation (e.g., a bounding box), and the boundaries of the detected content objects are determined using an image processing system (320) and a corresponding system (330).

[0040] In one or more embodiments, the artificial intelligence detection model (310) is based on a You Only Look Once (YOLO) object detection model. Various versions of YOLO exist, differing in terms of the type of layers used, the resolution of the training data, etc. However, a defining characteristic of all YOLO versions is that they can detect multiple objects of varying scales (e.g., content objects) in a single pass. Furthermore, recent YOLO architectures divide the input display data into grid cells, each grid cell having one or more associated anchor boxes used as potential bounding boxes. A brief overview of artificial intelligence and general or applicable model types will be provided later in this disclosure. The artificial intelligence detection model (310) may further include various pre-processing and post-processing steps such as normalization of pixel values ​​of the display data, cropping, etc.

[0041] Following the example in Figure 5, Figure 6 shows the first bounding box (522) and the second bounding box (524) of bounding box set A(520), along with the actual boundaries of the first image (512) and the second image (514) of display data A(510). The actual boundary (or true boundary or ground truth boundary) of the first image (512) is referred to as the first exemplary boundary (612) in Figure 6, and the actual boundary of the second image (514) is referred to as the second exemplary boundary (614) in Figure 6. As can be seen, the first bounding box (522) does not exactly coincide with the first exemplary boundary (612). Similarly, the second bounding box (524) does not exactly coincide with the second exemplary boundary (614). Therefore, for example, adjusting display settings based on an inaccurate bounding box (522,524) to change the resolution or bit depth of a display area may result in artifacts (e.g., halo effect) or defects (e.g., deleted images) appearing on the display.

[0042] Returning to Figure 4, the display data (410) is processed by the image processing system (320) of the content boundary determination system (300). The image processing system (320), having processed the display data, returns a set of contours. The contours accurately represent the actual boundaries of the associated content objects. The image processing system (320) includes one or more computer vision filters or functions (325) that modify the input image, e.g., the display data, or apply transformations to the display data. The computer vision filters or functions include the concepts of both image filtering and image warping, where image filtering modifies the extent of an image (i.e., pixel values) (e.g., the color of an image is changed without changing the pixel position), and image warping modifies the region of an image (i.e., pixel position) (e.g., a point is mapped to another point without changing its color). The computer vision filters or functions are used to modify or improve image properties and / or to extract valuable information from an image. The computer vision filters and functions may include, among other things, operations such as convolution by different kernels, edge detection, thresholding, morphological filtering, or dilation and deflation. In the context of computer vision filters and functions (325), display data (410) can be considered as images for image processing.

[0043] Figure 7 shows an image processing system (320) according to one or more embodiments. As can be seen in Figure 7, the image processing system applies an ordered set or sequence of computer vision functions or filters (325). According to one or more embodiments, the computer vision functions or filters (325) of the image processing system (320) include a grayscale converter (710), an edge filter (720), a morphological filter (730) including one or more morphological operations, and a contour extractor (740). In one or more embodiments, the order of the computer vision functions or filters (325) is as shown in Figure 7. That is, the display data (410) is first processed by the grayscale converter (710). Next, the output of the grayscale converter (710) is processed by the edge filter (720). Then, the output of the edge filter (720) is processed by the morphological filter (730). Next, the output of the morphological filter (730) is processed by the contour extractor (740), and the output of the contour extractor is a set of contours (430).

[0044] The grayscale converter (710) removes color from its input, if present, and outputs a version of the input that uses only the gray shade range from white to black. Various methods exist for converting color data to grayscale, and any known method may be used by the grayscale converter (710). Typically, these methods calculate grayscale values ​​to preserve the luminance of the original color input.

[0045] An edge filter (720) identifies edges in its input. The edge filter (720) can use one or more mathematical methods for identifying edges in the input, including search-based and zero-crossing-based methods. A search-based method can detect edges by first calculating a measure of edge intensity, such as the magnitude of the gradient, and then searching for local maxima of the edge intensity. A zero-crossing-based method generally applies a second-order differential representation to the input (e.g., pixels) and then searches for zero crossings to detect the location of edges. The edge filter (720) can also apply one or more preprocessing steps (e.g., a Gaussian filter), such as a smoothing or noise reduction step, to its input. In one or more embodiments, the edge filter (720) is a Canny edge detector.

[0046] A morphological filter (730) applies one or more operations to its input, generally adjusting the values ​​of the input pixels based on the values ​​of neighboring pixels. These operations are shape-based. Morphological operations may include, but are not limited to, contraction (to cut connected objects), expansion (to grow foreground pixels), opening (to contract, then expand to remove small foreground objects), and closing (to expand, then contract to remove small holes). In one or more embodiments, the morphological filter (730) applies a closing operation to its input. A closing operation improves the continuity of a contour by connecting sections of a contour cut by a small number of pixels, for example, and assists the contour extraction process described below.

[0047] The contour extractor (740) determines and returns the boundaries of an object (e.g., a content object) in its input. The contour extractor (740) can apply one or more mathematical concepts or algorithms to detect and extract contours. For example, mathematics defines the convex hull of a set of points as the smallest convex polygon enclosing all the points in the set. Thus, the vertices of a contour can be calculated using the convex hull. As another example, the journal article "Topological structural analysis of digitized binary images by border following" by Satoshi Suzuki and Keiichi A be details an algorithm for contour extraction (see Computer Vision, Graphics, and Image Processing, Volume 30, Issue 1, 1985, pp. 32-46, ISSN 0734-189X). In one or more embodiments, the contour extractor (740) applies or is based on the algorithm of Satoshi Suzuki and Keiichi A be.

[0048] Figure 8 shows an example of an image processing system (320) that processes display data (e.g., display data A(510)) and returns a set of contours (e.g., set of contours A(830)). In particular, Figure 8 shows an image processing system (320) processing an exemplary instance of display data referred to as display data A(510). Furthermore, in the example of Figure 8, the image processing system (320) is specified to include one or more computer vision (CV) filters or functions (325), including a grayscale converter (710), an edge filter (720), a morphological filter (730), and a contour extractor (740). Furthermore, Figure 8 shows the order in which one or more CV filters or functions (325) are applied.

[0049] As can be seen in Figure 8, display data A(510) is display data for a news site's webpage and includes two images, namely the first image (512) and the second image (514). Here, the cross-hatching used for images (512, 514) in display data A(510) is used to indicate that the display data is in color or contains color portions (for example, the first and second images (512, 514) are color images).

[0050] Referring to Figure 8, an exemplary image processing system (320) receives display data (e.g., display data A(510)) and applies a grayscale converter (710) to the display data (e.g., display data A(510)). The output of the grayscale converter (710) is called the grayscale converter output (e.g., grayscale converter output A(815)). Figure 8 shows the grayscale converter output A(815) resulting from applying the grayscale converter (710) to display data A(510). As can be seen from the figure, the grayscale converter output A(815) no longer contains cross-hatching to indicate that the data is grayscale (i.e., has no colored areas). Next, the grayscale converter output (e.g., grayscale converter output A(815)) is processed by an edge filter (720). The output of the edge filter (720) is called the edge filter output (e.g., edge filter output A(825)). The edge filter output is processed by a morphological filter (730). The output of the morphological filter (730) is called the morphological filter output (e.g., morphological filter output A(835)). In the example in Figure 8, the morphological filter (730) includes a dilation operation. Finally, in the image processing system (320), the morphological filter output is processed using a contour extractor (740) that returns a set of contours (e.g., set of contours A(830)). As seen in the example in Figure 8, set of contours A(830) includes four contours: a first contour (802), a second contour (804), a third contour (806), and a fourth contour (808). The contours are associated with a bounding box, and therefore a content object, using the corresponding system (330).

[0051] Returning to Figure 4, the corresponding system (330) receives a set of bounding boxes (420) generated by the artificial intelligence detection model (310) and a set of contours (430) generated by the image processing system (320), both of which operate independently on the display data (410). The corresponding system (330) compares the set of bounding boxes (420) and the set of contours (430) with determined corresponding pairs, each pair consisting of one bounding box and one contour. Once paired, the contour determined using the image processing system (320) is the content boundary of the content object detected by the artificial intelligence detection model (310). For example, the set of bounding boxes may include a first bounding box and a second bounding box. Similarly, the set of contours may include a first contour and a second contour. The corresponding system (330) may determine that the first bounding box corresponds to a first contour that forms a first pair. Continuing this example, the corresponding system (330) may further determine that the second bounding box does not correspond to a second contour, and therefore the second bounding box and the second contour do not form a pair. The bounding box-contour pair is used to form a content boundary (340). Generally, a given content boundary is determined based on its associated contour. In one or more embodiments, the content boundary of a content object is set to the contour of a bounding box-contour pair, and the bounding box is associated with the content object. That is, the bounding box is associated with the content object, and the content boundary of that content object is the contour paired with the bounding box by the corresponding system (330). In other embodiments, the content boundary is a weighted average of the paired bounding boxes and contours. In some implementations, the weights used in the weighted average are based on the confidence level or uncertainty associated with one or more of the bounding boxes and contours.For example, the artificial intelligence detection model (310) can be further configured to output a confidence or uncertainty that the bounding box closely fits the detected content object. The confidence or uncertainty can then be used to weight the aggregation of paired bounding boxes and contours when forming the content boundary.

[0052] Figure 9 shows a flowchart (900). According to one or more embodiments, the corresponding system (330) implements the flowchart (900) of Figure 9 to determine the corresponding contour and bounding box. As shown in Figure 9, in block 902, a set of bounding boxes (e.g., set of bounding boxes A(520)) is obtained by the corresponding system (330). Using mathematical notation, the set of bounding boxes is {b i Represented as}, i is used to index the bounding boxes within the set. For example, if the set of bounding boxes contains three bounding boxes, these bounding boxes can be referred to individually as b1, b2, and b3.

[0053] In block 904, a set of contours (e.g., set of contours A(830)) is obtained by the corresponding system (330). Using mathematical notation, the set of contours is {c j Represented as}, j is used to index the contours within the set. For example, if the set of contours contains four contours, these contours can be referred to individually as c1, c2, c3, and c4.

[0054] Continuing with Figure 9, block 906 encloses blocks 908 through 914, and the enclosed blocks form a set of bounding boxes, {b i This indicates that it applies to each bounding box in the set of bounding boxes A(520) (for example). In some embodiments, block 906 is a set of bounding boxes, {bi} is executed sequentially or iteratively by cycling through. In other embodiments, block 906, or rather the enclosed blocks thereof, are executed in parallel. Alternatively, block 906 can be adapted to each contour within a set of contours, {c j} as indicated by "{c j} for each c within j each". This alternative embodiment requires adaptation to block 908 described below.

[0055] In block 908, contours from a set of contours, {c j} are compared with a given bounding box b i [[ID=1 to determine whether the contour corresponds to the given bounding box. According to one or more embodiments, in block 908, a similarity function is used to calculate a correspondence score for each contour within a set of contours, {c i} with respect to a given bounding box, b j . That is, a correspondence score, S i,j is determined, and S i,j indicates the correspondence between the i-th bounding box and the j-th contour. In one or more embodiments, the similarity function is an intersection over union (IoU) function. The IoU function is the ratio of the intersection and union of two shapes. In the context of the present disclosure, the two shapes are the i-th bounding box and the j-th contour. In block 908, according to the similarity function, the contour, c i with the highest correspondence score for a given bounding box b j is identified or discovered if such a highest correspondence score exists.

[0056] In FIG. 9, block 908 determines the contour c i from a set of contours, {c j} having the largest intersection over union for a given bounding box b jThis specifies that it will be discovered. Mathematically, this is written as follows:

number

[0057] In some cases, two or more contours from a set of contours may have the same maximum correspondence score. In such cases, the solution can be resolved in various ways as selected by the user. For example, if two or more contours share the highest correspondence score, the first contour may be returned, no contours may be returned, or a given bounding box, b i If two or more contours share the best correspondence score, the correspondence scores of those contours can be calculated, if any, in relation to other bounding boxes in the set of bounding boxes, to determine which contour to return.

[0058] In an alternative embodiment, if block 906 represents an operation across contours within a set of contours, block 908 represents the bounding box b that has the highest correspondence score according to a given similarity function from the set of bounding boxes. i It is adapted to identify. For example, if we use the IoU function as the similarity function, block 908 can be expressed mathematically as follows:

number

[0059] In block 910, two conditions are checked. The first condition is that a given bounding box b has a similarity function. i When evaluated, it is checked that the contour from the set of contours guarantees the generation of the highest corresponding score. That is, the first condition is to check that block 908 has produced a valid output (discovered a contour from the set of contours). Block 908 is a contour, c j In relation to whether or not it was determined and output by block 908, this condition is "such as c jThe second condition is that given bounding box b i and the identified or discovered contour c j The second condition is checked to see if the correspondence score between (i.e., a contour from the set of contours that has the highest correspondence score with a given bounding box) and the found contour exceeds a threshold T. In one or more embodiments, the similarity function is the IoU function, and the correspondence score is the intersection over union of the given bounding box and the found contour. In one or more embodiments where the IoU function is used as the similarity function to determine the correspondence score, the threshold is set to 0.80. Block 910 then checks if the second condition is given bounding box b i The discovered contour c j The intersection overunion with is indicated as exceeding the threshold T. If at least one of the first and second conditions is not met in block 910, flowchart (900) proceeds to block 912. Block 912 represents "Pass", null, or no action ("no-op"), so that no action is performed. From block 912, flowchart (900) can return to block 908 if additional bounding boxes require evaluation by block 906, or a set of bounding boxes, {b i Block 916 can be reached if all bounding boxes within} have been evaluated. If both the first and second conditions of block 910 are met, the flowchart (900) proceeds to block 914. In block 914, given bounding box b i The discovered contour c j and are paired. Furthermore, given bounding box b i The content boundary for content objects detected by is the discovered contour c j Determined based on the discovered contour c. j is a given bounding box b iIt is determined that this is the boundary of the content object detected by [the system]. That is, in these embodiments, given bounding box b i The content boundary for the content object associated with it is the discovered contour c j It is set to. In other embodiments, the content boundary is the discovered contour c j Based on this, for example, given bounding box b i and the discovered contour c j It may be determined as an average or weighted average of the values. In one or more embodiments, block 914 determines the content boundaries (e.g., discovered contour c) j ) is added to or included in the content boundary (450) determined by the content boundary determination system (300). Block 916 returns the content boundary.

[0060] Using an ongoing example of this disclosure, Figure 10 shows a flowchart (900) or various steps of a corresponding system (330) applied to a set of bounding boxes A (520) and a set of contours A (830) previously determined based on display data A (510). Figure 10 shows a set of bounding boxes A (520) having a first bounding box (522), b1 and a second bounding box (524), b2. Thus, the set of bounding boxes A (520) is,

number

number

[0061] In Figure 10, the solid line extends between the first bounding box (522) and all the contours in contour set A (830). The solid line represents the determination of the correspondence score between the first bounding box (522), b1 and the contours in contour set A (830). Using the IoU function as the similarity function, the second contour (804), c2 is found to have the highest correspondence score with the first bounding box (522), b1, with a correspondence score of 0.90. In fact, the intersection overunion between the first bounding box (522), b1 and the remaining contours in contour set A (830) is 0.0. This represents block 908 in Figure 9, where i=1 for block 906. Since the contour (second contour (804), c2) with the highest correspondence score for the first bounding box (522), b1 has been found, the first condition of block 910 is satisfied. The second condition of block 910 is to compare the correspondence scores of the first bounding box (522), b1 and the second contour (804), c2 with a predetermined threshold, T. In this example, the predetermined threshold is set to 0.80. Furthermore, in the example of Figure 10, the correspondence score of the first bounding box (522), b1 and the second contour (804), c2 is 0.90. Therefore, in the example of Figure 10, the first and second conditions of block 910 in Figure 9 are satisfied by the first bounding box (522), b1 and the second contour (804), c2, and such the first bounding box (522), b1 and the second contour (804), c2 form a pair according to block 914 in Figure 9. Furthermore, the second contour (804), c2 is identified as a content boundary and is added to or included in a content boundary (450) which will be output by the content boundary determination system (300).

[0062] The dashed line extends between the second bounding box (524) and all the contours in contour set A (830). The dashed line represents the determination of the correspondence score between the second bounding box (524), b2 and the contours in contour set A (830). Using the IoU function as the similarity function, the fourth contour (808), c4 is found to have the highest correspondence score with the second bounding box (524), b2, with a correspondence score of 0.82. In fact, the intersection overunion of the second bounding box (524), b2 and the remaining contours in contour set A (830) is 0.0. This represents block 908 in Figure 9, and i=2 for block 906. Since we have found the contour with the highest correspondence score for the second bounding box (524), b2 (the fourth contour (808), c4), the first condition of block 910 is satisfied. The second condition of block 910 is to compare the corresponding scores of the second bounding box (524), b2 and the fourth contour (808), c4 with a predetermined threshold, T. In this example, the predetermined threshold is set to 0.80. Furthermore, in the example of Figure 10, the corresponding scores of the second bounding box (524), b2 and the fourth contour (808), c4 are 0.82. Therefore, in the example of Figure 10, the first and second conditions of block 910 in Figure 9 are satisfied by the second bounding box (524), b2 and the fourth contour (808), c4, and such a second bounding box (524), b2 and the fourth contour (808), c4 can be said to form a pair according to block 914 in Figure 9. Furthermore, the fourth contour (808), c4, is identified as a content boundary and is added to or included in a content boundary (450) which will be output by the content boundary determination system (300).

[0063] Continuing with the ongoing example of this disclosure, Figure 11 shows a content boundary (content boundary A(1150)) returned by the corresponding system (330) according to block 916 of Figure 9, and the content boundary determination system (300) is applied to display data A(510). As can be seen in Figure 11, content boundary A(1150) includes two content boundaries, namely a first content boundary (1102) and a second content boundary (1104). These content boundaries are determined based on their associated contours. In this example, the content boundaries are set for their respective contours. That is, the first content boundary (1102) is the second contour (804) of the set of contours A(830), paired by the corresponding system (330) with the first bounding box (522) of the set of bounding boxes A(520), b1, and c2. Furthermore, a first bounding box (522), b1 is detected in the first image (512) (or first content object) within display data A (510) so that the first content boundary (1102) is for the first image (512). The second content boundary (1104) is the fourth contour (808), c4 of the set of contours A (830) paired by the corresponding system (330) with the second bounding box (524), b2 of the set of contours A (520). Furthermore, a second bounding box (524), b2 is detected in the second image (514) (or second content object) within display data A (510) so that the second content boundary (1104) is for the second image (514).

[0064] In general terms, the content boundary determination system (300) includes an artificial intelligence detection model (310), an image processing system (320), and a correspondence system (330). The artificial intelligence detection model (310) and the image processing system (320) each independently process display data (i.e., what has been or will be rendered on the display) and return area representations (i.e., areas on the display) that are thought to correspond to content objects. Specifically, the artificial intelligence detection model (310) returns a set of bounding boxes, each bounding box in the set of bounding boxes relating to a content object detected by the artificial intelligence detection model (310), and the image processing system (320) returns a set of contours. The correspondence system (330) processes the set of bounding boxes and the set of contours to determine one or more content boundaries ("content boundaries"), which represent the actual areas or boundaries of the detected content objects.

[0065] Figure 12 shows a method according to one or more embodiments. The steps of the method in Figure 12 can be carried out using a content boundary determination system (300), a computer system, or a combination thereof, as described above. As shown in the figure, in step 1202, display data including a first content object is acquired. From one perspective, the display data is what is rendered or will be rendered on the display. The display can be part of a display device. Furthermore, the display may be adjustable to selectively change, for example, the resolution, bit depth, color mode, etc., of the display area.

[0066] In step 1204, the display data is processed by an artificial intelligence detection model to detect content objects within the display data. The artificial intelligence detection model returns a set of bounding boxes, each bounding box representing a portion of the display associated with the detected content objects. The set of bounding boxes determined using the artificial intelligence detection model includes a first bounding box associated with a first content object. In some embodiments, the artificial intelligence detection model is configured to detect content objects of a specified type, such as content objects having the content type of an image. Thus, in these embodiments, the set of bounding boxes includes only bounding boxes for content objects of the specified type (e.g., an image).

[0067] In step 1206, the display data is processed using an image processing system to determine a set of contours, which includes a first contour. According to one or more embodiments, the image processing system applies a series of computer vision filters or functions to the display data, and the final or final function extracts contours from the processed data.

[0068] In step 1208, it is determined that the first bounding box and the first contour correspond to each other. In one or more embodiments, the correspondence between the first bounding box and the first contour is determined using a correspondence system. The correspondence system calculates a correspondence score between the first bounding box and the first contour and determines that the first bounding box and the first contour correspond to each other in response to the correspondence score exceeding a threshold. The correspondence score can be the output of a similarity function, such as an intersection over union (IoU) function.

[0069] In step 1208, it is determined that the first contour corresponds to the first bounding box, and in step 1210, the content boundary for the first content object is determined based on the first contour. In one or more embodiments, the first contour is determined to be the content boundary of the first content object; that is, the content boundary of the first content object is set to the first contour.

[0070] In step 1212, the display settings of the display device are determined based on the content boundaries of the first object. Furthermore, in one or more embodiments, the display settings of the display device are adjusted to the determined display settings.

[0071] Embodiments of this disclosure include an artificial intelligence detection model (310). Artificial intelligence, broadly defined, includes the extraction and modeling use of patterns and insights from data. Thus, in some implementations, an artificial intelligence detection model determines results such as bounding boxes based on perceived patterns in received data, and the patterns or their identification have been previously learned by the model using a set of training data. Various types of artificial intelligence models can be used as the artificial intelligence detection model (310) without departing from the essence of this disclosure.

[0072] One type of machine learning model is a neural network. Neural networks can be used as a subcomponent of larger machine learning models. A neural network can be represented as a graph composed of nodes and edges. Generally, the edges of a neural network are "directed," and borrowing from graph language, a neural network can be classified as a directed acyclic graph (DAG).

[0073] Nodes may be grouped to form layers. Edges can or cannot connect to any node, regardless of which layer the node is in. That is, edges can form sparse and residual connections between nodes (e.g., so-called "skip" connections). A layer and its adjacent layers are said to be fully or tightly connected if all nodes in a layer are connected to all nodes in an adjacent layer.

[0074] A neural network has at least two layers: an "input layer" and an "output layer." There may be zero or more hidden layers between the input and output layers. These hidden layers are generally called "hidden layers." Furthermore, a neural network having at least one hidden layer is sometimes described as a "deep" neural network or a "deep learning method." The output layer of a neural network can have two or more nodes. When the output layer of a neural network has two or more nodes, it is sometimes called a "multi-goal" or "multi-output" network.

[0075] Furthermore, each edge in a neural network is associated with a numerical value. The numerical value of an edge, or even the edge itself, is often called a "weight" or "parameter." Thus, a neural network can be said to contain, or be parameterized by, a set of weights or parameters. A neural network is "trained" by assigning numerical values ​​to each trainable edge of the neural network through evaluation of a set of data commonly called training data (described later). Here, the distinction of "trainable edge" is introduced when a trainable edge is an edge whose numerical value can be adjusted during the training routine. Generally, non-trainable edges have numerical values, but their values ​​are determined using a process different from the training process, such as direct assignment by the user.

[0076] Similarly, nodes carry, pass through, or temporarily store numerical values ​​and are further associated with activation functions. Activation functions are not limited to any particular class of function, but traditionally, they apply a function to the inner product of an array of values ​​of nodes connected to or directed to the node to which the activation function should be applied ("activation node") ("input nodes"), and an array of weights or parameters of the edges connecting the input nodes to the activation node. An input node, when viewed as a graph, is a node with a directed arrow pointing to the activation node to which the numerical values ​​of the activation node are being calculated. Some commonly used activation functions include the linear function f(x)=x and the sigmoid function.

number

[0077] When a neural network receives an input, the input propagates through the network according to the activation functions of the neural network's nodes and the edge values ​​of the neural network. Therefore, the numerical values ​​of the nodes can change with each input received. In some cases, a fixed numerical value, such as 1, is assigned to a node that is unaffected by the input. Nodes with fixed numerical values ​​(invariant with respect to the input) are often called "biases" or "bias nodes."

[0078] In some implementations, the neural network may include special layers such as normalization layers, dropout layers, and concatenation layers. For brevity, such layers are not described herein, but those skilled in the art will recognize that including and using such layers with the neural network is not beyond the scope of this disclosure.

[0079] As described above, the process of training a neural network consists of at least assigning values ​​to the edges of the neural network. Training begins with a neural network that has edge values ​​initially provided through some initialization mechanism or procedure. Edge values ​​may be assigned randomly, according to a given distribution, manually, or by some other assignment procedure. With the initial edge values, it can be said that the neural network functions as a function that receives, inputs, and produces outputs. Thus, one or more inputs can propagate through the neural network to produce one or more associated outputs. During training, a training set or training data is provided to the neural network. The training set consists of inputs and associated goals, the goals representing the desired outputs, often observed values, or the "ground truth" associated with the observed inputs. During training, the neural network processes the inputs to produce outputs, which are compared to the associated goals. The comparison of the generated neural network output with the goals is performed using a "loss function" such as the mean squared error function, mean absolute error function, or logarithmic loss function (or binary cross-entropy function). Generally, a loss function provides a numerical assessment of the similarity between the neural network output and a given goal. In some implementations, the loss function may consist of multiple loss functions applied to different parts of the output-goal comparison. The loss function may also be configured to impose additional constraints on the values ​​assumed by the edges. For example, the loss function may include a physically-based regularization or penalty term that affects or otherwise restricts the edge values. Overall, the goal of the training process is to modify the edge values ​​so that the output of the neural network, when processing a given input, resembles the goal associated with that input. In other words, the intention of training is to promote similarity between the neural network output and the associated goal across the dataset provided for training (e.g., training data). The changes in edge values ​​are driven by the loss function, typically through a process called "backpropagation".

[0080] Backpropagation involves calculating the gradient of the loss function with respect to the values ​​of trainable edges. The gradient represents the change in edge values ​​and, when applied to an edge, brings about the greatest change in the loss function with respect to the training data provided when calculating the gradient. Edge values ​​are typically updated by "steps" in the direction of the gradient. The step size, often called the "learning rate," does not need to remain fixed during the training process. Furthermore, step size updates for edge values ​​may be notified by previously seen edge values ​​or previously calculated gradients.

[0081] Updates to edge values ​​in a neural network are applied iteratively. In other words, the training process consists of repeatedly calculating the gradient of the loss function with respect to the edge values ​​and updating the edge values ​​in steps induced by the gradient. This process continues until a termination criterion is reached. For example, the termination criterion may consist of one or more of the following: reaching a fixed number of edge updates, usually known as the iteration counter; noting that there is no significant change in the loss function between iterations (or that the change in edge values ​​between updates is below a given threshold); and reaching a specified performance metric, such as one evaluated on the training data or a separate holdout dataset. When the termination criterion is met and it is no longer intended that the edge values ​​will be updated, the neural network is said to be "trained". The loss function can be constructed such that if the loss function increases, the similarity between the output and the target increases, so the training process can be considered to be maximizing the loss function. Similarly, if the loss function decreases, the loss function can be constructed such that the similarity between the output and the target increases, so the training process can be considered to be minimizing the loss function. The tasks of maximizing and minimizing can be made equivalent through techniques such as negate.

[0082] The architecture of a machine learning model defines the "structure" of the machine learning model. For example, in the case of a neural network, the structure is specified by the number of hidden layers in the network, the type of activation function used, and the number of outputs, and, in particular, the use and location of specialized layers (e.g., batch normalization layers). The architecture of a machine learning model is specified by a set of "hyperparameters." For example, in the case of a neural network, the number of hidden layers and the number of nodes in each layer are the hyperparameters of the neural network.

[0083] Another type of machine learning model is the convolutional neural network (CNN). Like neural networks, a CNN can be thought of, or described, as consisting of a set of nodes connected by edges. However, it is useful to view a CNN as a structural grouping of weights, where the term structural indicates that the weights within a group have relationships. CNNs are broadly applicable when the input data also has structural relationships, such as spatial relationships where one element of the input is always considered "left" of another element of the input. For example, display data consisting of pixels can have structural relationships because each pixel (element) has an directional relationship with respect to its neighboring pixels.

[0084] Structural grouping or grouping of weights is referred to herein as “filters.” In a CNN, filters can be thought of as “sliding” over or convolving with the input data to form intermediate outputs or intermediate representations of the input data that still retain structural relationships. Similar to neural networks, intermediate outputs are often further processed with activation functions. Many filters may be applied to the input data to form many intermediate representations. Additional filters may be formed to operate on the intermediate representations and create many more intermediate representations. This process may be repeated as defined by the user. When convolving with the input, filters may move by a stride such that some elements of the input (e.g., pixels) are skipped. Grouping of intermediate output representations can be pooled, for example, by considering only the maximum value of the group in subsequent calculations. Intermediate representations may be downsampled using strides and pooling. Similar to neural networks, additional operations such as normalization, concatenation, dropout, and residual connection may be applied to the intermediate representations.

[0085] Similar to neural networks, CNNs are trained after the initialization of filter weights, and if present, the edge values ​​of the neural network are included using a backpropagation process that follows the loss function.

[0086] In one or more embodiments, the artificial intelligence detection model (310) disclosed herein is a CNN or based on a CNN. The You Only Look Once (YOLO) object detection model is based on a CNN. Therefore, in one or more embodiments, the artificial intelligence detection model (310) is one version of the YOLO object detection model.

[0087] Embodiments of the present disclosure have one or more of the following advantages. Embodiments of the present disclosure can provide real-time, high-precision content boundaries for content objects that are rendered or will be rendered on a display. Accurate determination of content boundaries reduces artifacts and defects on the display. Furthermore, accurate determination of content boundaries enables or improves the adjustment of display settings for the display. For example, a display device including a display may adjust the display settings of the display based on content objects. Adjusting display settings based on content objects may be beneficial for one or more reasons, such as selectively improving the display resolution based on the location of content objects, reducing power consumption of the display (e.g., to maintain the battery life of the display device), selectively changing the bit depth of pixels, and reducing latency (e.g., in the case of movies). Thus, embodiments of the present disclosure enable the adjustment of display settings based on content objects, more specifically, the type and location of each content object. Other adjustments to the display settings of a display device can be made based on content objects of display data that are rendered or will be rendered, without departing from the scope of the present disclosure. For example, the area of ​​the display associated with a detected content object (e.g., an image) can be enhanced using super-resolution techniques or methods.

[0088] Although only a few exemplary embodiments have been described in detail above, those skilled in the art will readily understand that many modifications are possible in the exemplary embodiments without substantially departing from the present invention. Accordingly, all such modifications are intended to be within the scope of this disclosure as defined in the appended claims.

Claims

1. A method for determining content boundaries, A step of obtaining display data including a first content object, A step of determining a set of bounding boxes based on the display data using an artificial intelligence detection model configured to detect content objects based on the display data, wherein the set of bounding boxes includes a first bounding box associated with the first content object. A step of determining a set of contours based on the display data using an image processing system that includes one or more computer vision filters or functions for converting the display data, wherein the set of contours includes a first contour, A step of calculating the correspondence score between the first bounding box and the first contour, A step of determining that the first bounding box and the first contour correspond to each other based on the correspondence score, A step of determining the content boundary of the first content object based on the first contour, The steps include adjusting the display settings of a display device configured to display the display data based on the content boundary of the first content object, and Methods that include...

2. The method according to claim 1, further comprising the step of displaying the display data using the display device.

3. The aforementioned display data further includes a second content object, The set of bounding boxes further includes a second bounding box associated with the second content object, The set of contours further includes a second contour, The method described above is A step of calculating another correspondence score for the second bounding box and the second contour, The steps include determining that the second bounding box and the second contour correspond to each other based on the aforementioned correspondence score, The steps of determining the content boundary of the second content object based on the second contour and The method according to claim 1, further comprising:

4. The first content object has a first content type, The artificial intelligence detection model is configured to detect content in the display data having the first content type. The method according to claim 1.

5. The method according to claim 4, wherein the first content type is an image.

6. The method according to claim 1, wherein the artificial intelligence detection model includes a convolutional neural network.

7. The method according to claim 1, wherein the one or more computer vision filters or functions are ordered to form an ordered set.

8. The method according to claim 7, wherein the step of determining the set of contours includes the step of applying the ordered set to the display data.

9. The ordered set, Grayscale converter, Edge filter and, Morphological filters and, Contour extractor and The method according to claim 7, comprising:

10. The step of determining that the first bounding box and the first contour correspond to each other is: The steps include determining that the correspondence score of the first bounding box and the first contour is greater than any other correspondence score for any other contour in the set of the first bounding box and the contour, The step of determining that the aforementioned correspondence score exceeds a threshold, The method according to claim 1, including the method described in claim 1.

11. The method according to claim 1, wherein the corresponding score includes the intersection overunion of the first bounding box and the first contour.

12. A computer system, An artificial intelligence detection model configured to receive display data, detect content objects within the display data, and output a set of bounding boxes, An image processing system configured to receive the aforementioned display data and output a set of contours, comprising one or more computer vision filters or functions for converting the aforementioned display data, A correspondence system configured to determine one or more correspondences between the set of bounding boxes and the set of contours. Equipped with, The aforementioned computer system The display data including the first content object is obtained, Using the artificial intelligence detection model, the set of bounding boxes is determined based on the display data, and the set of bounding boxes includes a first bounding box associated with the first content object. Using the image processing system, determine the set of contours based on the display data, and the set of contours includes the first contour. The correspondence score between the first bounding box and the first contour is calculated, Using the aforementioned correspondence system, it is determined that the first bounding box and the first contour correspond to each other based on the correspondence score. Based on the first contour, the content boundary of the first content object is determined. Adjust the display settings of a display device configured to display the display data based on the content boundary of the first content object. Computer system.

13. The computer system displays the display data using the display device. The computer system according to claim 12, further configured as follows.

14. The first content object has a first content type, The artificial intelligence detection model is further configured to detect content in the display data having the first content type. The computer system according to claim 12.

15. The computer system according to claim 14, wherein the first content type is an image.

16. The computer system according to claim 12, wherein the one or more computer vision filters or functions are ordered to form an ordered set.

17. The computer system according to claim 16, wherein determining the set of contours includes applying the ordered set to the display data.

18. The ordered set, Grayscale converter, Edge filter and, Morphological filters and, Contour extractor and The computer system according to claim 16, comprising:

19. Using the aforementioned correspondence system, it is determined that the first bounding box and the first contour correspond to each other. Determining that the correspondence score of the first bounding box and the first contour is greater than any other correspondence score for any other contour in the set of the first bounding box and the contour, The aforementioned correspondence score is determined to exceed a threshold. The computer system according to claim 12, including the above.

Images