Color Atmosphere Processing Methods and Systems
By parsing video files to extract color distribution and generating atmosphere description files, the problem of integrating ambient lighting devices in video applications is solved, realizing the non-continuous extraction and expression of color atmosphere and reducing dependence on the system.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HANGZHOU ARCVIDEO TECHNOLOGY CO LTD
- Filing Date
- 2023-02-10
- Publication Date
- 2026-05-26
AI Technical Summary
In existing video applications, integrating ambient lighting devices is difficult and cannot handle DRM-protected video content, resulting in the inability to acquire real-time image colors to activate the ambient lighting devices.
By parsing video files, the color distribution of each frame is extracted to form a color atmosphere description file, which is then mapped according to the spatial area configuration during playback to drive the atmosphere generation device to express the atmosphere.
It achieves decoupling of color atmosphere extraction and expression, and can express the atmosphere through pre-extracted atmosphere features even when the terminal device cannot decode in real time, reducing the reliance on content, application and system.
Smart Images
Figure CN116506677B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of video processing technology, specifically relating to a color atmosphere processing method and system. Background Technology
[0002] With the development of virtual technology and artificial intelligence, people can enjoy videos more attentively in different spaces. Furthermore, there's a desire for an immersive, accompanying atmosphere while watching video content. Currently, some products on the market can create atmosphere based on the color distribution of the video content, combined with ambient lighting in the space. A common approach is to collaborate with video application vendors, deeply integrating color analysis algorithms into the application. However, integrating with each of the numerous video application vendors presents significant challenges. Furthermore, directly capturing screen content through the system is not feasible for content protected by Digital Rights Management (DRM). Summary of the Invention
[0003] In view of the above-mentioned problems, the present invention provides a color atmosphere processing method and system, which is used to extract the color distribution of each frame of video content and form a color atmosphere description file corresponding to the video content. When playing the corresponding video content, the color distribution data corresponding to the video is mapped according to the spatial atmosphere region definition, and the mapping result is used as the color atmosphere to stimulate the atmosphere generating device in the space to express the color atmosphere.
[0004] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:
[0005] In a first aspect, the present invention provides a color atmosphere processing method, comprising the following steps:
[0006] Given a video file, the video file is parsed, the video content is decoded, and the decoded image frames are obtained; the image frames are segmented into regions, and the theme colors of each segmented region are extracted to form a region theme color distribution table for the image frames; the timestamp corresponding to each extracted image frame and the region theme color distribution table form a color atmosphere frame descriptor, and the combination of all color atmosphere frame descriptors forms a color atmosphere description file.
[0007] The color atmosphere description file is managed, and the color atmosphere description file and video file identification information are stored.
[0008] After obtaining the color atmosphere description file corresponding to the video content, extract the color distribution table of each image frame in the color atmosphere description file, and perform a theme color remapping operation on the spatial region according to the atmosphere configuration of the spatial region to generate an atmosphere theme color distribution table suitable for the spatial region.
[0009] Obtain the atmospheric theme color distribution table corresponding to the current playback time of the video, and drive the atmospheric generation device in the relevant area of the atmospheric space to express the atmosphere.
[0010] In one possible implementation, the process of remapping the theme color of a spatial region to generate an atmosphere theme color distribution table suitable for that spatial region involves: acquiring and parsing all color atmosphere frame descriptors in the color atmosphere description file to form an image with the same dimensions as the video image. Each sub-region in the image is a region in the descriptor, and the color in the region is filled with the theme color corresponding to that region in the color atmosphere frame descriptor. Then, the spatial atmosphere distribution map is stretched non-proportionally according to the image dimensions of the video. The sub-rectangles in the newly generated distribution map are then extracted again according to the theme color extraction algorithm defined in the descriptor to form the atmosphere theme distribution table for that space.
[0011] In one possible implementation, the ambient theme color distribution table corresponding to the frame includes color information of all segmented regions in the frame, and the color information of each region includes the start and end coordinates of the region and the theme color value.
[0012] In one possible implementation, the decoded image frame is obtained by sampling each frame of the video file or at intervals of several frames.
[0013] Secondly, the present invention provides a color atmosphere processing system, comprising a color atmosphere acquisition module, a color atmosphere management module, a color atmosphere space mapping module, and a color atmosphere expression module, wherein...
[0014] The color atmosphere acquisition module is used to obtain video content, decode the video content, obtain decoded image frames, segment the decoded image frames into regions, extract the theme color of each segmented region, and form a region theme color distribution table of the image frame. The timestamp corresponding to each extracted image frame and the region theme color distribution table form a color atmosphere frame descriptor. The combination of all color atmosphere frame descriptors forms a color atmosphere description file.
[0015] The color atmosphere management module is used to manage the color atmosphere description file and save the color atmosphere description file and video file identification information in relevant storage.
[0016] The color atmosphere space mapping module is used to obtain the atmosphere description file corresponding to a certain video content from the color atmosphere management module, extract the regional main color distribution table of each frame in the description file, and then perform the theme color remapping operation of the spatial region according to the spatial region atmosphere configuration of the atmosphere space to generate an atmosphere theme color distribution table suitable for the spatial region.
[0017] The color atmosphere expression module is used to drive the atmosphere generating devices in the relevant areas of the atmosphere space to express the atmosphere according to the atmosphere theme color distribution table output by the color atmosphere space mapping module.
[0018] In one possible implementation, the process of remapping the theme color of a spatial region to generate an atmosphere theme color distribution table suitable for that spatial region involves: acquiring and parsing all color atmosphere frame descriptors in the color atmosphere description file to form an image with the same dimensions as the video image. Each sub-region in the image is a region in the descriptor, and the color in the region is filled with the theme color corresponding to that region in the color atmosphere frame descriptor. Then, the spatial atmosphere distribution map is stretched non-proportionally according to the image dimensions of the video. The sub-rectangles in the newly generated distribution map are then extracted again according to the theme color extraction algorithm defined in the descriptor to form the atmosphere theme distribution table for that space.
[0019] In one possible implementation, the theme color distribution table corresponding to the frame contains color information of all segmented regions in the frame, and the color information of each region includes the start and end coordinates of the region and the theme color value.
[0020] In one possible implementation, the color atmosphere space mapping module is deployed on the terminal device. After the terminal device requests an atmosphere description file from the color atmosphere management module through the video file identification information, it generates a spatial atmosphere distribution map through the color atmosphere space mapping module, which is then used by the atmosphere expression module.
[0021] In one possible implementation, the color atmosphere management module and the color atmosphere space mapping module are deployed on the central device. When the terminal device requests an atmosphere description file through video file identification information, it also uploads a spatial atmosphere distribution map to the central device. After obtaining the atmosphere description file from the color atmosphere management module, the central device calls the color atmosphere space mapping module to form a new atmosphere description file and returns it to the terminal device. The terminal device then directly parses the file for use by the atmosphere expression module.
[0022] In one possible implementation, the decoded image frame is obtained by sampling each frame of the video file or at intervals of several frames.
[0023] The present invention has the following beneficial effects:
[0024] (1) Color atmosphere extraction and expression can be decoupled, that is, atmosphere extraction and expression are carried out in two non-continuous stages.
[0025] (2) Even when the terminal cannot obtain the decoded image in real time, it can still obtain the pre-extracted atmospheric features through out-of-band means to express the atmosphere, thereby reducing the reliance on content, application and system. Attached Figure Description
[0026] Figure 1 This is a flowchart illustrating the steps of a color atmosphere processing method according to an embodiment of the present invention;
[0027] Figure 2 This is a schematic diagram of a color atmosphere processing system according to an embodiment of the present invention. Detailed Implementation
[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0029] See Figure 1 The diagram shows a flowchart of a color atmosphere processing method according to an embodiment of the present invention, including the following steps:
[0030] S10, Given a video file, parse the video file, decode the video content, and obtain the decoded image frames; segment the image frames into regions, extract the theme color of each segmented region, and form a region theme color distribution table for the image frame; the timestamp corresponding to each extracted image frame and the region theme color distribution table form a color atmosphere frame descriptor, and the combination of all color atmosphere frame descriptors forms a color atmosphere description file; the theme color distribution table corresponding to the frame contains the color information of all segmented regions in the frame, and the color information of each region includes the start and end coordinates of the region and the theme color value.
[0031] S20 manages the color atmosphere description file and stores the color atmosphere description file and video file identification information; the video file identification information can be an independent identifier, such as the video content ID, or a combined identifier, such as a combination of the video content name and the video duration.
[0032] S30, after obtaining the color atmosphere description file corresponding to the video content, extract the color distribution table of each image frame in the color atmosphere description file, and perform a theme color remapping operation on the spatial region according to the atmosphere configuration of the spatial region to generate an atmosphere theme color distribution table suitable for the spatial region.
[0033] S40, obtain the ambient theme color distribution table corresponding to the current video playback time, and drive the ambient generation device in the relevant area of the ambient space to express the atmosphere, such as driving the ambient light device in the corresponding area to emit the color light of the theme color corresponding to that area.
[0034] In a specific application example of this invention, step S10, obtaining the decoded image frame, can be done by sampling each frame in the video file or by sampling at intervals of several frames. The interval between these intervals can be fixed or determined by scene detection and the corresponding frame interval for each scene.
[0035] In a specific application example of this invention, the region segmentation of the image frame in step S10 can be performed using a uniform region segmentation template, i.e., an MxN template, which divides the image horizontally into M equal parts and vertically into N equal parts. Alternatively, other image segmentation algorithms can be used to analyze the image and form non-uniform regions. Furthermore, black borders can be identified in the image frame first, and the regions after removing the black borders can then be segmented.
[0036] In a specific application example of this invention, the theme color extraction calculation method for extracting theme colors from each segmented region in step S10 can be achieved by calculating the average value of the RGB components of all pixels in the region, finding the color value with the highest proportion in the region, K-Means clustering algorithm, or color quantization algorithms such as median segmentation or octree algorithm.
[0037] In one embodiment of the present invention, step S30, which involves remapping the theme color of a spatial region to generate an atmosphere theme color distribution table suitable for that spatial region, specifically includes: acquiring and parsing all color atmosphere frame descriptors in the color atmosphere description file to form an image with the same dimensions as the video image. Each sub-region in the image is a region in the descriptor, and the color in the region is filled with the theme color corresponding to that region in the color atmosphere frame descriptor. Then, the spatial atmosphere distribution map is stretched non-uniformly according to the image dimensions of the video. The sub-rectangles in the newly generated distribution map are then extracted again according to the theme color extraction algorithm defined in the descriptor to form the atmosphere theme distribution table for that space.
[0038] The spatial atmosphere distribution map is a rectangle, which can further contain sub-rectangles. Each sub-rectangle corresponds to the area affected by an atmosphere generating device within that space. In terms of data, the spatial atmosphere distribution map includes the dimensions of the rectangle and the coordinates and dimensions of the sub-rectangles within it. During the video playback preparation stage, the color atmosphere description file corresponding to the video is retrieved from the color atmosphere management module using the video content ID or video name and duration. This color atmosphere description file is then used for atmosphere space mapping.
[0039] For the method implementation examples, see [link to relevant documentation]. Figure 2 This invention also provides a color atmosphere processing system 100, including a color atmosphere acquisition module 101, a color atmosphere management module 102, a color atmosphere spatial mapping module 103, and a color atmosphere expression module 104. The color atmosphere acquisition module 101 acquires video content, decodes the video content to obtain decoded image frames, segments the decoded image frames into regions, extracts the theme color of each segmented region, forming a region theme color distribution table for the image frame. The timestamp corresponding to each extracted image frame and the region theme color distribution table together form a color atmosphere frame descriptor. All color atmosphere frame descriptors are combined to form a color atmosphere description file. The theme color distribution table corresponding to a frame contains color information for all segmented regions within that frame. The color information for each region includes the start and end coordinates of that region and its theme color value. The color atmosphere management module 102 manages the color atmosphere description file, storing the color atmosphere description file and video file identification information in relevant storage. The video file identification information can be an independent identifier, such as a video content ID, or a combined identifier, such as a combination of video content name and video duration. The color atmosphere space mapping module 103 is used to obtain the atmosphere description file corresponding to a certain video content from the color atmosphere management module, extract the regional main color distribution table of each frame in the description file, and then perform the theme color remapping operation of the spatial region according to the spatial region atmosphere configuration of the atmosphere space to generate an atmosphere theme color distribution table suitable for the spatial region; the color atmosphere expression module 104 is used to drive the atmosphere generating device in the relevant region of the atmosphere space to express the atmosphere according to the atmosphere theme color distribution table output by the color atmosphere space mapping module, such as driving the corresponding ambient light device in the corresponding region to emit the color light of the theme color corresponding to the region.
[0040] In a specific application example of this invention, the color atmosphere acquisition module 101 can sample and extract the decoded image frames by sampling each frame of the video file or by sampling at intervals of several frames. The interval between these intervals can be fixed or determined by scene detection based on the frame interval corresponding to the scene.
[0041] In a specific application example of this invention, the color atmosphere acquisition module 101 performs region segmentation on the image frame. This can be done using a uniform region segmentation template, i.e., an MxN template, which divides the image horizontally into M equal parts and vertically into N equal parts. Alternatively, other image segmentation algorithms can be used to analyze the image and create non-uniform regions. Furthermore, black borders can be identified first in the image frame, and the regions after removing the black borders can then be segmented.
[0042] In a specific application example of the present invention, the theme color extraction calculation method in the color atmosphere acquisition module 101 for extracting theme colors from the segmented regions can be achieved by calculating the average value of the RGB components of all pixels in the region, finding the color value with the largest proportion in the region, K-Means clustering algorithm, or color quantization algorithms such as median segmentation or octree algorithm.
[0043] In one embodiment of the present invention, the color atmosphere space mapping module 103 performs a theme color remapping operation on a spatial region to generate an atmosphere theme color distribution table suitable for the spatial region. Specifically, this includes: acquiring and parsing all color atmosphere frame descriptors in the color atmosphere description file to form an image with the same size as the video image. Each sub-region in the image is a region in the descriptor. The color in the region is filled with the theme color corresponding to that region in the color atmosphere frame descriptor. Then, the spatial atmosphere distribution map is stretched non-uniformly according to the image size of the video. The sub-rectangles in the newly generated distribution map are extracted again according to the theme color extraction algorithm defined in the descriptor to form the atmosphere theme distribution table of the space.
[0044] The spatial atmosphere distribution map is a rectangle, which can further contain sub-rectangles. Each sub-rectangle corresponds to the area affected by an atmosphere generating device within that space. During the video playback preparation stage, the color atmosphere description file corresponding to the video is retrieved from the color atmosphere management module using the video content ID or video name and duration. This color atmosphere description file is then used for atmosphere space mapping.
[0045] In a specific application example, the color atmosphere space mapping module 103 can be deployed on the terminal-side device. After requesting an atmosphere description file from the color atmosphere management module using video file identification information, the terminal-side device generates a spatial atmosphere distribution map through the color atmosphere space mapping module for use by the atmosphere expression module. This method reduces the computational load on the central side and saves the deployment cost of the central-side device.
[0046] In another specific application example, the color atmosphere management module and the color atmosphere space mapping module can be deployed on a central-side device, which can be understood as a device with stronger storage and computing performance than the terminal-side device. When the terminal-side device requests an atmosphere description file using video file identification information, it simultaneously uploads a spatial atmosphere distribution map to the central-side device. After retrieving the atmosphere description file from the color atmosphere management module, the central-side device calls the color atmosphere space mapping module to generate a new atmosphere description file, which is then returned to the terminal-side device. The terminal-side device directly parses this file for use by the color atmosphere expression module. This approach reduces the computational burden on the terminal-side device while also protecting the original atmosphere description file, returning only the content suitable for that space.
[0047] As seen from the deployment examples above, the color atmosphere management module can be set up on the central device, the color atmosphere expression module can be set up on the terminal device (such as an in-vehicle terminal or a mobile terminal), and the color atmosphere space mapping module can be set up on either the central device or the terminal device. The color atmosphere acquisition module can be set up on either the central device or the terminal device, and even on the terminal device, it will be a different terminal than the color atmosphere expression module. The color atmosphere expression module is basically set up on the in-vehicle terminal, while the color atmosphere acquisition module may be completed using a PC.
[0048] It should be understood that the exemplary embodiments described herein are illustrative and not restrictive. Although one or more embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art will understand that various changes in form and detail may be made without departing from the spirit and scope of the invention as defined by the appended claims.
Claims
1. A method for processing color atmosphere, characterized in that, Includes the following steps: Given a video file, parse the video file, decode the video content, and obtain the decoded image frames; The decoded image frames are obtained by sampling and extracting several frames from the video file at intervals. The interval between these frames is the frame interval corresponding to the scene after scene detection. The image frame is segmented into regions, and the theme color of each segmented region is extracted to form a region theme color distribution table of the image frame. The timestamp corresponding to each extracted image frame and the region theme color distribution table form a color atmosphere frame descriptor. The combination of all color atmosphere frame descriptors forms a color atmosphere description file. Before performing region segmentation on the image frame, black borders are first identified in the image frame, and the region after removing the black borders is then segmented. The color atmosphere description file is managed, and the color atmosphere description file and video file identification information are stored. After obtaining the color atmosphere description file corresponding to the video content, extract the color distribution table of each image frame in the color atmosphere description file, and perform a theme color remapping operation on the spatial region according to the atmosphere configuration of the spatial region to generate an atmosphere theme color distribution table suitable for the spatial region. Obtain the atmospheric theme color distribution table corresponding to the current video playback time, and drive the atmospheric generation devices in the relevant areas of the atmospheric space to express the atmosphere. The process of remapping the theme color of a spatial region to generate an atmosphere theme color distribution table suitable for that spatial region involves: acquiring and parsing all color atmosphere frame descriptors in the color atmosphere description file to form an image with the same dimensions as the video image. Each sub-region in the image is a region in the descriptor, and the color in the region is filled with the theme color corresponding to that region in the color atmosphere frame descriptor. Then, the spatial atmosphere distribution map is stretched non-uniformly according to the image size of the video. The sub-rectangles in the newly generated distribution map are then extracted again according to the theme color extraction algorithm defined in the descriptor to form the atmosphere theme color distribution table for that space. The atmosphere theme color distribution table corresponding to the frame includes the color information of all segmented regions in that frame. The color information of each region includes the start and end coordinates of that region and the theme color value.
2. A color atmosphere processing system, characterized in that, It includes a color atmosphere acquisition module, a color atmosphere management module, a color atmosphere space mapping module, and a color atmosphere expression module. The color atmosphere acquisition module is used to obtain video content, decode the video content, and obtain the decoded image frame. The decoded image frame is obtained by sampling and extracting the video file at intervals of several frames. The interval of several frames is the frame interval corresponding to the scene after scene detection. The decoded image frame is segmented into regions, and the theme color of each segmented region is extracted to form a region theme color distribution table for the image frame. The timestamp corresponding to each extracted image frame and the region theme color distribution table form a color atmosphere frame descriptor. The combination of all color atmosphere frame descriptors forms a color atmosphere description file. Before segmenting the image frame, black borders are identified, and the regions after removing the black borders are then segmented. The color atmosphere management module is used to manage the color atmosphere description file and save the color atmosphere description file and video file identification information in relevant storage. The color atmosphere space mapping module is used to obtain the atmosphere description file corresponding to a certain video content from the color atmosphere management module, extract the regional main color distribution table of each frame in the description file, and then perform the theme color remapping operation of the spatial region according to the spatial region atmosphere configuration of the atmosphere space to generate an atmosphere theme color distribution table suitable for the spatial region. The color atmosphere expression module is used to drive the atmosphere generating devices in the relevant areas of the atmosphere space to express the atmosphere according to the atmosphere theme color distribution table output by the color atmosphere space mapping module. The process of remapping the theme color of a spatial region to generate an atmosphere theme color distribution table suitable for that spatial region is as follows: obtain and parse all color atmosphere frame descriptors in the color atmosphere description file to form an image with the same size as the video image. Each sub-region in the image is a region in the descriptor. The color in the region is filled with the theme color corresponding to that region in the color atmosphere frame descriptor. Then, the spatial atmosphere distribution map is stretched non-uniformly according to the image size of the video. The sub-rectangles in the newly generated distribution map are extracted again according to the theme color extraction algorithm defined in the descriptor to form the atmosphere theme color distribution table of the space. The theme color distribution table corresponding to the frame contains the color information of all segmented regions in that frame. The color information of each region includes the start and end coordinates of that region and the theme color value. The color atmosphere space mapping module is deployed on the terminal-side device. After requesting an atmosphere description file from the color atmosphere management module using video file identification information, the terminal-side device generates a spatial atmosphere distribution map through the color atmosphere space mapping module, which is then used by the atmosphere expression module. Alternatively, the color atmosphere management module and the color atmosphere space mapping module are deployed on the central-side device. When the terminal-side device requests an atmosphere description file using video file identification information, it also uploads the spatial atmosphere distribution map to the central-side device. After obtaining the atmosphere description file from the color atmosphere management module, the central-side device calls the color atmosphere space mapping module to generate a new atmosphere description file, which is then returned to the terminal-side device. The terminal-side device directly parses the file and uses it for the atmosphere expression module.