Ambient light control method, device, and computer program product

Abstract

The present application relates to an ambient light control method, a device, and a computer program product. The method comprises: acquiring an image frame in a video stream generated by a camera device capturing a display, and determining a display area, which falls within an imaging range of the display, in the image frame and color-picking point information of color-picking points set around the display area; partitioning the display area on the basis of the color-picking point information to obtain a plurality of subdivided areas, and using the plurality of subdivided areas circumferentially arranged around the display area as light effect color-picking areas; performing color sampling on each light effect color-picking area to determine a target color corresponding to each light effect color-picking area; and constructing a light effect control signal on the basis of the target color corresponding to each light effect color-picking area, and outputting the light effect control signal to an ambient light device, so as to control the ambient light device to emit light according to the light effect control signal.

Description

Ambient lighting control methods and equipment, computer program products Technical Field

[0001] This application relates to the field of lighting control technology, and in particular to an ambient light control method and device, and a computer program product. Background Technology

[0002] With the rapid development of artificial intelligence (AI) technology, especially the significant advancements in deep learning algorithms and neural processing unit (NPU) chip technology, AI applications have gradually expanded into the consumer electronics market. In this field, fisheye image correction technology has become an important application. Fisheye cameras are widely used in panoramic monitoring, vehicle surround view systems, and many other areas due to their wide field of view. However, due to the lens characteristics of fisheye cameras, the captured images are often accompanied by significant distortion, which limits their effectiveness in certain applications. Therefore, developing effective fisheye image correction methods to improve image quality and user experience has become a key requirement for technological development. Summary of the Invention

[0003] The primary objective of this application is to solve at least one of the above-mentioned problems by providing an ambient lighting control method and device, and a computer program product.

[0004] One of the purposes of this application is to provide an ambient lighting control method, which includes the following steps:

[0005] The system acquires image frames from the video stream captured by the camera on the display, determines the display area within the image frame that belongs to the display's imaging range, and identifies the color sampling point information of each color sampling point set around the display area. Based on the color sampling point information, the display area is divided into multiple subdivided areas, and these subdivided areas arranged circumferentially along the display area are used as lighting effect color sampling areas. Color sampling is performed on each lighting effect color sampling area to determine the target color corresponding to each lighting effect color sampling area. A lighting effect control signal is constructed based on the target color corresponding to each lighting effect color sampling area, and the lighting effect control signal is output to the ambient light device to control the ambient light device to emit light according to the lighting effect control signal.

[0006] On the other hand, to suit another purpose of this application, a computer device is provided, which includes a central processing unit and a memory, the central processing unit being used to invoke and run a computer program stored in the memory to perform the steps of the above-described method.

[0007] In another aspect, to suit another purpose of this application, a computer program product is provided, comprising a computer program / instructions that, when executed by a processor, implement the steps of the above-described method. Attached Figure Description

[0008] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0009] Figure 1 is a schematic diagram of the circuit principle of an ambient light control device according to a typical embodiment of this application;

[0010] Figure 2 is a flowchart illustrating a typical embodiment of the ambient lighting control method of this application;

[0011] Figure 3 is a schematic diagram of the position distribution information of the eight predetermined points in the anchor frame region, which is obtained by taking an image frame of the example of this application using a single fisheye camera.

[0012] Figure 4 is a schematic diagram of the position distribution information of the five predetermined points in the anchor frame region, obtained by the left fisheye camera of the dual fisheye camera camera device.

[0013] Figure 5 is an example image frame of this application, which was captured by the right fisheye camera of a dual fisheye camera camera device, and is a schematic diagram of the corresponding position distribution information of the five predetermined points in the anchor frame area.

[0014] Figure 6 is a flowchart illustrating another embodiment of the ambient lighting control method of this application;

[0015] Figure 7 is a schematic diagram of the subdivision of a fisheye image of a display according to an example of this application;

[0016] Figure 8 is a schematic diagram of the location distribution information of each predetermined point in the anchor frame area of ​​this application, showing the corresponding predetermined point area.

[0017] Figure 9 is a schematic diagram of the structure of a computer device used in this application. Detailed Implementation

[0018] In a typical embodiment of this application, the ambient lighting control method is implemented based on an ambient lighting control device 10. Referring to FIG1, the ambient lighting control device 10 includes a camera device 11, a control unit 12, and an ambient lighting device 13.

[0019] The camera device 11, control unit 12, and ambient lighting device 13 can be connected via a wired connection or via a network. The network is typically the Internet, but can be any network, including but not limited to any combination of local area networks (LANs), metropolitan area networks (MANs), wide area networks (WANs), mobile networks, wired or wireless networks, private networks, or virtual private networks. In some embodiments, the camera device 11, control unit 12, and ambient lighting device 13 can also communicate via specific communication protocols, including but not limited to BLE, WLAN, Bluetooth, ZigBee, or Wi-Fi protocols. The network may also include network entities such as routers and gateways, which are not shown in the figure.

[0020] The control unit 12 is connected to the camera device 11, enabling the control unit 12 to read the video stream captured by the camera device, or for the camera device to send the captured video stream to the control unit. The control unit is also connected to the ambient lighting device 13, enabling the control unit 12 to control the lighting effects of the ambient lighting device.

[0021] The camera device 11 can be a single fisheye camera, which captures images using a short focal length and a 180-degree field of view. The control unit 12 can be a SOC, CPU, MPU, NPU, etc., and can be configured according to actual usage needs; this application does not impose any restrictions on this.

[0022] In one embodiment, the ambient light device 13 may be equipped with a WS2811 chip to receive and output the lighting effect control signal from the corresponding processing control unit, so as to emit light according to the lighting effect control signal.

[0023] In some embodiments, the ambient lighting device 13 can be a strip light, a bar light, a lamp post, or a plane light (containing multiple LEDs). In some embodiments, the ambient lighting device 13 can include one or more lighting units, which can be combined according to actual usage needs. In some embodiments, each lighting unit can contain one or more LEDs, and the control unit 12 can control the brightness, color, etc., of the LEDs in the lighting unit, and can also control the number of lights on, thereby achieving different lighting effects of the ambient lighting device 13. In some embodiments, the position and number of the ambient lighting device 13 can be set according to actual usage needs, and the length, height, and area of ​​the lighting effects can also be controlled according to the position of the ambient lighting device 13.

[0024] Please refer to Figure 2. An ambient light control method according to this application, in a typical embodiment, includes the following steps:

[0025] Step S2100: Obtain image frames from the video stream generated by the camera device capturing images of the display, determine the display area in the image frame that belongs to the imaging range of the display, and the predetermined point information of each predetermined point set around the display area.

[0026] For a single fisheye camera, eight predetermined points are set around the display area of ​​the complete image surrounding the monitor, corresponding to the four corner points, two midpoints of the long side, and two midpoints of the wide side of the monitor. For a dual fisheye camera, ten predetermined points are set around the display area of ​​the complete image surrounding the monitor. Specifically, five predetermined points are set around the display area of ​​the left half of the monitor, corresponding to the two corner points, one midpoint of the upper long side, one midpoint of the left wide side, and one point to the left of the midpoint of the lower long side at a predetermined distance. Five predetermined points are set around the display area of ​​the right half of the monitor, corresponding to the two corner points, one midpoint of the upper long side, one midpoint of the right wide side, and one point to the right of the midpoint of the lower long side at a predetermined distance. The predetermined distances can be flexibly set by those skilled in the art.

[0027] The camera in the ambient lighting control device captures images within its field of view, generating a corresponding video stream. The control unit connects to the camera, allowing it to read the video stream captured by the camera on the monitor. Therefore, when a single fisheye camera is used to capture the monitor, the control unit obtains the video stream and performs frame segmentation, dividing the video stream into multiple image frames to obtain frames arranged chronologically. When a dual fisheye camera is used, the control unit obtains the video stream, which includes sub-video streams simultaneously captured by the left and right fisheye cameras. Each sub-video stream is then segmented into frames to obtain frames arranged chronologically within each sub-video stream. The left fisheye camera captures the left half of the monitor, and the right fisheye camera captures the right half.

[0028] Frame-by-frame processing can be achieved using tools like ffmpeg (audio and video processing tools) and the OpenCV library. For example, you can use OpenCV's VideoCapture class to read video streams and use the read method to read image frames from the video stream one by one.

[0029] A pre-defined image segmentation model is used, taking an image frame as input, to perform object detection and image instance segmentation on the image frame. The image segmentation model extracts the corresponding semantic features of the image frame by performing convolution operations, including the image frame's color, texture, boundary, contour, and deep, complex, and abstract semantic information, generating corresponding multi-scale feature maps. The display area within the imaging range of the display in the image frame is used as the object to be detected. Multiple candidate box regions are predicted on the feature map. Overlapping candidate box regions are filtered by a non-maximum suppression algorithm to obtain the anchor box region of the object to be detected within the box. In addition, the image segmentation model classifies each pixel in the image frame and constructs a pixel-level mask for the display area to segment the display area. The Sobel algorithm is used to perform boundary detection on the display area to determine the coordinate information of each boundary point surrounding the boundary of the display area, forming the boundary part point information.

[0030] The image segmentation model is pre-trained to convergence, learning to determine the display area within the imaging range of the display in the input image, its boundary point information, and the anchor box region of the display area within the frame. Recommended image segmentation models are the YOLO series, such as YOLOv8. Other models equally suitable for object detection and instance segmentation, such as U-Net, DeepLab series, BiSeNet, etc., can also be selected and implemented according to the needs of those skilled in the art.

[0031] Furthermore, the boundary point information surrounding the display area is determined using an image segmentation model. Since the corresponding boundary points cannot completely surround the display area, this boundary point information is treated as boundary point information. Then, a linear interpolation algorithm is used to complete the boundary point information, resulting in complete boundary point information that ensures the corresponding boundary points completely surround the display area, presenting a complete boundary outline. Further, noise reduction processing is applied to the complete boundary point information to obtain the boundary frame point information. Based on the preset region positioning features, including the positional distribution information of the predetermined point regions within the anchor frame region, each predetermined point region can be determined within the anchor frame region. Then, based on the corresponding horizontal and vertical coordinate ranges of each predetermined point region, the coordinate information of each boundary point in the boundary frame point information is traversed to determine the predetermined point region to which each boundary point belongs. For each predetermined point region, all boundary points belonging to that region are treated as individual region contour points, and the coordinate information of these region contour points is collected to constitute the region contour point information of that predetermined point region. Based on the preset regional fixed-point features and the contour point information of each region, the coordinate information of the predetermined points in each predetermined point region is determined to form the predetermined point information. The coordinate information includes the corresponding horizontal and vertical coordinates of the predetermined point in the anchor frame region.

[0032] The location distribution information includes the corresponding horizontal and vertical coordinate ranges of the predetermined point areas within the anchor frame area. Those skilled in the art can set regional positioning features based on prior knowledge or experimental data, or implement them according to the relevant disclosures in subsequent embodiments. Regional positioning features include locating the coordinate distribution patterns of predetermined points within each predetermined point area. Those skilled in the art can set regional positioning features based on prior knowledge or experimental data, or implement them according to the relevant disclosures in subsequent embodiments.

[0033] Step S2200: Divide the display area into multiple subdivided areas according to the predetermined point information, and use the multiple subdivided areas arranged along the periphery of the display area as the color sampling areas for the lighting effect.

[0034] In one embodiment, each predetermined point can be located in the display area based on the predetermined point information. This can be achieved by connecting pairs of predetermined points that are opposite each other in the left and right or in the top and bottom positions in the display area with a preset distortion angle and a preset connection method between the corresponding points. Then, every two adjacent predetermined points are connected with a straight line to divide the display area into multiple subdivided areas.

[0035] It should be noted that the refined region refers to the region after subdivision correction. A preset fisheye correction algorithm is applied to correct the subdivision images corresponding to multiple target subdivision regions in the fisheye image of the display, resulting in the corresponding corrected subdivision images.

[0036] In other embodiments, each predetermined point can be located in the display area based on the predetermined point information. Furthermore, for some predetermined points, partition points near these predetermined points can be located. These predetermined points, or all predetermined points, and these partition points, are connected according to their relative left-right or vertical positions, using preset distortion angles and connection methods. Then, every two adjacent points are connected by straight lines to partition the display area, resulting in multiple subdivided regions. The coordinate information of the partition points near the predetermined points can be flexibly set by those skilled in the art based on prior knowledge or experimental data to locate the partition points.

[0037] The preset connection method can be any one or more of the following: straight line, curve, or polygonal line. The preset distortion angle can be flexibly set by those skilled in the art based on prior knowledge or experimental data.

[0038] Step S2300: Perform color sampling on each light effect color sampling area to determine the target color corresponding to each light effect color sampling area;

[0039] In one embodiment, for each lighting effect color sampling area, all pixels in the area are sampled to determine the color of each pixel. Colors are typically represented by RGB values. The sum of all pixel colors is then divided by the total number of pixels to obtain the average color of the lighting effect color sampling area. Since the display area captured by the camera is darker than the actual display area, color enhancement is applied to the average color to approximate a realistic visual effect. This is done by adding the increment values ​​of any one or more of the user-preset R, G, and B components to the values ​​of the same components in the average color to obtain the target color of the lighting effect color sampling area. For illustrative purposes, the average color is R: 100, G: 100, B: 100, the user-preset increment values ​​for R are 20 and B: 5, and the target color is R: 120, G: 100, B: 105.

[0040] Step S2400: Construct a lighting effect control signal based on the target color corresponding to each lighting effect color sampling area, and output the lighting effect control signal to the ambient light device to control the ambient light device to emit light according to the lighting effect control signal.

[0041] The control device can predetermine the mapping relationship between each lighting effect color sampling area and the corresponding light unit in the ambient lighting device. Specifically, since the ambient lighting device and each lighting effect color sampling area are arranged circumferentially around the display, all light units in the ambient lighting device can be divided into light units representing the total number of lighting effect color sampling areas. This ensures that each light unit and each lighting effect color sampling area have a positional equivalent relationship, thereby establishing a one-to-one mapping relationship between each light unit and each lighting effect color sampling area. Therefore, the control device can encapsulate lighting effect control signals based on the signal format of the WS2811 chip, according to the target color corresponding to each lighting effect color sampling area and the mapping relationship between each lighting effect color sampling area and the corresponding light unit in the ambient lighting device, to control each light unit of the ambient lighting device to emit light corresponding to the color of its mapped associated lighting effect color sampling area.

[0042] The control device connects to the ambient lighting device and outputs a lighting effect control signal to the device. This allows the ambient lighting device's lighting effect control circuit to control the light-emitting elements within the device to emit light accordingly, creating the desired lighting effect. The light-emitting elements can be monochromatic or RGB (red, green, and blue) light-emitting elements.

[0043] As can be seen from the above embodiments, the technical solution of this application has many advantages, including but not limited to the following aspects:

[0044] This application first extracts image frames from the video stream generated by the camera capturing the display, and then determines the display area of ​​the display's light emission imaging in each image frame, as well as the predetermined point information of each predetermined point set around the display area. This ensures that the display content of the display is captured accurately in real time, and determines each predetermined point on the boundary of the corresponding display area, laying the foundation for subsequent lighting effect color picking.

[0045] Next, based on the predetermined point information, the display area is divided into multiple sub-areas. These sub-areas, arranged circumferentially along the display area, are used as color sampling areas for lighting effects. The target color corresponding to each color sampling area is sampled, and lighting effect control information is constructed and output to the ambient lighting device. The ambient lighting device is then controlled to emit light accordingly, realizing the conversion of the target color corresponding to the displayed content to the corresponding ambient lighting effect. This extends the ambient lighting effect to the environment outside the monitor, enabling real-time dynamic adaptation to changes in the displayed content to present matching ambient lighting effects, creating a natural and beautiful viewing atmosphere, and significantly improving the user's visual experience and satisfaction.

[0046] In a further embodiment, step S2300, which involves color sampling of each lighting effect color sampling area to determine the target color corresponding to each lighting effect color sampling area, includes the following steps:

[0047] Step S2310: For each light effect color sampling area, perform pixel-by-pixel sampling on the light effect color sampling area, and determine the average color based on the total number of sampled pixels and the color of the pixels.

[0048] Pixel-by-pixel sampling involves sampling the color of one pixel within the color sampling area of ​​the lighting effect. The color is usually represented by RGB values, and there is a one-pixel interval between two consecutive samplings. The average color is obtained by summing the colors of all sampled pixels and then dividing by the total number of pixels.

[0049] Step S2320: Enhance the average color to obtain the target color of the color sampling area of ​​the lighting effect.

[0050] Since the display area captured by the camera is darker than the actual display area, color enhancement is performed on the average color to approximate a realistic visual effect. In one embodiment, the average color represented by RGB values ​​is converted to an average color represented by HSV values. The values ​​of the H and S components remain unchanged, while the value of the V component is increased by a preset increment to improve brightness. Then, the corresponding color represented by the HSV values ​​is converted to a color represented by RGB values ​​and used as the target color for the light effect's color sampling area. The preset increment value can be set as needed by those skilled in the art or can be customized by the user. In other embodiments, preset increment values ​​can also be added to the H and / or S components to optimize a personalized visual effect; those skilled in the art can flexibly adapt this to achieve the desired result.

[0051] In this embodiment, each lighting effect color sampling area is sampled at every other pixel, and the corresponding average color is calculated and then enhanced to obtain the target color. On the one hand, sampling the colors of spaced-out pixels reduces the amount of computation while still obtaining enough color to represent the entire lighting effect color sampling area. On the other hand, it can compensate for the error between the camera capture and the actual image, resulting in a better lighting effect display.

[0052] In a further embodiment, step S2100, determining the display area in the image frame that belongs to the imaging range of the display, and the predetermined point information of each predetermined point set around the display area, includes the following steps:

[0053] Step S2110: Use a preset image segmentation model to perform target detection and instance segmentation on the image frame, and determine the display area and its boundary point information in the image frame, as well as the anchor frame area of ​​the display area in the frame;

[0054] The image segmentation model is pre-trained to convergence, acquiring the ability to determine the display area within the imaging range of the display in the input image frame, its boundary point information, and the anchor box region of the display area within the frame. The description of the selection of the image segmentation model can be found in the relevant descriptions in the foregoing embodiments, and will not be repeated here.

[0055] An image segmentation model is employed, using image frames as input to perform object detection and instance segmentation. The model extracts corresponding semantic features from the image frame through convolution operations, including color, texture, boundaries, contours, and deep, complex, and abstract semantic information, generating multi-scale feature maps. The display area within the screen's imaging range is used as the target detection object. Multiple candidate bounding boxes are predicted on the feature maps. Overlapping candidate bounding boxes are filtered using non-maximum suppression to derive the anchor bounding box region of the target object. Furthermore, the model classifies each pixel in the image frame, constructing a pixel-level mask for the display area to segment it. Boundary detection is then performed to determine the coordinates of boundary points surrounding the display area, forming the boundary point information. The coordinates include the x-coordinate and y-coordinate of each boundary point within the anchor bounding box region.

[0056] Step S2120: After linearly completing the boundary partial point information, the corresponding obtained complete boundary point information is subjected to noise reduction processing to obtain the boundary frame point information;

[0057] Specifically, for the completion process, for each pair of adjacent boundary points in the boundary location information, the boundary point with the smaller relative x-coordinate and / or y-coordinate is used as the starting point, and the other boundary point as the ending point. The Euclidean distance, the absolute value of the difference in x-coordinate, and the absolute value of the difference in y-coordinate are calculated between these two boundary points. When the Euclidean distance is greater than or equal to a preset interval, the Euclidean distance is divided by the preset interval and rounded down to obtain the corresponding arithmetic result. Subtracting 1 from this arithmetic result gives the total number of boundary points to be added between these two boundary points. Then, based on this arithmetic result, a series of parameters uniformly distributed between 0 and 1 (excluding 0 and 1) are generated. For each parameter, it is multiplied by the absolute value of the difference in x-coordinate and the absolute value of the difference in y-coordinate, and then added to the x-coordinate and y-coordinate of the starting point respectively to obtain the coordinate information of the single boundary point to be added between the starting and ending points. When the Euclidean distance is less than the preset interval, it means that no additional boundary points need to be added between the two adjacent boundary points. The preset interval can be set as needed by those skilled in the art. For ease of understanding, let's illustrate with an example: the starting point's coordinates are (x1:0, y1:20), the ending point's coordinates are (x2:50, y2:40), and the preset spacing is 5. The Euclidean distance between these two boundary points is approximately 53.86. The absolute value of the difference in the x-coordinates is 50, and the absolute value of the difference in the y-coordinates is 20. Dividing this Euclidean distance by the preset spacing and rounding down yields a result of 10. The total number of boundary points needed to be added between these two boundary points is 9, which is between 0 and 1 (not included in the original text). The parameters (including 0 and 1) are uniformly distributed as 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, and 0.9, respectively. The coordinate information of each boundary point to be added is (x:5,y:22), (x:10,y:24), (x:15,y:26), (x:20,y:28), (x:25,y:30), (x:30,y:32), (x:35,y:34), (x:40,y:36), and (x:45,y:38).

[0058] Furthermore, the Ramer-Douglas-Peucker algorithm (RDP algorithm) is used to denoise the complete boundary point information, removing redundant boundary point coordinate information and retaining the coordinate information of multiple necessary boundary points that constitute the boundary contour of the display image without affecting its true shape, thus obtaining the boundary frame point information.

[0059] Step S2130: Based on the preset regional positioning features and boundary frame point information, determine the information of each predetermined point region and its regional outline points in the anchor frame region.

[0060] The regional positioning features include the location distribution information of each predetermined point's region within the anchor frame region. This location distribution information includes the corresponding horizontal and vertical coordinate ranges of the predetermined point's region within the anchor frame region. Those skilled in the art can set the regional positioning features based on prior knowledge or experimental data, or implement them according to the relevant disclosures in subsequent embodiments.

[0061] The width of the anchor frame region is the difference between the maximum and minimum x-coordinates in the complete boundary point information, and the height of the anchor frame region is the difference between the maximum and minimum y-coordinates in the complete boundary point information.

[0062] In one embodiment, if the image frame is captured by a single fisheye camera, based on eight predetermined points set for the complete imaging display area surrounding the display, corresponding to the four corner points, two midpoints of the long side, and two midpoints of the wide side of the display boundary, the positional distribution information corresponding to the predetermined point area of ​​each predetermined point in the anchor frame area is preset, as shown in Figure 3, which represents the x-coordinate range: [x] min 1 / 8*x max ], the range of the vertical axis is: [1 / 3*y max y max ]}(P1), {Horizontal coordinate range: [1 / 32*x max 7 / 32*x max ], the range of the ordinate is: [5 / 12*y max ,11 / 12*y max ]}(P2), {Horizontal coordinate range: [1 / 8*x max 1 / 4*x max ], the range of the ordinate is: [2 / 3*y max y max ]}(P3), {Horizontal coordinate range: [3 / 8*x max 5 / 8*x max ], the range of the ordinate is: [5 / 6*y max y max ]}(P4), {Horizontal coordinate range: [5 / 8*x max 3 / 4*x max ], the range of the ordinate is: [5 / 6*y max y max ]}(P5), {Horizontal coordinate range: [21 / 32*x max 31 / 32*x max ], the range of the vertical axis is: [3 / 12*y max ,11 / 12*y max ]}(P6), {Horizontal coordinate range: [7 / 8*x max x max ], the range of the vertical axis is: [1 / 6*ymax y max ]}(P7), {Horizontal coordinate range: [3 / 8*x max 5 / 8*x max ], y-axis range: [y min , 1 / 6*y max ]}(P8), which combines all location distribution information to form regional positioning features. x max x min y min y max These are the minimum x-coordinate, maximum x-coordinate, minimum y-coordinate, and maximum y-coordinate in the complete boundary point information.

[0063] If the image frame is captured by the left fisheye camera of a dual fisheye camera camera device, five predetermined points are set according to the display area surrounding the left half of the display. These points correspond to two corner points on the left edge of the display, the midpoint of the upper long side, the midpoint of the left wide side, and a point to the left of the midpoint of the lower long side at a predetermined distance. The positional distribution information of each predetermined point in the anchor frame area is predetermined. Please refer to Figure 4 for the following: {Horizontal coordinate range: [3 / 4*x] max， x max ], y-axis range: [y min , 1 / 6*y max ]}(P1), {x range: [x min 1 / 8*x max ], the range of the vertical axis is: [1 / 2y min ,2 / 3*y max ]}(P2), {Horizontal coordinate range: [1 / 16*x max， 9 / 16*x max ], the range of the ordinate is: [5 / 12*y max ,11 / 12*y max ]}(P3), {Horizontal coordinate range: [3 / 8*x max， 5 / 8*x max ], the range of the ordinate is: [5 / 6*y max y max ]}(P4), {Horizontal coordinate range: [3 / 4*x max， x max ], the range of the ordinate is: [5 / 6*y max y max ]}(P5), which combines all location distribution information to form regional positioning features. x max x min y min y max These are the minimum x-coordinate, maximum x-coordinate, minimum y-coordinate, and maximum y-coordinate in the complete boundary point information.

[0064] If the image frame is captured by the right fisheye camera of a dual fisheye camera camera device, five predetermined points are set according to the display area surrounding the right half of the display. These points correspond to two corner points on the right edge of the display, the midpoint of the upper long side, the midpoint of the right wide side, and a point located to the right of the midpoint of the lower long side at a predetermined distance from that point. The positional distribution information of each predetermined point within the anchor frame area is then predetermined. Please refer to Figure 5 for the following: {Horizontal coordinate range: [x]} min 1 / 4*x max ], y-axis range: [y min , 1 / 6*y max ]}(P1), {x range: [x min 1 / 4*x max ], the range of the ordinate is: [5 / 6*y max y max ]}(P2), {Horizontal coordinate range: [3 / 8*x max， 5 / 8*x max ], the range of the ordinate is: [5 / 6*y max y max ]}(P3), {Horizontal coordinate range: [7 / 16*x max， 15 / 16*x max ], the range of the ordinate is: [5 / 12*y max ,11 / 12*y max ]}(P4), {Horizontal coordinate range: [7 / 8*x max， x max ], the range of the ordinate is: [1 / 2*y max 5 / 6*y max ]}(P5), which combines all location distribution information to form regional positioning features. x max x min y min y max These are the minimum x-coordinate, maximum x-coordinate, minimum y-coordinate, and maximum y-coordinate in the complete boundary point information.

[0065] It is easy to understand that, based on the location distribution information of each predetermined point area in the regional positioning features, each predetermined point area can be determined in the anchor frame area. Then, based on the corresponding horizontal and vertical coordinate ranges of each predetermined point area, the coordinate information of each boundary point in the boundary frame point information is traversed to determine the predetermined point area to which each boundary point belongs. Then, for each predetermined point area, all the boundary points belonging to that predetermined point area are taken as individual area contour points, and the coordinate information of these area contour points is collected to form the area contour point information of that predetermined point area.

[0066] Step S2140: Based on the preset regional fixed point features and the contour point information of each region, determine the coordinate information of the predetermined points in each predetermined point region to form the predetermined point information.

[0067] The regional positioning features include locating the coordinate distribution pattern of predetermined points in each predetermined point region. Those skilled in the art can set the regional positioning features based on prior knowledge or experimental data, or implement them according to the relevant disclosures in the subsequent embodiments.

[0068] If the image frame is captured by a camera device using a single fisheye camera, for {the horizontal coordinate range is: [3 / 8*x] max 5 / 8*x max ], y-axis range: [y min , 1 / 6*y max The predetermined point area is defined by determining the area contour point information of the predetermined point area. The x-coordinate is the value obtained by multiplying the width of the anchor frame area by a first preset ratio and then adding the minimum x-coordinate from the boundary contour information. The y-coordinate is the minimum y-coordinate from the area contour point information. The coordinate information of the corresponding area contour point is used as the coordinate information of the predetermined point of the predetermined point area. The first preset ratio can be flexibly set by a technician in the field based on the monitor size and the shooting angle of the monocular camera, for example, 50%. For the x-coordinate range: [3 / 8*x...] max 5 / 8*x max ], the range of the ordinate is: [5 / 6*y max y max For the predetermined point area, the region contour information of the predetermined point area is determined. The horizontal coordinate is the value obtained by multiplying the width of the anchor frame area by a first preset ratio and adding the minimum horizontal coordinate in the boundary contour information. The vertical coordinate is the region contour point corresponding to the maximum vertical coordinate in the region contour point information. The coordinate information of the corresponding region contour point is used as the coordinate information of the predetermined point of the predetermined point area. For the horizontal coordinate range: [x min 1 / 8*x max ], the range of the vertical axis is: [1 / 3*y max y max In one embodiment, the coordinates of the region contour point with the smallest abscissa in the region contour information of the predetermined point region are determined, or the coordinates of the region contour point with the largest ordinate among N region contour points with smaller abscissas in the region contour information are determined, and these coordinates are used as the coordinates of the predetermined point of the predetermined point region. N can be set as needed by those skilled in the art, for example, 5; for {abscissa range: [7 / 8*x} max x max ], the range of the vertical axis is: [1 / 6*y max y maxIn one embodiment, for the predetermined point region, the coordinates of the region contour point with the largest abscissa in the region contour information of the predetermined point region are determined, or the coordinates of the region contour point with the largest ordinate among the N region contour points with large abscissas in the region contour information are determined, and these coordinates are used as the coordinates of the predetermined point of the predetermined point region; for {abscissa range: [1 / 8*x] max 1 / 4*x max ], the range of the ordinate is: [2 / 3*y max y max The predetermined point area of ​​]}, {x-coordinate range: [5 / 8*x max 3 / 4*x max ], the range of the ordinate is: [5 / 6*y max y max For the predetermined point region, firstly, following the clockwise direction around the boundary outline of the display image, the coordinate information of each boundary point in the boundary frame point information is sorted to obtain a boundary point sequence. From the boundary point sequence, the vector formed by each boundary point and its next boundary point is determined sequentially. Every two vectors are adjacent in position. Therefore, the inverse cosine value between every two vectors is calculated, which is the angle between these two vectors. Then, for each predetermined point region, the coordinate information of the intersection point between the two adjacent vectors with the largest angle is determined from the boundary point information of that predetermined point region, and this is used as the predetermined point information of that predetermined point region; for {x-coordinate range: [1 / 32*x max 7 / 32*x max ], the range of the ordinate is: [5 / 12*y max ,11 / 12*y max The predetermined point area of ​​]}, {x-coordinate range: [21 / 32*x max 31 / 32*x max ], the range of the vertical axis is: [3 / 12*y max ,11 / 12*y max For each predetermined point region, the coordinate information of the contour points of the two adjacent predetermined points that are equidistant from the predetermined point region is determined, and these coordinates are used as the coordinate information of the predetermined points in that predetermined point region.

[0069] If the image frame is captured by the left fisheye camera of a dual fisheye camera camera device, for {the horizontal coordinate range is: [3 / 4*x] max x max ], y-axis range: [y min , 1 / 6*y maxThe predetermined point area is defined by determining the area contour point information of the predetermined point area. The x-coordinate is the value obtained by multiplying the width of the anchor frame area by a second preset ratio and adding the minimum x-coordinate from the boundary contour information, and the y-coordinate is the minimum y-coordinate from the area contour point information. The coordinate information of the corresponding area contour point is used as the coordinate information of the predetermined point of the predetermined point area. The second preset ratio can be flexibly set by technical personnel according to the monitor size and the shooting angle of the monocular camera, for example, 68%; for {x-coordinate range: [3 / 4*x max x max ], the range of the ordinate is: [5 / 6*y max y max For the predetermined point area, the region contour information of the predetermined point area is determined. The horizontal coordinate is the value obtained by multiplying the width of the anchor frame area by the second preset ratio and adding the minimum horizontal coordinate in the boundary contour information. The vertical coordinate is the region contour point corresponding to the maximum vertical coordinate in the region contour point information. The coordinate information of the corresponding region contour point is used as the coordinate information of the predetermined point of the predetermined point area. For the horizontal coordinate range: [x min 1 / 8*x max ], the range of the vertical axis is: [1 / 2y min ,2 / 3*y max In one embodiment, the coordinates of the region contour point with the smallest abscissa in the region contour information of the predetermined point region are determined, or the coordinates of the region contour point with the largest ordinate among N region contour points with smaller abscissas in the region contour information are determined, and these coordinates are used as the coordinates of the predetermined point of the predetermined point region. N can be set as needed by those skilled in the art, for example, 5; for {abscissa range: [3 / 8*x} max 5 / 8*x max ], the range of the ordinate is: [5 / 6*y max y max For the predetermined point region, firstly, following the clockwise direction around the boundary outline of the display image, the coordinate information of each boundary point in the boundary frame point information is sorted to obtain a boundary point sequence. From the boundary point sequence, the vector formed by each boundary point and its next boundary point is determined sequentially. Every two vectors are adjacent in position. Therefore, the inverse cosine value between every two vectors is calculated, which is the angle between these two vectors. Then, for the predetermined point region, the coordinate information of the intersection point between the two adjacent vectors with the largest angle is determined from the boundary point information of the predetermined point region, and this is used as the predetermined point information of the predetermined point region; for {x-coordinate range: [1 / 16*x max 9 / 16*x max ], the range of the ordinate is: [5 / 12*y max ,11 / 12*y maxThe coordinate information of the outline points of the region where the ratio of the distance between two adjacent predetermined points in the predetermined point region belongs to the first preset value range is determined, and these coordinates are used as the coordinate information of the predetermined points in the predetermined point region. The first preset value range can be flexibly set by those skilled in the art according to the display size. For example, the first preset value range corresponding to a 65-inch display can be [0.52-0.55].

[0070] If the image frame is captured by the right fisheye camera of a dual fisheye camera camera device, for the x-coordinate range: [x min 1 / 4*x max ], y-axis range: [y min , 1 / 6*y max The predetermined point area is defined by determining the area contour point information of the predetermined point area. The x-coordinate is the width of the anchor frame area multiplied by a third preset ratio, plus the minimum x-coordinate in the boundary contour information. The y-coordinate is the minimum y-coordinate in the area contour point information. The coordinate information of the corresponding area contour point is used as the coordinate information of the predetermined point of the predetermined point area. The third preset ratio can be flexibly set by the technical personnel based on the monitor size and the shooting angle of the monocular camera, for example, 32%. For the x-coordinate range: [x...] min 1 / 4*x max ], the range of the ordinate is: [5 / 6*y max y max For the predetermined point area, the region contour information of the predetermined point area is determined. The horizontal coordinate is the value obtained by multiplying the width of the anchor frame area by the third preset ratio and adding the minimum horizontal coordinate in the boundary contour information. The vertical coordinate is the region contour point corresponding to the maximum vertical coordinate in the region contour point information. The coordinate information of the corresponding region contour point is used as the coordinate information of the predetermined point of the predetermined point area. For the horizontal coordinate range: [7 / 8*x max x max ], the range of the ordinate is: [1 / 2*y max 5 / 6*y max In one embodiment, the coordinates of the region contour point with the largest abscissa in the region contour information of the predetermined point region are determined, or the coordinates of the region contour point with the largest ordinate among N region contour points with large abscissas in the region contour information are determined, and these coordinates are used as the coordinates of the predetermined point of the predetermined point region. N can be set as needed by those skilled in the art, for example, 5; for {abscissa range: [3 / 8*x} max 5 / 8*x max ], the range of the ordinate is: [5 / 6*y max y maxFor the predetermined point region, firstly, following the clockwise direction around the boundary outline of the display image, the coordinate information of each boundary point in the boundary frame point information is sorted to obtain a boundary point sequence. From the boundary point sequence, the vector formed by each boundary point and its next boundary point is determined sequentially. Every two vectors are adjacent in position. Therefore, the inverse cosine value between every two vectors is calculated, which is the angle between these two vectors. Then, for the predetermined point region, the coordinate information of the intersection point between the two adjacent vectors with the largest angle is determined from the boundary point information of the predetermined point region, and this is used as the predetermined point information of the predetermined point region; for {x-coordinate range: [1 / 16*x max 9 / 16*x max ], the range of the ordinate is: [5 / 12*y max ,11 / 12*y max The coordinate information of the outline points of the region where the ratio of the distance between two adjacent predetermined points in the predetermined point region belongs to the second preset value range is determined, and these coordinates are used as the coordinate information of the predetermined points in the predetermined point region. The second preset value range can be flexibly set by those skilled in the art according to the display size. For example, the first preset value range corresponding to a 65-inch display can be [0.45-0.48].

[0071] In this embodiment, by employing an image segmentation model to perform target detection and instance segmentation on the image frame, the display area and its boundary point information, as well as the anchor frame area of ​​the display area within the frame, are determined. A linear interpolation algorithm is used to complete the boundary point information, resulting in complete boundary point information. Based on preset area positioning features and complete boundary point information, each predetermined point area and its area contour point information within the anchor frame area are determined. Based on preset area positioning features and each area contour point information, the coordinate information of the predetermined points in each predetermined point area is determined to constitute the predetermined point information. This ensures the accuracy and reliability of the coordinate information of each predetermined point determined from each predetermined point area, based on the boundary points of the complete boundary contour of the display area.

[0072] In a further embodiment, step S2140, determining the coordinate information of predetermined points in each predetermined point region to constitute predetermined point information based on preset regional fixed-point features and contour point information of each region, includes any one or more of the following steps:

[0073] Step S2141: For each predetermined point region containing a predetermined point belonging to the midpoint of the long side of the display, determine the coordinate information of the predetermined point in the predetermined point region from the region outline point information of the predetermined point region based on the position of the predetermined point region in the anchor frame region.

[0074] If the image frame is captured by a camera device using a single fisheye camera, and the predetermined point area is located directly above the anchor frame area ({horizontal coordinate range: [3 / 8*x) max 5 / 8*x max ], y-axis range: [y min , 1 / 6*y max From the region contour point information of the predetermined point area, determine the region contour point whose horizontal coordinate is the width of the anchor frame area multiplied by a first preset ratio and then added to the minimum horizontal coordinate in the boundary contour information, and whose vertical coordinate is the region contour point corresponding to the minimum vertical coordinate in the region contour point information. This region contour point is then used as the coordinate information of the predetermined point of the predetermined point area. The first preset ratio can be flexibly set by those skilled in the art based on the monitor size and the shooting angle of the monocular camera, for example, 50%.

[0075] If the image frame is captured by a camera device using a single fisheye camera, and the predetermined point area is located directly below the anchor frame area ({horizontal coordinate range: [3 / 8*x) max 5 / 8*x max ], the range of the ordinate is: [5 / 6*y max y max From the region contour point information of the predetermined point area, determine the region contour point whose horizontal coordinate is the width of the anchor frame area multiplied by the first preset ratio and then added to the minimum horizontal coordinate in the boundary contour information, and whose vertical coordinate is the region contour point corresponding to the maximum vertical coordinate in the region contour point information. This region contour point is used as the predetermined point of the predetermined point area, and the coordinate information of the region contour point is used as the coordinate information of the predetermined point of the predetermined point area.

[0076] If the image frame is captured by the left fisheye camera of a camera device employing dual fisheye cameras, and the predetermined point area is located at the upper right corner of the anchor frame area ({x-coordinate range: [3 / 4*x) max， x max ], y-axis range: [y min , 1 / 6*y max From the region contour information of the predetermined point area, the x-coordinate is determined by multiplying the width of the anchor frame area by a second preset ratio and adding the minimum x-coordinate in the boundary contour information, and the y-coordinate is the region contour point corresponding to the minimum y-coordinate in the region contour point information. This y-coordinate is then used as the predetermined point of the predetermined point area, and the coordinate information of this region contour point is used as the coordinate information of the predetermined point of the predetermined point area. The second preset ratio can be flexibly set by a technician in the field according to the monitor size and the shooting angle of the left fisheye camera, for example, 68%.

[0077] If the image frame is captured by the left fisheye camera of a camera device employing dual fisheye cameras, and the predetermined point area is located at the lower right corner of the anchor frame area ({x-coordinate range: [3 / 4*x) max， x max ], the range of the ordinate is: [5 / 6*y max y max From the region contour information of the predetermined point area, determine the x-coordinate as the width of the anchor frame area multiplied by the second preset ratio and then added to the minimum x-coordinate in the boundary contour information, and the y-coordinate as the region contour point corresponding to the maximum y-coordinate in the region contour point information, and use it as the predetermined point of the predetermined point area, and use the coordinate information of the region contour point as the coordinate information of the predetermined point of the predetermined point area.

[0078] If the image frame is captured by the right fisheye camera of a camera device employing dual fisheye cameras, and the predetermined point region is located at the upper left corner of the anchor frame region ({x-coordinate range: [ ... min 1 / 4*x max ], y-axis range: [y min , 1 / 6*y max From the region contour information of the predetermined point area, the x-coordinate is determined by multiplying the width of the anchor frame area by a third preset ratio and adding the minimum x-coordinate in the boundary contour information, and the y-coordinate is the region contour point corresponding to the minimum y-coordinate in the region contour point information. This y-coordinate is then used as the predetermined point of the predetermined point area, and the coordinate information of this region contour point is used as the coordinate information of the predetermined point of the predetermined point area. The third preset ratio can be flexibly set by technical personnel according to the monitor size and the shooting angle of the right fisheye camera, for example, 32%.

[0079] If the image frame is captured by the right fisheye camera of a camera device employing dual fisheye cameras, and the predetermined point area is located at the lower left corner of the anchor frame area ({x-coordinate range: [ ... min 1 / 4*x max ], the range of the ordinate is: [5 / 6*y max y max From the region contour information of the predetermined point area, determine the x-coordinate as the width of the anchor frame area multiplied by the third preset ratio and then added to the minimum x-coordinate in the boundary contour information, and the y-coordinate as the region contour point corresponding to the maximum y-coordinate in the region contour point information, and use it as the predetermined point of the predetermined point area, and use the coordinate information of the region contour point as the coordinate information of the predetermined point of the predetermined point area.

[0080] Step S2142: For each predetermined point region containing predetermined points belonging to the corner points of the display, determine the vector formed by each two adjacent boundary points and the angle between each two adjacent vectors based on the boundary frame point information. From the region boundary point information of the predetermined point region, determine the coordinate information of the intersection point between the two adjacent vectors with the largest angle, and use it as the predetermined point information of the predetermined point region.

[0081] If the image frame is captured by a camera device using a single fisheye camera, the regions of each predetermined point, including the predetermined points belonging to the corner points of the display, are respectively located within the anchor frame region {x | x}} min 1 / 8*x max ], the range of the vertical axis is: [1 / 3*y max y max ]}, {Horizontal coordinate range: [1 / 8*x max 1 / 4*x max ], the range of the ordinate is: [2 / 3*y max y max ]}、{Horizontal coordinate range: [5 / 8*x max 3 / 4*x max ], the range of the ordinate is: [5 / 6*y max y max ]}、{Horizontal coordinate range: [7 / 8*x max x max ], the range of the vertical axis is: [1 / 6*y max y max First, following the clockwise contour of the image on the display, the coordinates of each boundary point in the boundary frame point information are sorted to obtain a boundary point sequence. From the boundary point sequence, the vector formed by each boundary point and its next boundary point is determined sequentially. Every two vectors are adjacent in position. Thus, the inverse cosine value between every two vectors is calculated, which is the angle between the two vectors. Then, for each predetermined point area, the coordinates of the intersection point between the two adjacent vectors with the largest angle are determined from the boundary point information of the predetermined point area, and this is used as the predetermined point information of the predetermined point area.

[0082] If the image frame is captured by the left fisheye camera of a camera device using dual fisheye cameras, the regions of each predetermined point, including the predetermined points belonging to the corners of the display, are respectively located within the anchor frame region {x_0, x_1, x_2, x_3, x_4, x_5, x_6, x_7, x_8, x_9, x_1, x_2 ... min 1 / 8*x max ], the range of the vertical axis is: [1 / 2y min ,2 / 3*y max ]}、{Horizontal coordinate range: [3 / 8*x max， 5 / 8*x max], the range of the ordinate is: [5 / 6*y max y max First, following the clockwise contour of the image on the display, the coordinates of each boundary point in the boundary frame point information are sorted to obtain a boundary point sequence. From the boundary point sequence, the vector formed by each boundary point and its next boundary point is determined sequentially. Every two vectors are adjacent in position. Thus, the inverse cosine value between every two vectors is calculated, which is the angle between the two vectors. Then, for each predetermined point area, the coordinates of the intersection point between the two adjacent vectors with the largest angle are determined from the boundary point information of the predetermined point area, and this is used as the predetermined point information of the predetermined point area.

[0083] If the image frame is captured by the right fisheye camera of a camera device using dual fisheye cameras, the regions of each predetermined point, including the predetermined points belonging to the corners of the display, are respectively located in the anchor frame region {horizontal coordinate range: [3 / 8*x] max， 5 / 8*x max ], the range of the ordinate is: [5 / 6*y max y max ]}、{Horizontal coordinate range: [7 / 8*x max， x max ], the range of the ordinate is: [1 / 2*y max 5 / 6*y max First, following the clockwise contour of the image on the display, the coordinates of each boundary point in the boundary frame point information are sorted to obtain a boundary point sequence. From the boundary point sequence, the vector formed by each boundary point and its next boundary point is determined sequentially. Every two vectors are adjacent in position. Thus, the inverse cosine value between every two vectors is calculated, which is the angle between the two vectors. Then, for each predetermined point area, the coordinates of the intersection point between the two adjacent vectors with the largest angle are determined from the boundary point information of the predetermined point area, and this is used as the predetermined point information of the predetermined point area.

[0084] Step S2143: For each predetermined point region containing a predetermined point belonging to the midpoint of the wide edge of the display, based on the coordinate information of two predetermined points adjacent to the predetermined point region, determine the coordinate information of the region contour points whose distance ratio to the two predetermined points satisfies a preset condition from the region contour point information of the predetermined point region, and use them as the coordinate information of the predetermined points in the predetermined point region.

[0085] If the image frame is captured by a camera device using a single fisheye camera, the regions of each predetermined point, including the predetermined point belonging to the midpoint of the wide edge of the display, are respectively located within the anchor frame region {horizontal coordinate range: [1 / 32*x] max 7 / 32*xmax ], the range of the ordinate is: [5 / 12*y max ,11 / 12*y max ]}、{Horizontal coordinate range: [21 / 32*x max 31 / 32*x max ], the range of the vertical axis is: [3 / 12*y max ,11 / 12*y max For each predetermined point area, the coordinate information of the area contour points that are equidistant from the area contour point information of the predetermined point area is determined, and these coordinates are used as the coordinate information of the predetermined point in the predetermined point area.

[0086] If the image frame is captured by the left fisheye camera of a camera device using dual fisheye cameras, the regions of each predetermined point, including the predetermined point belonging to the midpoint of the wide edge of the display, are respectively located in the anchor frame region {horizontal coordinate range: [1 / 16*x] max， 9 / 16*x max ], the range of the ordinate is: [5 / 12*y max ,11 / 12*y max From the region contour point information of the predetermined point region, the coordinate information of the region contour points whose distance ratio between two adjacent predetermined points belongs to the first preset value range is determined, and used as the coordinate information of the predetermined points in the predetermined point region. The first preset value range can be flexibly set by those skilled in the art according to the display size. For example, the first preset value range corresponding to a 65-inch display can be [0.52-0.55].

[0087] If the image frame is captured by the right fisheye camera of a camera device using dual fisheye cameras, the regions of each predetermined point, including the predetermined point belonging to the midpoint of the wide edge of the display, are respectively located in the anchor frame region {horizontal coordinate range: [7 / 16*x] max， 15 / 16*x max ], the range of the ordinate is: [5 / 12*y max ,11 / 12*y max From the region contour point information of the predetermined point region, the coordinate information of the region contour points whose distance ratio between two adjacent predetermined points belongs to the second preset value range is determined, and used as the coordinate information of the predetermined points in the predetermined point region. The second preset value range can be flexibly set by those skilled in the art according to the display size. For example, the second preset value range corresponding to a 65-inch display can be [0.45-0.48].

[0088] This embodiment reveals the process of locating predetermined points in each predetermined point region, ensuring that each predetermined point corresponds to the midpoint of the long side, the midpoint of the wide side, and the corner point of the display boundary.

[0089] In a further embodiment, step S2300, which involves color sampling of each lighting effect color sampling area to determine the target color corresponding to each lighting effect color sampling area, may further include the following steps:

[0090] Step S2301: For each light effect color sampling area, determine the core hue range of the light effect color sampling area and the colors of multiple pixels whose corresponding hue values ​​meet the preset conditions, and calculate the corresponding average color.

[0091] Taking a single lighting effect color sampling area as an example, the area is sampled pixel by pixel to determine the color of each corresponding pixel. Colors are typically represented by RGB values. The RGB values ​​of each pixel are then converted to their corresponding HSV values. Multiple preset hue intervals are obtained. These intervals can be defined by those skilled in the art as needed, such as dividing the area into [0°, 360°] intervals. For example, they can be divided into six hue intervals: [0°, 60°), [60°, 120°), [120°, 180°), [180°, 240°), [240°, 300°), and [300°, 360°]. Based on the hue value in the HSV representation of each pixel, the hue interval to which each pixel belongs can be determined. For each hue interval, the total number of pixels belonging to that interval is determined, and the hue interval with the largest total number of pixels is selected as the core hue region. For all pixels belonging to the core hue range, the colors represented by HSV are sorted in descending order of hue value. The top N pixels in terms of HSV color are selected, converted to RGB color, and their average value is calculated. This average value is used as the average color of the lighting effect's color sampling area. N can be set as needed by those skilled in the art, for example, 10.

[0092] Step S2302: Enhance the average color to obtain the target color of the color sampling area of ​​the lighting effect.

[0093] Since the display effect captured by the camera is darker than the actual display effect, in order to approximate the real visual effect, the average color is enhanced by adding the incremental value of any one or more of the R, G, and B components preset by the user to the value of the same component of the average color to obtain the target color of the color sampling area of ​​the lighting effect.

[0094] In this embodiment, each lighting effect color sampling area is sampled at every other pixel, and the average color corresponding to the core hue interval of the lighting effect color sampling area is calculated and then enhanced to obtain the target color. On the one hand, sampling the colors of spaced-out pixels reduces the amount of computation while still obtaining enough color to represent the entire lighting effect color sampling area. On the other hand, it can compensate for the error between the camera capture and the actual image, resulting in a better subsequent lighting effect display.

[0095] In a further embodiment, step S2143, for each predetermined point region containing a predetermined point belonging to the midpoint of the wide edge of the display, determines the coordinate information of the region contour points whose distance ratio to the two predetermined points satisfies a preset condition from the region contour point information of the predetermined point region based on the coordinate information of two predetermined points adjacent to the predetermined point region, and uses them as the coordinate information of the predetermined points in the predetermined point region, including the following steps:

[0096] Step S21431: When the image frame belongs to the single-eye shooting type, determine the coordinate information of the region contour points of two adjacent predetermined points with the same distance as the predetermined point region, and use them as the coordinate information of the predetermined points in the predetermined point region.

[0097] It is understandable that different camera identification codes can be preset for three types of camera devices: one using a single fisheye camera, one using the left fisheye camera of a dual fisheye camera device, and one using the right fisheye camera of a dual fisheye camera device. Therefore, when the camera identification code of the camera device from which the image frame originates is identified as a single fisheye camera, it is determined that the image frame belongs to the monocular shooting type. At this point, based on the coordinate information of the predetermined points in the two predetermined point regions adjacent to the predetermined point region, the region contour points with the same Euclidean distance as these two predetermined points are determined from the coordinate information of the region contour points of that predetermined point region as the predetermined points in that predetermined point region. The coordinate information of these region contour points is then used as the coordinate information of the predetermined points in that predetermined point region.

[0098] Step S21432: When the image frame belongs to the left-side binocular shooting type, determine the coordinate information of the region contour point whose ratio of the distance between two adjacent predetermined points belongs to the first preset value range, and use it as the coordinate information of the predetermined point in the predetermined point region.

[0099] When the camera identification code of the camera device from which the image frame originates is identified as the camera identification code of the left eye camera, it is determined that the image frame belongs to the binocular left-side shooting type. At this time, based on the coordinate information of the predetermined points in the two predetermined point regions adjacent to the predetermined point region, each region contour point is arranged in descending order of its horizontal coordinate from the coordinate information of the region contour points in the predetermined point region. This process is repeated for each region contour point. The Euclidean distance between the region contour point and the predetermined point to its left is divided by the Euler distance between the region contour point and the predetermined point to its right to obtain the distance ratio. When the distance ratio is within the first preset value range, the region contour point is taken as the predetermined point in the predetermined point region, and its coordinate information is taken as the coordinate information of the predetermined point in the predetermined point region, and the traversal stops. When the distance ratio is not within the first preset value range, the next region contour point is traversed.

[0100] The first preset value range can be flexibly set by those skilled in the art according to the size of the display. For example, the first preset value range corresponding to a 65-inch display can be [0.52-0.55].

[0101] Step S21433: When the image frame belongs to the binocular right-side shooting type, determine the coordinate information of the region contour point whose ratio of the distance between two adjacent predetermined points belongs to the second preset value range, and use it as the coordinate information of the predetermined point in the predetermined point region.

[0102] When the camera identification code of the camera device from which the image frame originates is identified as the camera identification code of the right-side fisheye camera, it is determined that the image frame belongs to the binocular right-side shooting type. At this time, based on the coordinate information of the predetermined points in the two predetermined point regions adjacent to the predetermined point region, each region contour point is arranged in descending order of its horizontal coordinate from the coordinate information of the region contour points in the predetermined point region. This process is repeated for each region contour point. The Euclidean distance between the region contour point and the predetermined point to its left is divided by the Euler distance between the region contour point and the predetermined point to its right to obtain the distance ratio. When the distance ratio is within the second preset value range, the region contour point is taken as the predetermined point in the predetermined point region, and its coordinate information is taken as the coordinate information of the predetermined point in the predetermined point region, and the traversal stops. When the distance ratio is not within the second preset value range, the next region contour point is traversed.

[0103] The second preset value range can be flexibly set by those skilled in the art according to the size of the display. For example, the second preset value range corresponding to a 65-inch display can be [0.45-0.48].

[0104] In this embodiment, for image frames belonging to the monocular shooting type, the binocular left-side shooting type, and the binocular right-side shooting type respectively, a method for determining the coordinate information of the predetermined point in the corresponding predetermined point region is disclosed, which can accurately locate the predetermined point for different shooting angles.

[0105] In other embodiments, a pre-trained image classification model, trained to a convergent state, can be used to determine whether an image frame belongs to the binocular right-side shooting type, or the binocular left-side shooting type, or the monocular shooting type. Those skilled in the art can flexibly implement the image classification model based on the disclosure herein.

[0106] In a further embodiment, before step S2110, which involves using a preset image segmentation model to perform target detection and instance segmentation on the image frame to determine the display area and its boundary point information in the image frame, as well as the anchor frame area of ​​the display area within the frame, the ambient light control method further includes:

[0107] Step S2000: Obtain a preset training set and train the image segmentation model to a convergent state, enabling it to learn the ability to determine the display area and its boundary point information belonging to the imaging range of the display from the input image, as well as the anchor box area of ​​the display area within the frame. The training set includes multiple training samples and their supervision labels. The training samples are images containing the display area, and the supervision labels annotate the target boundary detection information corresponding to the display area in the corresponding training samples. The training samples in the training set are obtained by taking pictures of the display using a single fisheye camera, and / or include pictures of the display taken by the left fisheye camera of a dual fisheye camera and the right fisheye camera of a dual fisheye camera respectively.

[0108] In the recommended embodiment, multiple images of the display produced by a single fisheye camera captured by a camera device under different environments are pre-acquired. Similarly, multiple images of the display produced by the left fisheye camera of a dual fisheye camera device under different environments are also pre-acquired, as are multiple images of the display produced by the right fisheye camera of the same dual fisheye camera device under different environments. It is easy to understand that each acquired image contains a display area corresponding to the display's imaging range. For example, the image produced by the single fisheye camera shows the complete image of the display, the image produced by the left fisheye camera shows the left half of the display, and the image produced by the right fisheye camera shows the right half of the display. Different environments can be any one or more of the following: different times, different objects placed next to the display, the display placed on different objects, the display attached to or suspended from different objects, or the display in different spaces. This enhances the generalization ability, reliability, and accuracy of the image segmentation model.

[0109] Each image is used as a single training sample. Furthermore, the labelme and labelimg labeling tools are used to label each training sample. Taking a single training sample as an example, the display area in the image is taken as the target detection object. The labelimg tool is used to outline the specific location and size of the target detection object in the image. Additionally, the labelme tool is used to set corresponding boundary points along the boundary contour of the target detection object in the image. These boundary points are connected to form a boundary contour that completely encloses the target detection object. After annotation, the labelme and labelimg tools generate corresponding annotation files containing the target boundary detection information of the display area, which serve as the supervision labels for the training sample. The target boundary detection information includes the coordinates of the center pixel of the display area in the corresponding image (which is also the coordinates of the center pixel of the anchor frame that frames the display area), the width and height of the display area in the image (which is also the width and height of the anchor frame), and the coordinates of each boundary point of the display area's boundary contour in the image.

[0110] The description of the selection of image segmentation model can be found in the relevant descriptions in the foregoing embodiments, and will not be repeated here.

[0111] The image segmentation model uses a single training sample and its supervision label from the training set as input to perform object detection and instance segmentation. By performing convolution operations on the training sample, the model extracts corresponding semantic features, including color, texture, boundary, contour, and deep, complex, and abstract semantic information, generating multi-scale feature maps. Using the display area in the training sample as the target detection object, the model predicts multiple candidate bounding boxes on the feature maps. Non-maximum suppression filters overlapping candidate bounding boxes, predicting the anchor bounding box region of the target object within the box. Furthermore, the image segmentation model classifies each pixel in the training sample, constructing a pixel-level mask for the display area to segment the predicted display area. It also performs boundary detection on the predicted display area, determining the coordinates of each boundary point surrounding the predicted display area's boundary, thus forming the predicted boundary point information. The cross-entropy loss function is used based on the supervision labels of the training samples to calculate the cross-entropy loss corresponding to the predicted anchor box region, predicted display region, and predicted boundary part point information. When the loss value is lower than the preset threshold, it indicates that the image segmentation model has been trained to a convergent state, and the training of the image segmentation model can be terminated. When the loss value is greater than or equal to the preset threshold, it indicates that the image segmentation model has not converged. Therefore, gradient updates are performed on the image segmentation model according to the loss value. Typically, backpropagation is used to correct the weight parameters of each link in the image segmentation model to make the image segmentation model further approach convergence. Then, other training samples and their supervision labels in the training set are used to iteratively train the image segmentation model until the image segmentation model is trained to a convergent state.

[0112] This embodiment reveals the training process of the image segmentation model, ensuring the accuracy and reliability of the image segmentation model in recognizing the point information of the display area and its boundary parts, and is able to process images generated by the display captured by the left fisheye camera of the camera device using a single fisheye camera, the right fisheye camera of the camera device using a dual fisheye camera, and the right fisheye camera of the camera device using a dual fisheye camera.

[0113] In a further embodiment, step S2200, dividing the display area into multiple subdivided areas based on the predetermined point information, and using the multiple subdivided areas arranged circumferentially along the display area as the lighting effect color sampling areas, includes the following steps:

[0114] Step S2210: Use the fisheye correction algorithm to correct the image in each subdivided region;

[0115] The fisheye correction algorithm of the OpenCV library can be used to correct each subdivided region, thereby indirectly achieving image correction of the display area. Of course, those skilled in the art can also use other open source fisheye correction algorithms, such as a deep learning model that has been pre-trained to convergence, which learns to correct the image captured by fisheye through training.

[0116] Step S2220: For each corrected subdivision area arranged circumferentially along the display area, the central part of the subdivision area is cropped out as the color sampling area for the lighting effect.

[0117] For each corrected subdivided region arranged circumferentially along the display area, considering that the imaging height of the central portion of each subdivided region is close to that of the imaging from a conventional viewing angle, i.e., the error is minimal, the central portion of each subdivided region is cropped out as the color sampling area for the lighting effect. The central portion can be set as needed by those skilled in the art; for example, the central portion may occupy 85% of its respective subdivided region, and the center point of both regions may be the same.

[0118] In this embodiment, by correcting the subdivided image and then cropping out the central part of it to obtain the color sampling area for the lighting effect, visual distortion can be corrected, ensuring the color accuracy in the color sampling area for the lighting effect, laying the foundation for the subsequent accurate color sampling and display of the lighting effect, and ensuring the user's viewing experience.

[0119] Please refer to Figure 6. In another typical embodiment of the ambient light control method of this application, step 2210, which involves using a fisheye correction algorithm to perform image correction on each subdivided region, further includes the following steps:

[0120] Step S1100: Obtain the fisheye image of the display. Perform boundary positioning on the image instance segmentation result of the fisheye image of the display according to the preset boundary positioning rules to obtain the boundary point information of the corresponding region of the fisheye image of the display.

[0121] The camera in the ambient lighting control device captures images within its field of view, generating a corresponding video stream. The control unit, connected to the camera, can read the video stream captured by the camera and displayed on the screen. Therefore, when a single fisheye camera captures images of the screen, the control unit obtains the video stream and performs frame segmentation processing, dividing the video stream into multiple image frames to obtain the frames arranged in chronological order. The description of the frame segmentation processing of the video stream can be found in the aforementioned embodiments and will not be repeated here.

[0122] A pre-defined image segmentation model is used, taking an image frame as input, to perform object detection and image instance segmentation on the image frame. The image segmentation model extracts the corresponding semantic features of the image frame by performing convolution operations, including the image frame's color, texture, boundaries, and deep, complex, and abstract semantic information, generating corresponding multi-scale feature maps. The display fisheye image, which belongs to the imaging range of the display in the image frame, is used as the object detection object. Multiple candidate box regions are predicted on the feature map. The non-maximum suppression algorithm is used to filter overlapping candidate box regions to obtain the anchor box region of the object detection object in the box. In addition, the image segmentation model classifies each pixel in the image frame to construct a pixel-level mask of the display fisheye image, thereby segmenting the display fisheye image. The Sobel algorithm is used to perform boundary detection on the display fisheye image to determine the coordinate information of each boundary point of the display fisheye image, forming the boundary part point information.

[0123] The image segmentation model is pre-trained to convergence, acquiring the ability to determine the display fisheye image and its boundary point information within the display's imaging range in the input image, as well as the anchor box region of the display fisheye image within the frame. The description of the selection of the image segmentation model can be found in the relevant descriptions in the preceding embodiments, and will not be repeated here.

[0124] Furthermore, image segmentation models typically determine the boundary point information of the display's fisheye image. Since the corresponding boundary points cannot completely surround the fisheye image, a linear interpolation algorithm is used to linearly complete the boundary point information, resulting in complete boundary point information that ensures the corresponding boundary points completely surround the fisheye image, presenting a complete boundary of the display's fisheye image. Further, noise reduction processing is applied to the complete boundary point information to obtain the boundary frame point information. Based on the preset subdivision region positioning features, including the positional distribution information of each predetermined point's display predetermined point region within the anchor frame region, each display predetermined point region can be determined within the anchor frame region. Then, based on the corresponding horizontal and vertical coordinate ranges of each display predetermined point region, the coordinate information of each boundary point in the boundary frame point information is traversed to determine the display predetermined point region to which each boundary point belongs. For each display predetermined point region, each boundary point belonging to that region is treated as a single region boundary point, and the coordinate information of these region boundary points is collected to constitute the predetermined region point information of that display predetermined point region. Based on the preset regional fixed-point characteristics and the location information of each predetermined area, the coordinate information of the preset points in each display predetermined point area is determined to form the predetermined point location information. The coordinate information includes the corresponding horizontal and vertical coordinates of the predetermined point in the anchor frame area.

[0125] For every three adjacent predetermined point information, the center point information corresponding to two target predetermined point information in the three predetermined point information is adapted and distorted to obtain the center point information after adaptation and distortion adjustment as the first regional subdivision point information. From all predetermined point information, the second regional subdivision point information matching the preset regional subdivision point features is determined. All second regional subdivision point information and all first regional subdivision point information constitute the regional boundary point information.

[0126] The camera device uses a single fisheye camera, with 9 predetermined points set around the boundary of the monitor's fisheye image. These points correspond to the four corner points of the monitor's bezel (numbered 2, 4, 6, and 8 in Figure 7), the midpoint of the lower long side (numbered 5 in Figure 7), the midpoints of the two wide sides (numbered 3 and 7 in Figure 7), and the two points where the long side of the monitor intersects with the monitor's fisheye image (numbered 1 and 9 in Figure 7).

[0127] The location distribution information includes the corresponding horizontal and vertical coordinate ranges of the predefined display point areas within the anchor frame area. Those skilled in the art can set subdivided area positioning features based on prior knowledge or experimental data, or implement them according to the relevant disclosures in subsequent embodiments. Area positioning features include locating the coordinate distribution pattern of predefined points within each display predefined point area. Those skilled in the art can set area positioning features based on prior knowledge or experimental data, or implement them according to the relevant disclosures in subsequent embodiments.

[0128] Step S1200: Based on the area boundary point information and the preset area point mapping relationship, determine the area point information corresponding to the fisheye image of the display. Based on the area boundary point information and the area point information, subdivide the fisheye image of the display to obtain multiple target subdivided areas.

[0129] The regional point mapping relationship includes the relative coordinate offset relationship between the coordinate information of each point within a region and the coordinate information of at least one corresponding point in the region boundary point information. This allows the coordinate information of points within each region to be determined based on the relative coordinate offset relationships and the coordinate information of at least one corresponding point in the region boundary point information. The relative coordinate offset relationship includes the horizontal and vertical mathematical expressions for the x and y coordinates of the corresponding points within a region relative to the horizontal and vertical coordinates of at least one corresponding point in the region boundary point information. Points within each region belong to the corner points of the corresponding region obtained from the fisheye view of the display, excluding the points corresponding to the region boundary point information. Those skilled in the art can, in advance, obtain a corresponding fisheye image of the display and its regional boundary information by using a camera device according to the aforementioned disclosure. Then, based on the regional boundary information of the fisheye image, they can divide the fisheye image into regions using straight lines to obtain all regions corresponding to the fisheye image, as well as the coordinate information of each corner point in each region. The corner points, excluding any point corresponding to the regional boundary information, are taken as the intra-regional points of the fisheye image. For each intra-regional point, at least one corresponding point in the regional boundary information connected to that intra-regional point is determined. Based on the coordinate information of these points and the intra-regional point's coordinate information, the corresponding horizontal and vertical mathematical expressions can be determined. For example, the horizontal mathematical expression represents the horizontal coordinate of the intra-regional point's coordinate information, obtained by multiplying the horizontal coordinate of at least one point opposite to the intra-regional point by their respective weights and then summing the results. The vertical mathematical expression represents the vertical coordinate of the intra-regional point's coordinate information, obtained by multiplying the vertical coordinate of at least one point opposite to the intra-regional point by their respective weights and then summing the results.

[0130] Based on the boundary point information and the point information within the region, the corresponding points in the fisheye image can be determined. By connecting some or all of these points with straight lines according to a preset set of point-to-point connections, the fisheye image can be subdivided into multiple regions. Further, multiple regions corresponding to the edges including the display bezel are identified from these regions and designated as target subdivision regions. The set of point-to-point connections includes pairs of connected points, which can be preset by those skilled in the art according to their needs.

[0131] Figure 7 illustrates the fisheye image segmentation effect of a display as an example. In Figure 7, numbers 18 to 23 represent points within each region of the fisheye image, and numbers 1 to 17 represent points corresponding to the region boundary information of the fisheye image. Multiple target subdivision regions are as follows: regions formed by corner points (numbers 2, 10, 11, and 18); regions formed by corner points (numbers 10, 1, 18, and 20); regions formed by corner points (numbers 11, 18, 12, and 19); regions formed by corner points (numbers 12, 19, 4, and 13); regions formed by corner points (numbers 2, 10, 11, and 18); and regions formed by corner points (numbers 19, 12, 19, 4, and 13). The regions formed by the corner points are 21, 13, and 5 respectively; the regions formed by the corner points are 21, 5, 14, and 22 respectively; the regions formed by the corner points are 14, 22, 6, and 15 respectively; the regions formed by the corner points are 22, 15, 23, and 16 respectively; the regions formed by the corner points are 23, 17, 8, and 16 respectively; and the regions formed by the corner points are 9, 17, 23, and 20 respectively.

[0132] Step S1300: Apply the preset fisheye correction algorithm to perform image correction on the subdivided images corresponding to multiple target subdivided regions in the fisheye image of the display, and obtain the corresponding corrected subdivided images.

[0133] For each target subdivision region in the fisheye image of the display, the `cv2.getPerspectiveTransform` function in the OpenCV library is called. Using the corner point information corresponding to each corner point of the target subdivision region and the preset region resolution as input parameters, the perspective matrix parameters corresponding to the perspective transformation of the subdivision image of the target subdivision region into a matrix image with the preset region resolution are obtained. The preset region resolution can be set as needed by those skilled in the art.

[0134] Furthermore, the cv2.warpPerspective function in the OpenCV library is called, using the perspective matrix parameters and the subdivided image as input parameters to obtain the matrix image, thus completing the image correction of the subdivided image.

[0135] As can be seen from the above embodiments, the technical solution of this application has many advantages, including but not limited to the following aspects:

[0136] This application segmentes image instances and then uses regular point positioning to define the boundaries of the display fisheye image, thereby obtaining the region boundary point information corresponding to each point surrounding the boundary, ensuring real-time and accurate capture of the display content. Next, based on the point mapping relationship between the boundary points of the display fisheye image and points within the image, corresponding points within the display fisheye image are located. Based on these points, the display fisheye image is divided to obtain multiple target subdivision regions and their subdivision images, and visual distortion in each subdivision image is corrected. This allows for precise, efficient, and fine-grained segmentation of target subdivision regions and their subdivision images within the display fisheye image, ensuring the accuracy and reliability of the image content in the subdivision images.

[0137] In a further embodiment, step S1100, performing boundary point determination on the image instance segmentation result of the display fisheye image according to a preset boundary point determination rule to obtain the region boundary point information corresponding to the display fisheye image, includes the following steps:

[0138] Step S1110: Use a preset image segmentation model to perform image instance segmentation on the fisheye image of the display, and obtain the point information of the boundary part of the fisheye image of the display, as well as the anchor frame area of ​​the fisheye image of the display within the frame.

[0139] The image segmentation model is pre-trained to convergence and learns to determine the display fisheye image and its boundary point information that belong to the display's imaging range in the input image, as well as the anchor frame region of the display fisheye image within the frame.

[0140] The selection of the image segmentation model and the description of obtaining the boundary point information using the image segmentation model can be referred to the relevant descriptions in the foregoing embodiments, and will not be repeated here.

[0141] Step S1120: After linearly completing the boundary partial point information, the corresponding obtained complete boundary point information is subjected to noise reduction processing to obtain the boundary frame point information;

[0142] The description of the point information of the boundary section of the linear completion can be found in the relevant description in the foregoing embodiments, and will not be repeated here.

[0143] Furthermore, the RDP algorithm is used to denoise the complete boundary point information, remove redundant boundary point coordinate information, and retain the coordinate information of multiple necessary boundary points that constitute the boundary of the fisheye image of the display without affecting its true shape, thus obtaining the boundary frame point information.

[0144] Step S1130: Based on the preset subdivided area positioning features and boundary frame point information, determine each display predetermined point area and its predetermined area point information in the anchor frame area.

[0145] The subdivided region positioning features include the positional distribution information of the display predetermined point areas where each predetermined point is located within the anchor frame area. This positional distribution information includes the corresponding horizontal and vertical coordinate ranges of the respective display predetermined point areas within the anchor frame area. Those skilled in the art can set the subdivided region positioning features based on prior knowledge or experimental data, or implement them according to the relevant disclosures in subsequent embodiments.

[0146] The width of the anchor frame region is the difference between the maximum and minimum x-coordinates in the complete boundary point information, and the height of the anchor frame region is the difference between the maximum and minimum y-coordinates in the complete boundary point information.

[0147] In one embodiment, nine predetermined points are set based on the fisheye image of the display, which is a complete image surrounding the display. These points correspond to the four corner points of the display bezel, the midpoint of one lower long side, the midpoints of two wider sides, and the two points where the long side of the display intersects with the fisheye image. The positional distribution information corresponding to the display predetermined point area for each predetermined point in the anchor frame region is preset. Please refer to Figure 8 for the following: {x...} min 1 / 8*x max ], the range of the vertical axis is: [1 / 3*y max y max ]}(P1), {Horizontal coordinate range: [1 / 32*x max 7 / 32*x max ], the range of the ordinate is: [5 / 12*y max ,11 / 12*y max ]}(P2), {Horizontal coordinate range: [1 / 8*x max 1 / 4*x max ], the range of the ordinate is: [2 / 3*y max y max ]}(P3), {Horizontal coordinate range: [3 / 8*x max 5 / 8*x max ], the range of the ordinate is: [5 / 6*y max y max ]}(P4), {Horizontal coordinate range: [5 / 8*x max 3 / 4*x max ], the range of the ordinate is: [5 / 6*y max y max ]}(P5), {Horizontal coordinate range: [21 / 32*x max 31 / 32*x max ], the range of the vertical axis is: [3 / 12*y max ,11 / 12*y max ]}(P6), {Horizontal coordinate range: [7 / 8*x max x max], the range of the vertical axis is: [1 / 6*y max y max ]}(P7), {x range: [x min 3 / 8*x max ], y-axis range: [y min , 1 / 6*y max ]}(P8), {Horizontal coordinate range: [5 / 8*x max x max ], y-axis range: [y min , 1 / 6*y max ]}(P9), which combines all location distribution information to form subdivided regional positioning features. x max x min y min y max These are the minimum x-coordinate, maximum x-coordinate, minimum y-coordinate, and maximum y-coordinate in the complete boundary point information.

[0148] It is not difficult to understand that, based on the location distribution information of each display pre-point area in the subdivided area positioning features, each display pre-point area can be determined in the anchor frame area. Then, based on the corresponding horizontal and vertical coordinate ranges of each display pre-point area, the coordinate information of each boundary point in the boundary frame point information is traversed to determine the display pre-point area to which each boundary point belongs. Then, for each display pre-point area, all boundary points belonging to that display pre-point area are taken as individual area boundary points, and the coordinate information of these area boundary points is collected to form the pre-point area point information of that display pre-point area.

[0149] Step S1140: Based on the preset regional fixed-point characteristics and the location information of each predetermined regional point, determine the predetermined point information corresponding to each display predetermined point region;

[0150] The regional positioning features include locating the coordinate distribution pattern of predetermined points in each display predetermined point area. Those skilled in the art can set the regional positioning features based on prior knowledge or experimental data, or implement them according to the relevant disclosures in the following embodiments.

[0151] For {x range: [x min 3 / 8*x max ], y-axis range: [y min , 1 / 6*y maxThe display predefined point area is determined by identifying the minimum ordinate in the predefined area point information, and then determining the coordinates of the corresponding boundary points. From these boundary point coordinates, the coordinates of the boundary point with the minimum abscissa are selected as the coordinates of the predefined points in the display predefined point area, thus forming the predefined point information. For the abscissa range: [5 / 8*x...] max x max ], y-axis range: [y min , 1 / 6*y max The display predefined point area is determined by identifying the minimum ordinate in the predefined area point information, and then determining the coordinates of the corresponding boundary points. From these boundary point coordinates, the boundary point with the largest x-coordinate is selected as the predefined point coordinate information for the display predefined point area, thus forming the predefined point information. For the x-coordinate range: [3 / 8*x...] max 5 / 8*x max ], the range of the ordinate is: [5 / 6*y max y max The display predefined point area is determined by defining the area outline information of the display predefined point area. The horizontal coordinate is the value obtained by multiplying the width of the anchor frame area by a first preset ratio and adding the minimum horizontal coordinate in the complete boundary point information. The vertical coordinate is the area boundary point corresponding to the maximum vertical coordinate in the predefined area point information. The coordinate information of the corresponding area boundary point is used as the coordinate information of the predefined point of the display predefined point area, which constitutes the predefined point information. For the horizontal coordinate range: [x min 1 / 8*x max ], the range of the vertical axis is: [1 / 3*y max y max In one embodiment, the coordinates of the boundary point with the smallest abscissa in the region contour information of the display pre-point area are determined, or the coordinates of the boundary point with the largest ordinate among N boundary points with smaller abscissas in the region contour information are determined, and these coordinates are used as the coordinates of the pre-points of the display pre-point area, constituting the pre-point location information. N can be set as needed by those skilled in the art, for example, 5; for {abscissa range: [7 / 8*x} max x max ], the range of the vertical axis is: [1 / 6*y max y maxIn one embodiment, the coordinates of the boundary point with the largest abscissa in the region contour information of the display pre-defined point area are determined, or the coordinates of the boundary point with the largest ordinate among the N boundary points with large abscissas in the region contour information are determined, and these coordinates are used as the coordinates of the pre-defined point of the display pre-defined point area, constituting the pre-defined point information; for {abscissa range: [1 / 8*x} max 1 / 4*x max ], the range of the ordinate is: [2 / 3*y max y max The display of the predetermined point area, {x-coordinate range: [5 / 8*x]} max 3 / 4*x max ], the range of the ordinate is: [5 / 6*y max y max The display pre-defined point area is first sorted clockwise around the fisheye view boundary of the display, and the coordinates of each boundary point in the boundary frame point information are sorted to obtain a boundary point sequence. From the boundary point sequence, the vector formed by each boundary point and its next boundary point is determined sequentially. Every two vectors are adjacent in position. The inverse cosine value between each two vectors is calculated, which is the angle between the two vectors. Then, for each display pre-defined point area, the coordinates of the intersection point between the two adjacent vectors with the largest angle are determined from the pre-defined area point information of that display pre-defined point area, and this is used as the pre-defined point information of that display pre-defined point area; for {x-coordinate range: [1 / 32*x max 7 / 32*x max ], the range of the ordinate is: [5 / 12*y max ,11 / 12*y max The display of the predetermined point area, {x-coordinate range: [21 / 32*x]} max 31 / 32*x max ], the range of the vertical axis is: [3 / 12*y max ,11 / 12*y max For each display predefined point area, the coordinate information of the boundary point of the area that is equidistant from the two predefined points adjacent to the display predefined point area is determined, and used as the coordinate information of the predefined point in the display predefined point area.

[0152] Step S1150: For every three adjacent predetermined point information, perform adaptation distortion adjustment on the center point information corresponding to the two target predetermined point information in the three predetermined point information, and obtain the center point information after adaptation distortion adjustment as the first area subdivision point information.

[0153] The predetermined point whose position is in the middle of the three predetermined point information is taken as a single target predetermined point information. Then, the x-coordinate or y-coordinate of the corresponding predetermined point is determined from these three predetermined point information. The predetermined point information with a position smaller than the x-coordinate or y-coordinate of the predetermined point in the target predetermined point information is taken as another target predetermined point information. The coordinate information of the midpoint between the two predetermined points corresponding to the two target predetermined point information is obtained as the center point information. Then, the horizontal distortion offset value and vertical distortion offset value of the midpoint corresponding to the two predetermined points are obtained from the preset distortion adjustment parameter set. The horizontal distortion offset value and vertical distortion offset value are added to the x-coordinate and y-coordinate of the center point information to complete the adaptation distortion adjustment of the center point information. The resulting adapted distortion-adjusted center point information is taken as the first region subdivision point information.

[0154] The distortion adjustment parameter set includes the lateral distortion offset value and the longitudinal distortion offset value of the midpoint between two predetermined points corresponding to the position information of each pair of adjacent predetermined points, which are used to adapt to the distortion of fisheye imaging and make corresponding adjustments. Those skilled in the art can flexibly and adapt the distortion adjustment parameter set based on the disclosure herein.

[0155] Step S1160: From all the predetermined point information, determine the second regional subdivision point information that matches the preset regional subdivision point characteristics, and use all the second regional subdivision point information and all the first regional subdivision point information to form the regional boundary point information.

[0156] The region subdivision point features include the features of points on the display bezel corresponding to each predetermined point required for dividing the display fisheye image. Those skilled in the art can flexibly and adapt the region subdivision point features according to their own needs for dividing the display fisheye image. In one embodiment, the region subdivision point features include the four corner points (labeled 2, 4, 6, and 8 in Figure 7) and the midpoint of the lower long side (labeled 5 in Figure 7) corresponding to each predetermined point required for dividing the display fisheye image, and the two points where the long side of the display intersects with the display fisheye image (labeled 1 and 9 in Figure 7). Therefore, from all predetermined point information, each predetermined point information matching the region subdivision point features is used as the second region subdivision point information, and then all the second region subdivision point information and all the first region subdivision point information are combined to form the region boundary point information.

[0157] In this embodiment, the boundary point information of the region corresponding to the boundary of the fisheye image of the display is disclosed. Based on the boundary points of the complete boundary of the fisheye image of the display, the accuracy and reliability of the coordinate information of each predetermined point from each predetermined point region, as well as the intermediate point information after adaptation distortion adjustment between each pair of adjacent predetermined point information, can be ensured.

[0158] In a further embodiment, step S1140, determining the predetermined point information corresponding to each predetermined display point area based on preset regional fixed-point features and each predetermined regional point information, includes any one or more of the following steps:

[0159] Step S1141: For each display predetermined point area that includes a predetermined point belonging to the intersection of the long side of the display and the fisheye diagram of the display, or belonging to the midpoint of the long side of the display, determine the predetermined point information of the display predetermined point area from the predetermined area point information of the display predetermined point area according to the position of the display predetermined point area in the anchor frame area.

[0160] Two display predetermined point regions, including the predetermined point belonging to the intersection of the long side of the display and the fisheye view of the display, are defined as follows: {Horizontal coordinate range: [x] min 3 / 8*x max ], y-axis range: [y min , 1 / 6*y max The display predefined point area is determined by identifying the minimum ordinate in the predefined area point information, and then determining the coordinates of the corresponding boundary points. From these boundary point coordinates, the coordinates of the boundary point with the minimum abscissa are selected as the coordinates of the predefined points in the display predefined point area, thus forming the predefined point information; {abscissa range: [5 / 8*x} max x max ], y-axis range: [y min , 1 / 6*y max The display of the predetermined point area is determined by identifying the minimum ordinate in the predetermined area point information and the corresponding coordinates of each boundary point. Then, the coordinates of the boundary point with the minimum abscissa in these boundary points are determined as the coordinates of the predetermined point in the display of the predetermined point area, thus forming the predetermined point information.

[0161] The display predetermined point area, including the predetermined point belonging to the midpoint of the long side of the display, has a horizontal coordinate range of [3 / 8*x]. max 5 / 8*x max ], the range of the ordinate is: [5 / 6*y max y max The region outline information of the display pre-defined point area is determined by multiplying the width of the anchor frame area by a first preset ratio and adding the minimum horizontal coordinate in the boundary complete point information, and the vertical coordinate is the region boundary point corresponding to the maximum vertical coordinate in the pre-defined region point information. The coordinate information of the corresponding region boundary point is used as the coordinate information of the pre-defined point of the display pre-defined point area, thus constituting the pre-defined point information.

[0162] Step S1142: For each display predetermined point area containing predetermined points belonging to the corner points of the display, determine the vector formed by each two adjacent boundary points and the angle between each two adjacent vectors according to the boundary frame point information. From the predetermined area point information of the display predetermined point area, determine the coordinate information of the intersection point between the two adjacent vectors with the largest angle, and use it as the predetermined point information of the display predetermined point area.

[0163] The display predefined point regions, which include the predefined points belonging to the corner points of the display, are as follows: [x min 1 / 8*x max ], the range of the vertical axis is: [1 / 3*y max y max ]}, {Horizontal coordinate range: [1 / 8*x max 1 / 4*x max ], the range of the ordinate is: [2 / 3*y max y max ]}、{Horizontal coordinate range: [5 / 8*x max 3 / 4*x max ], the range of the ordinate is: [5 / 6*y max y max ]}、{Horizontal coordinate range: [7 / 8*x max x max ], the range of the vertical axis is: [1 / 6*y max y max First, following a clockwise rotation around the fisheye view boundary of the display, the coordinates of each boundary point in the boundary frame point information are sorted to obtain a boundary point sequence. From this sequence, the vector formed by each boundary point and its next boundary point is determined sequentially. Each pair of vectors is then adjacent in position. The inverse cosine of each pair of vectors is calculated, representing the angle between them. Then, for each predetermined display point area, the coordinates of the intersection point between the two adjacent vectors with the largest angle are determined from the predetermined area point information of that area, serving as the predetermined point information for that display point area.

[0164] Step S1143: For each display predetermined point area containing a predetermined point belonging to the midpoint of the wide edge of the display, based on the position information of two predetermined points adjacent to the display predetermined point area, determine the coordinate information of the boundary point of the area whose distance ratio to the two predetermined point information satisfies a preset condition from the predetermined area position information of the display predetermined point area, and use it as the predetermined point information of the display predetermined point area.

[0165] The display predefined point regions, including the predefined point belonging to the midpoint of the wide side of the display, are as follows: {Horizontal coordinate range: [1 / 32*x max 7 / 32*x max ], the range of the ordinate is: [5 / 12*y max ,11 / 12*y max ]}、{Horizontal coordinate range: [21 / 32*x max 31 / 32*x max ], the range of the vertical axis is: [3 / 12*y max ,11 / 12*y max For each display predefined point area, the coordinate information of the boundary point of the area that is equidistant from the predefined area point information of the display predefined point area is determined, and used as the coordinate information of the color sampling point in the color sampling point area.

[0166] This embodiment discloses the process of determining the predetermined point information corresponding to each predetermined point area of ​​the display, ensuring that each predetermined point corresponds to the four corner points, one lower long side midpoint, and two wide side midpoints of the display frame, as well as the two points where the long side of the display intersects with the fisheye view of the display.

[0167] In a further embodiment, step S1150, for every three adjacent predetermined point information, performs adaptation distortion adjustment on the center point information corresponding to two target predetermined point information in the three predetermined point information, and obtains the adapted distortion adjusted center point information as the first region subdivision point information, includes the following steps:

[0168] Step S1151: For every three adjacent preset point information, determine the distortion type and distortion ratio corresponding to the distortion imaging effect of the three preset point information;

[0169] The predetermined point information whose position is in the middle of the three predetermined point information is taken as the target predetermined point information. The sum of the ordinates of the remaining two predetermined point information is calculated and then divided by two to obtain the mean of the ordinates. If the ordinate of the target predetermined point information is less than the mean of the ordinates, the distortion type corresponding to the distortion imaging effect of the three predetermined point information is determined to be concave; if the ordinate of the target predetermined point information is greater than the mean of the ordinates, the distortion type corresponding to the distortion imaging effect of the three predetermined point information is determined to be convex.

[0170] Based on the remaining two predetermined point information, the slope and slope intercept of the straight line connecting the two corresponding predetermined points can be calculated, thus deriving the mathematical expression of the straight line. Applying the point-to-line distance formula, based on the target predetermined point information and the mathematical expression of the straight line, the distance between the corresponding predetermined point and the straight line can be calculated.

[0171] Using Euclidean algorithm, the distance between the two corresponding predetermined points is calculated based on the remaining two predetermined point information. The distance from the predetermined point corresponding to the target predetermined point information to the straight line is divided by the distance between the two corresponding predetermined points to obtain the distortion ratio corresponding to the distortion imaging effect of the three predetermined point information.

[0172] Step S1152: Based on the two target preset point information from the three preset point information, determine the corresponding center point information, the distance between the two points, and the slope;

[0173] Furthermore, from the three predetermined point information, the x-coordinate or y-coordinate of the corresponding predetermined point is determined. If the x-coordinate or y-coordinate of the predetermined point is smaller than that of the predetermined point in the target predetermined point information, the corresponding predetermined point information is used as another target predetermined point information.

[0174] Based on the two target preset point information, the coordinates of the midpoint between the two corresponding preset points are calculated as the center point information. The slope of the straight line connecting the two preset points is also calculated, as well as the distance between the two preset points, which is the spacing between the two points.

[0175] Step S1153: Determine the point offset information corresponding to the center point information based on the distortion type, distortion ratio, distance between two points, and slope.

[0176] Calculate the arctangent corresponding to the absolute value of the slope to obtain the slope angle. Calculate the distance between the two points and multiply it by the distortion ratio to obtain the distortion distance. Calculate the sine value corresponding to the slope angle and multiply it by the distortion distance to obtain the horizontal axis offset. Calculate the cosine value corresponding to the slope angle and multiply it by the distortion distance to obtain the vertical axis offset.

[0177] If the distortion type is convex and the slope is greater than zero, the increment value corresponding to the x-coordinate in the center point information is determined as the x-axis offset, and the increment value corresponding to the y-coordinate in the center point information is determined as the y-axis offset. The increment values ​​corresponding to the x and y coordinates are used as the point offset information.

[0178] If the distortion type is convex and the slope is less than zero, the increment value corresponding to the horizontal coordinate in the center point information is determined to be the negative value of the horizontal axis offset, and the increment value corresponding to the vertical coordinate in the center point information is determined to be the vertical axis offset. The increment values ​​corresponding to the horizontal and vertical coordinates are used as the point offset information.

[0179] If the distortion type is concave and the slope is greater than zero, determine that the increment value corresponding to the horizontal coordinate in the center point information is the negative value of the horizontal axis offset, and the increment value corresponding to the vertical coordinate in the center point information is the negative value of the vertical axis offset. Use the increment values ​​corresponding to the horizontal and vertical coordinates as the point offset information.

[0180] If the distortion type is concave and the slope is less than zero, the increment value corresponding to the horizontal coordinate in the center point information is determined to be the horizontal axis offset, and the increment value corresponding to the vertical coordinate in the center point information is the negative value of the vertical axis offset. The increment values ​​corresponding to the horizontal and vertical coordinates are used as the point offset information.

[0181] Step S1154: Adjust the center point information according to the point offset information, and obtain the adjusted center point information as the first area subdivision point information.

[0182] Add the corresponding incremental values ​​from the point offset information to the horizontal and vertical coordinates of the center point information. Use the resulting horizontal and vertical coordinates to form the center point information, which is the adjusted center point information, as the first area subdivision point information.

[0183] In this embodiment, it is disclosed that for every three adjacent preset point information, the center point information corresponding to two target preset point information in the three preset point information is adjusted according to the distortion imaging effect of the three preset point information, and used as the first area subdivision point information to ensure the accuracy and reliability of the first area subdivision point information.

[0184] In a further embodiment, step S1300, applying a preset fisheye correction algorithm to perform image correction on the subdivided images corresponding to multiple target subdivided regions in the fisheye image of the display, to obtain the corresponding corrected subdivided images, includes the following steps:

[0185] Step S1310: For each target subdivision region corresponding to the subdivision image in the fisheye image of the display, determine the perspective matrix parameters corresponding to the matrix image of the target subdivision region that has been transformed into a matrix image with a preset region resolution based on the corner point information of each corner point of the target subdivision region.

[0186] For each target subdivision region in the fisheye image of the display, the `cv2.getPerspectiveTransform` function in the OpenCV library is called. Using the corner point information corresponding to each corner point of the target subdivision region and the preset region resolution as input parameters, the perspective matrix parameters corresponding to the perspective transformation of the subdivision image of the target subdivision region into a matrix image with the preset region resolution are obtained. The preset region resolution can be set as needed by those skilled in the art.

[0187] Step S1320: Perform perspective transformation correction on the subdivided image according to the perspective matrix parameters to obtain a matrix image, which is used as the image after image correction and subdivision.

[0188] The OpenCV library's cv2.warpPerspective function is called, taking the perspective matrix parameters and the subdivided image of the target subdivision region as input parameters to obtain the matrix image and complete the image correction of the subdivision image.

[0189] This embodiment reveals the process of image correction for the subdivided image of the target subdivision region, which can correct the subdivided image very efficiently.

[0190] To address the aforementioned technical problems, this application also provides a computer device. Figure 9 shows a schematic diagram of the internal structure of the computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. The computer-readable storage medium stores an operating system, a database, and computer-readable instructions. The database may store a sequence of control information. When the computer-readable instructions are executed by the processor, the processor can implement an ambient light control method. The processor of the computer device provides computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor can execute the ambient light control method of this application. The network interface of the computer device is used for communication with a terminal. Those skilled in the art will understand that the structure shown in Figure 9 is merely a block diagram of a portion of the structure related to the solution of this application and does not constitute a limitation on the computer device to which the solution of this application is applied. A specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0191] This application also provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the ambient light control method of any embodiment of this application.

[0192] This application also provides a computer program product including a computer program / instructions, which, when executed by a processor, implements the steps of the ambient light control method of any embodiment of this application.

Claims

1. An ambient lighting control method, comprising: The image frames in the video stream generated by the camera device capturing the display are acquired, and the display area belonging to the imaging range of the display in the image frame is determined, as well as the color point information of each color point set around the display area. The display area is divided into multiple subdivided areas based on the color sampling point information, and the multiple subdivided areas arranged along the circumference of the display area are used as the color sampling areas for the lighting effect. Color samples are taken from each lighting effect color sampling area to determine the target color corresponding to each lighting effect color sampling area; and A lighting effect control signal is constructed based on the target color corresponding to each lighting effect color sampling area, and the lighting effect control signal is output to the ambient lighting device to control the ambient lighting device to emit light according to the lighting effect control signal.

2. The ambient light control method of claim 1, wherein, Color sampling is performed on each lighting effect color sampling area to determine the target color corresponding to each lighting effect color sampling area, including the following steps: For each light effect color sampling area, every other pixel is sampled for that light effect color sampling area, and the average color is determined based on the total number of sampled pixels and the color of the pixels. The average color is enhanced to obtain the target color of the light effect color sampling area.

3. The ambient light control method of claim 1, wherein, Determining the display area within the image frame that belongs to the imaging range of the display, and the color point information of each color sampling point defined around the display area, includes the following steps: The image frame is subjected to target detection and instance segmentation using a preset image segmentation model to determine the display area in the image frame and the point information of the boundary part to be completed, as well as the anchor frame area of ​​the display area in the frame; After linearly completing the point information of the boundary part to be completed, the corresponding boundary contour point information is subjected to noise reduction processing to obtain the boundary frame point information; Based on the preset regional positioning features and the boundary frame point information, the color sampling point regions and their regional outline point information in the anchor frame region are determined. Based on the preset regional fixed-point features and the contour point information of each region, the coordinate information of the color sampling points in each color sampling point region is determined to form the color sampling point information.

4. The ambiance light control method of claim 3, wherein, Based on the preset regional fixed-point features and the contour point information of each region, the coordinate information of the color sampling points in each color sampling point region is determined to form the color sampling point information, including: For each color sampling point region that includes a color sampling point belonging to the midpoint of the long side of the display, the coordinate information of the color sampling point in the color sampling point region is determined from the region outline point information of the color sampling point region based on the position of the color sampling point region in the anchor frame region. For each color sampling point region that includes color sampling points belonging to the corner points of the display, the vector formed by each two adjacent boundary points and the angle between each two adjacent vectors are determined based on the boundary frame point information. From the region boundary point position information of the color sampling point region, the coordinate information of the intersection point between the two adjacent vectors with the largest angle is determined as the color sampling point position information of the color sampling point region. For each color sampling point region that includes a color sampling point belonging to the midpoint of the wide edge of the display, based on the coordinate information of two color sampling points adjacent to the color sampling point region, the coordinate information of the region contour points whose distance ratio to the two color sampling points satisfies a preset condition is determined from the region contour point information of the color sampling point region, and these coordinates are used as the coordinate information of the color sampling points in the color sampling point region.

5. The ambient light control method of claim 1, wherein, Color sampling is performed on each lighting effect color sampling area to determine the target color corresponding to each lighting effect color sampling area, including the following steps: For each light effect color sampling area, determine the core hue range of that light effect color sampling area and the colors of multiple pixels whose corresponding hue values ​​meet the preset conditions, and calculate the corresponding average color. The average color is enhanced to obtain the target color of the light effect color sampling area.

6. The ambiance light control method of claim 4, wherein, For each color sampling point region containing a color sampling point belonging to the midpoint of the wide edge of the display, based on the coordinate information of two color sampling points adjacent to the color sampling point region, the coordinate information of the region contour points whose distance ratio to the two color sampling points satisfies a preset condition is determined from the region contour point information of the color sampling point region, and used as the coordinate information of the color sampling points in the color sampling point region, including: When the image frame belongs to the single-eye shooting type, the coordinate information of the region contour point with the same distance between two adjacent color points is determined as the coordinate information of the color point in the color point region; When the image frame belongs to the type of shooting from the left side of both eyes, the coordinate information of the contour point of the region where the ratio of the distance between two adjacent color points in the color sampling area belongs to the first preset value range is determined, and used as the coordinate information of the color sampling point in the color sampling area; When the image frame belongs to the binocular right-side shooting type, the coordinate information of the region contour point whose ratio of the distance between two adjacent color sampling points is within the second preset value range is determined, and used as the coordinate information of the color sampling point in the color sampling point region.

7. The ambient light control method of claim 3, wherein, Before performing target detection and instance segmentation on the image frame using a preset image segmentation model to determine the display area and its boundary point information to be filled in, as well as the anchor frame area of ​​the display area within the frame, the method includes: Obtain a preset training set and train the image segmentation model to convergence, enabling it to learn the ability to determine the display area belonging to the imaging range of the display and the point information of its boundary parts, as well as the anchor box area of ​​the display area within the frame, from the input image. The training set includes multiple training samples and their supervision labels. The training samples are images containing display areas, and the supervision labels label the target boundary detection information corresponding to the display areas in the corresponding training samples. The training samples in the training set are obtained by taking pictures of the display using a single fisheye camera, and / or by taking pictures of the display using the left fisheye camera of a dual fisheye camera and the right fisheye camera of a dual fisheye camera respectively.

8. The ambient light control method of claim 1, wherein, Using multiple refined areas arranged circumferentially along the display area as color sampling areas for lighting effects includes the following steps: The fisheye correction algorithm is used to correct the image in each subdivided region; For each corrected subdivided region arranged circumferentially along the display area, the central part of the subdivided region is cropped out as the color sampling area for the lighting effect.

9. The ambiance light control method of claim 8, wherein, The fisheye correction algorithm is used to correct the image in each subdivided region, including the following steps: Obtain the fisheye image of the monitor, and perform boundary point determination on the image instance segmentation result of the fisheye image of the monitor according to the preset boundary point determination rules to obtain the boundary point information of the corresponding region of the fisheye image of the monitor. Based on the area boundary point information and the preset area point mapping relationship, the area point information corresponding to the fisheye image of the display is determined. Based on the area boundary point information and the area point information, the fisheye image of the display is subdivided into multiple target subdivided areas. By applying a preset fisheye correction algorithm, image correction is performed on the subdivided images corresponding to multiple target subdivision regions in the fisheye image of the display, resulting in the corresponding corrected subdivision images.

10. The ambiance light control method of claim 9, wherein, Based on preset boundary point determination rules, the image instance segmentation results of the monitor fisheye image are subjected to boundary point determination to obtain the corresponding region boundary point information of the monitor fisheye image, including the following steps: The display fisheye image is segmented using a preset image segmentation model to obtain the boundary point information of the display fisheye image and the anchor frame area of ​​the display fisheye image within the frame. After linearly completing the boundary partial point information, the corresponding obtained complete boundary point information is subjected to noise reduction processing to obtain the boundary frame point information; Based on the preset subdivided area positioning features and the boundary frame point information, each display predetermined point area and its predetermined area point information in the anchor frame area are determined. Based on the preset regional fixed-point characteristics and the location information of each predetermined regional point, the corresponding predetermined point information of each display predetermined point region is determined; For every three adjacent predetermined point information, the center point information corresponding to two target predetermined point information in the three predetermined point information is adapted and distorted to obtain the center point information after adaptation and distortion adjustment as the first area subdivision point information. From all the predetermined point information, the second regional subdivision point information that matches the preset regional subdivision point characteristics is determined, and the regional boundary point information is composed of all the second regional subdivision point information and all the first regional subdivision point information.

11. The ambilight control method according to claim 10, wherein, Based on the preset regional fixed-point characteristics and the location information of each predetermined region, the predetermined point location information corresponding to each display predetermined point region is determined, including: For each predetermined display point region that includes a predetermined point that is the intersection of the long side of the display and the fisheye view of the display, or the midpoint of the long side of the display, the predetermined point information of the predetermined display point region is determined from the predetermined region point information of the predetermined display point region based on the position of the predetermined display point region in the anchor frame region. For each display pre-defined point area containing pre-defined points belonging to the corner points of the display, the vector formed by each two adjacent boundary points and the angle between each two adjacent vectors are determined according to the boundary frame point information. From the pre-defined area point information of the display pre-defined point area, the coordinate information of the intersection point between the two adjacent vectors with the largest angle is determined as the pre-defined point information of the display pre-defined point area. For each display predetermined point area that includes a predetermined point belonging to the midpoint of the wide edge of the display, based on the position information of two predetermined points adjacent to the display predetermined point area, the coordinate information of the boundary point of the area whose distance ratio to the two predetermined point information satisfies a preset condition is determined from the predetermined area position information of the display predetermined point area, and this coordinate information is used as the predetermined point information of the display predetermined point area.

12. The ambient light control method of claim 10, wherein, For every three adjacent predetermined point information, the center point information corresponding to two target predetermined point information among the three predetermined point information is adapted and distorted to obtain the adapted and distorted center point information as the first region subdivision point information, including the following steps: For every three adjacent preset point information, determine the distortion type and distortion ratio corresponding to the distortion imaging effect of the three preset point information; Based on the two target preset point information among the three preset point information, the corresponding center point information, the distance between the two points and the slope are determined; Based on the distortion type, distortion ratio, distance between two points, and slope, the point offset information corresponding to the center point information is determined. The center point information is adjusted based on the point offset information to obtain the adjusted center point information as the first area subdivision point information.

13. The ambient light control method of claim 9, wherein, The image correction is performed on the subdivided images corresponding to multiple target subdivided regions in the fisheye image of the display using a preset fisheye correction algorithm, to obtain the corresponding corrected subdivided images, including the following steps: For each target subdivision region in the fisheye image of the display, the perspective matrix parameters corresponding to the perspective transformation of the subdivision image of the target subdivision region into a matrix image with a preset region resolution are determined based on the corner point information corresponding to each corner point of the target subdivision region. The perspective transformation of the subdivided image is performed according to the perspective matrix parameters to obtain the matrix image, which is used as the image after image correction of the subdivided image.

14. A computer device comprising a central processor and a memory, wherein, The central processing unit is used to invoke and run a computer program stored in the memory to perform the steps of the method as described in any one of claims 1 to 13.

15. A computer program product, wherein, The computer program product includes a computer program that, when executed by a processor, implements the steps of the method as described in any one of claims 1 to 13.

Images