Edge extraction method, system and device of display device, electronic equipment and medium

Abstract

The application discloses an edge extraction method, system and device of a display device, electronic equipment and a medium. The edge extraction method comprises the following steps: acquiring a picture image collected by an image collection device, wherein the display device is located in the image collection range of the image collection device; determining the display edge of the display device from the picture image according to the relative position information of the image collection device and the display device, the size information of the display device and the parameter information of the image collection device, wherein the display edge is the corresponding position of the edge of the display device in the picture image. Therefore, the edge of the display device can be automatically identified from the picture image according to the relative position information of the image collection device and the display device, the size information of the display device and the parameter information of the image collection device, without manual operation of the user, so that the error of edge extraction is reduced, the efficiency of edge extraction is improved, and the use experience of the user is improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to an edge extraction method, system, apparatus, electronic device, and medium for a display device. Background Technology

[0002] In related technologies, an image acquisition device acquires a screen image from a display device, and the screen image is displayed on the user terminal's interactive interface. In order to obtain the edges of the display device in the screen image, the user usually needs to participate in a calibration operation, and then further extract the edges of the display device from the screen image based on the user's calibration operation.

[0003] However, this method requires users to perform manual calibration, which results in large errors and low efficiency, affecting the user experience. Summary of the Invention

[0004] In view of the above problems, the present invention provides an edge extraction method, system, apparatus, device and medium for display devices to improve the above problems.

[0005] In a first aspect, embodiments of this application provide an edge extraction method for a display device. The edge extraction method includes: acquiring a screen image captured by an image acquisition device; wherein the display device is located within the image acquisition range of the image acquisition device. The display edge of the display device is determined from the screen image based on the relative position information between the image acquisition device and the display device, the size information of the display device, and the parameter information of the image acquisition device. The display edge is the position corresponding to the edge of the display device in the screen image.

[0006] Secondly, embodiments of this application provide an edge extraction system for a display device, comprising an image acquisition device and a controller. The image acquisition device is used to acquire a screen image, wherein the display device is located within the image acquisition range of the image acquisition device. The controller is used to acquire the screen image and is also used to determine the display edge of the display device from the screen image based on the relative position information between the image acquisition device and the display device, the size information of the display device, and the parameter information of the image acquisition device. The display edge is the position corresponding to the edge of the display device in the screen image.

[0007] Thirdly, embodiments of this application also provide an edge extraction device for a display device, comprising: an image acquisition module and an edge acquisition module. The image acquisition module is used to acquire a screen image captured by an image acquisition device. The display device is located within the image acquisition range of the image acquisition device. The edge extraction module is used to determine the display edge of the display device from the screen image based on the relative position information between the image acquisition device and the display device, the size information of the display device, and the parameter information of the image acquisition device. The display edge is the position of the edge of the display device in the screen image.

[0008] Fourthly, embodiments of this application also provide an electronic device, which includes one or more processors, a memory, and one or more application programs. The one or more application programs are stored in the memory and configured to be executed by the one or more processors, and are configured to perform the method described in the first aspect above.

[0009] Fifthly, embodiments of this application also provide a computer-readable storage medium storing program code that can be invoked by a processor to execute the method described in the first aspect.

[0010] The technical solution provided by this invention acquires a screen image captured by an image acquisition device, wherein a display device is located within the image acquisition range of the image acquisition device. Based on the relative position information between the image acquisition device and the display device, the size information of the display device, and the parameter information of the image acquisition device, the display edge of the display device is determined from the screen image. The display edge is the position of the edge of the display device in the screen image. Therefore, the edge of the display device can be automatically identified from the screen image based on the relative position information between the image acquisition device and the display device, the size information of the display device, and the parameter information of the image acquisition device, without requiring manual operation by the user. This reduces edge extraction errors, improves edge extraction efficiency, and enhances the user experience. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments and drawings obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0012] Figure 1 This paper shows a structural block diagram of an edge extraction system for a display device that displays the first surface of a display device according to an embodiment of this application.

[0013] Figure 2 A structural block diagram of an edge extraction system for a display device, which displays the second side of a display device according to an embodiment of this application, is shown.

[0014] Figure 3 A schematic flowchart of an edge extraction method for a display device according to an embodiment of this application is shown.

[0015] Figure 4 A schematic flowchart of step S120, which determines the display edge of the display device, is shown in one embodiment of this application.

[0016] Figure 5 This illustration shows a schematic diagram of a display device acquiring a screen image according to an embodiment of the present application.

[0017] Figure 6 This illustration shows another schematic diagram of a display device acquiring a screen image in one embodiment of this application.

[0018] Figure 7 This illustration shows a flowchart of step S220, which involves determining the display edge of the display device based on the vertex image position, in one embodiment of this application.

[0019] Figure 8 The diagram illustrates a flowchart of step S223 of obtaining the display edge based on the second edge connection in one embodiment of this application.

[0020] Figure 9 A schematic flowchart of the edge extraction method of the display device after step S120 in one embodiment of this application is shown.

[0021] Figure 10 This illustration shows another schematic diagram of a display device acquiring a screen image in one embodiment of this application.

[0022] Figure 11 The present application provides a schematic diagram of the process of step S330, which controls the lighting effect according to the target image data, in one embodiment of the present application.

[0023] Figure 12 This document illustrates a flowchart of steps S411 to S414 in one embodiment of the present application for determining the target color value with the highest percentage.

[0024] Figure 13 A structural block diagram of a lighting effect control device for a lighting equipment according to an embodiment of this application is shown.

[0025] Figure 14 A structural block diagram of an electronic device according to an embodiment of this application is shown.

[0026] Figure 15 A structural block diagram of a computer-readable storage medium according to an embodiment of this application is shown. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0028] In related technologies, an image acquisition device acquires images of a display device, which are then displayed on the user terminal's interactive interface. To obtain the edges of the display device in the image, the user typically needs to perform a calibration operation. This involves the user moving multiple calibration points on the terminal's interactive interface to the vertex positions of the display device in the image. Based on the user's calibration operation, the area where the display device is located is determined, and then the edges of the display device are further extracted from the image.

[0029] However, this method requires users to perform manual calibration, which results in large errors and low efficiency, affecting the user experience.

[0030] To address the aforementioned issues, the inventors of this application propose an edge extraction method, system, apparatus, and medium for display devices. This method involves acquiring an image captured by an image acquisition device, with the display device located within the image acquisition range of the image acquisition device. Based on the relative position information between the image acquisition device and the display device, the size information of the display device, and the parameter information of the image acquisition device, the display edge of the display device is determined from the image. The display edge is the position of the edge of the display device within the image. This allows for automatic identification of the display device's edge from the image based on the relative position information between the image acquisition device and the display device, the size information of the display device, and the parameter information of the image acquisition device. This eliminates the need for manual user operation, reduces edge extraction errors, improves edge extraction efficiency, and enhances the user experience.

[0031] It should be noted that the "and / or" mentioned in this article describes the relationship between related objects, indicating that there can be three kinds of relationships. For example, A and / or B can represent three cases: A exists alone, A and B exist simultaneously, and B exists alone.

[0032] The application environment of the lighting effect control system of the lighting equipment provided in the embodiments of the present invention will be described below.

[0033] Please see Figure 1 , Figure 1 An edge extraction system for a display device is shown in an embodiment of this application. The edge extraction system 100 includes an image acquisition device 110 and a controller 120.

[0034] In the embodiments of this application, the image acquisition device 110 is used to acquire screen images. The image acquisition device 110 can be, for example, a camera or other device capable of image acquisition. The image acquisition device 110 can be installed on the edge of the display device. The display device 200 is located within the acquisition range of the image acquisition device 110.

[0035] In some embodiments, the image acquisition device 110 may include an image acquisition unit 111, an image mounting unit 112, and a connecting unit 113, wherein the connecting unit 113 connects the image mounting unit 112 and the image acquisition unit 111. The image acquisition unit 111 includes an acquisition lens for acquiring images. The image mounting unit 113 is used to connect to the display device 200.

[0036] To acquire an image from the display device, the image acquisition unit 111 can be positioned facing the display device 200. In some embodiments, after the image acquisition unit 110 is mounted on the display device 200, the acquisition range of the image acquisition unit 111 can be adjusted. Optionally, the orientation of the image acquisition unit 111 can be adjusted. Specifically, the image acquisition unit 111 can be rotated 360° around the connecting part 113 as a rotation axis, thereby adjusting the orientation of the image acquisition unit 111. Optionally, the distance between the image acquisition unit 111 and the display device 200 can be adjusted. Specifically, the image acquisition unit 111 can be adjusted to move closer to or further away from the display device 200 around the image mounting part 112 as a fulcrum, thereby adjusting the distance between the image acquisition unit 111 and the display device 200.

[0037] In some implementations, the user can manually adjust the installation position, orientation, and distance from the display device 200 of the image acquisition device 110.

[0038] In some embodiments, the image acquisition device 110 may also be adjusted according to the received adjustment instructions to adjust the orientation of the image acquisition device 110 and its distance from the display device 200.

[0039] The controller 120 is used to acquire data required in the embodiments of this application, such as the image captured by the image acquisition device 110. It is also used to determine the display edge of the display device 200 from the image based on the relative position information between the image acquisition device 110 and the display device 200, the size information of the display device, and the parameter information of the image acquisition device 110. The display edge refers to the position of the edge of the display device 200 in the image.

[0040] In the embodiments of this application, Figure 1 In the edge extraction system of the display device, the display device presents a first surface. In this embodiment, the first surface of the display device is the surface where the display area is located. Figure 2In the edge extraction system of the displayed device, the display device shows a second side, which is related to the first side. For example... Figure 2 As shown, the edge extraction system 100 may further include a lighting device 130. In embodiments of this application, the lighting device 130 may include at least one lighting component 131, wherein the lighting component 131 is a lighting module that can be controlled individually within the lighting device 130. For example, the lighting device 130 may be a light strip, which may be arranged around the display device 200. Optionally, the light strip may be arranged around the edge of the display device 200 or around the rear edge of the display device. The light strip may include multiple light segments, wherein each light segment is provided with a control IC and an LED, and each light segment can be controlled individually. Therefore, each light segment can be regarded as a lighting component. The control command for the light strip includes the light segment commands of all light segments in the light strip. After receiving the control command of the light strip, another light segment in the light strip (e.g., the first light segment) extracts the light segment command corresponding to itself from the control command and sends the control command corresponding to other light segments in the control command to the next light segment, so that the next light segment extracts the light segment command corresponding to itself from it, and so on, until all light segments in the light strip have extracted the light segment command corresponding to themselves.

[0041] Specifically, the controller 120 can determine the display edge of the display device from the screen image, and then accurately extract the image data corresponding to the display device from the screen image acquired by the image acquisition device 110. Based on the distribution of the target display area in the display device, the controller can further determine the image data corresponding to the target display area from the extracted image data corresponding to the display device, and thus control the lighting component 131 corresponding to the target display area based on the image data corresponding to the target display area.

[0042] In some implementations, each lighting component 131 corresponds to one of the target display areas A of the display device 200. During the display process of the display device 200, the lighting component can follow the color change of the target display area A in the display device 200 to display the corresponding lighting effect. For example, the lighting color of the lighting component can be the same as or similar to the color of the target display area in the display device 200, thereby creating an immersive picture playback environment and improving the user experience.

[0043] In the embodiments of this application, the display device 200 may include at least one target display area A. The number of target display areas A can be set according to actual usage needs, and the number of target display areas can be selected between 1 and 512.

[0044] Optionally, the target display area A may be, for example, the edge area of ​​the display device 200.

[0045] Optionally, when there are multiple target display areas A, the area sizes of the multiple target display areas A can be the same or different, and this application does not impose any restrictions on this.

[0046] like Figure 1 As shown, this embodiment of the application takes the display device 200 as an example, which includes ten target display areas A located around the display device 200. Each target display area A corresponds to a lighting component 131.

[0047] In some other embodiments, the lighting device 130 can also be a lamp post, which can be vertically erected on both sides of the display device 200. In this case, there are two target display areas A, located near the two side edges of the display device 200. When the lighting device 130 includes only one lighting component 131, multiple lighting devices 130 can be arranged on the display device 200 to present a rich lighting effect.

[0048] In some implementations, the controller 120 can establish communication connections with the image acquisition device 110 and the lighting device 130 via wired or wireless means, respectively, and then transmit data according to the established communication connections. The controller 130 can also communicate with the smart terminal 300 via wireless communication, thereby obtaining information such as the size of the display device 200 and the relative position information between the image acquisition device 120 and the display device according to the input commands of the smart terminal 300.

[0049] Optionally, the controller 120 can establish a communication connection with the lighting device 130, the image acquisition device 110, and the smart terminal 300 via a network to transmit data. The network is typically the Internet, but can also be any other network, including but not limited to any combination of Local Area Network (LAN), Metropolitan Area Network (MAN), Wide Area Network (WAN), mobile, wired or wireless networks, private networks, or virtual private networks. In some embodiments, the smart light strip and devices such as mobile terminals can communicate via specific communication protocols, including but not limited to BLE (Bluetooth Low Energy), WLAN (Wireless Local Area Network), Bluetooth, ZigBee, or Wi-Fi (Wireless Fidelity).

[0050] The following will illustrate this with specific examples.

[0051] Please see Figure 3 This application provides an edge extraction method for a display device. This edge extraction method can be applied to a controller. The lighting effect control method of this application may include steps S110 to S120.

[0052] Step S110: Acquire the image captured by the image acquisition device. The display device is located within the image acquisition range of the image acquisition device.

[0053] In the embodiments of this application, the display device is located within the acquisition range of the image acquisition device, so that the image captured by the image acquisition device includes the image data required by the display device to acquire the display edge.

[0054] In some implementations, the display device is entirely within the acquisition range of the image acquisition device. Thus, since the display device is entirely within the image acquisition range of the image acquisition device, the image on the screen also includes all the image data corresponding to the display device, such as the image data corresponding to the vertices of the display device, the image data corresponding to the target display area on the display device used to control the lighting effects, etc.

[0055] In some embodiments, the display device may not be entirely within the acquisition range of the image acquisition device. For example, a key area of ​​the display device may be entirely within the acquisition range of the image acquisition device, while areas outside the key area may not be within its acquisition range. The key area may be, for example, the vertex position of the display device. Alternatively, the key area may include the discrete positions of some discrete points on the display device. In some embodiments, the key area may also include the target display area of ​​the display device for lighting effects based on the target display area. It is understood that the key area of ​​the display device is entirely within the acquisition range of the image acquisition device, meaning the acquired image includes the image data required to obtain the display edge. Therefore, even if some areas outside the key area of ​​the display device do not fall within the acquisition range of the image acquisition device, it does not affect the implementation of the edge extraction method of this application. For example, assuming that some areas on the edge of the display device, excluding vertices, do not fall within the acquisition range of the image acquisition device, the display edge corresponding to that edge can be determined subsequently through vertex positions, such as lines connecting vertex positions.

[0056] Image data can include the number of pixels in an image, their coordinates, and the color data of each pixel. It's understandable that color data can represent the color of each pixel in numerical form using various color modes, such as RGB (Red, Green, Blue), CMYK (Cyan, Magenta, Yellow, Black), and HSB (Hue, Saturation, Brightness). It's also understandable that different color modes can be converted to each other; that is, pixel color data can be converted from RGB to CMYK, etc. Based on image data, the color distribution of the image can be determined, and then the lighting effects of lighting equipment can be controlled according to this color distribution.

[0057] In the embodiments of this application, the image acquisition device can acquire images according to a preset detection cycle, that is, acquire images at each detection moment. Furthermore, the lighting effect of the lighting equipment at the same moment can be controlled according to the image acquired at each detection moment, so that the lighting effect of the lighting equipment can change synchronously with the changes in the image.

[0058] In some embodiments, the carrier of the display device can be a product with display function such as a mobile phone, laptop computer, tablet computer, or television; the display device can be used to display television programs, on-demand videos, live videos, game screens, and advertising videos. The carrier and purpose of the display device can be set according to actual needs, and this application does not impose specific limitations on them.

[0059] In some embodiments, the image acquisition device may be a camera, camcorder, scanner, or other device with a photo-taking function (mobile phone, tablet computer, etc.), and this application does not limit it.

[0060] Step S120: Determine the display edge of the display device from the image image based on the relative position information of the image acquisition device and the display device, the size information of the display device, and the parameter information of the image acquisition device. The display edge is the position of the edge of the display device in the image image.

[0061] In the embodiments of this application, the relative position information between the image acquisition device and the display device may be the installation position information of the image acquisition device on the display device, such as the image acquisition device being installed at the center of one side of the display device.

[0062] In some implementations, the relative position information can also be the relative angle between the image acquisition device and the display device. Optionally, the image acquisition device can be adjusted in angle. Figure 1In this configuration, the image acquisition device 110 can be rotated along the connecting portion 113 to adjust its angle. When the image acquisition device is mounted on the display device, the angle of the image acquisition device can be adjusted to adjust the relative angle between the image acquisition device and the display device. It is understood that if the relative angle between the image acquisition device and the display device is different, the position of the display device within the image acquisition range of the image acquisition device will also be different, and consequently, the position of the image corresponding to the display device in the image captured by the image acquisition device will also be different.

[0063] Optionally, the relative position information between the image acquisition device and the display device can be a preset fixed value. In this case, the user needs to adjust the image acquisition device to ensure that the relative position information between the image acquisition device and the display device is at the preset value during use.

[0064] Optionally, the relative position information between the image acquisition device and the display device can be adjusted according to the user's needs. Thus, during use, the user needs to adjust the image acquisition device to achieve the desired relative position with the display device. Optionally, the user can input the adjusted relative position information between the image acquisition device and the display device through the user interface of the mobile terminal, and the controller can obtain the user-input relative position information through the mobile terminal.

[0065] In some embodiments, the size information of the display device may be the side length of the display device, such as the length between adjacent vertices of the display device. In some embodiments, the size information of the display device may also include angle information between each side and adjacent sides; for example, the size information may include the relative angle between each side and adjacent sides. When the shape of the display device is rectangular, the relative angle between each side and adjacent sides is 90°.

[0066] Optionally, when the controller is applied to display devices of different sizes, for example, if the controller was originally applied to display device A with a first size, and the display device is changed to display device B with a second size, in order for the controller to still be able to extract the edges of display device B with a different size, the user can input the size information of the corresponding display device, so that the controller can obtain the size information of the current display device, thereby adapting to display devices of different sizes. For example, the user can input the size information of the display device on the user interface of a mobile terminal, so that the controller can obtain the size information of the display device input by the user through the mobile terminal.

[0067] In some implementations, the parameter information of the image acquisition device may be parameters such as the focal length and length of the image acquisition device. These parameters can affect the acquisition range of the image acquisition device, and thus affect the image captured by the image acquisition device.

[0068] Furthermore, the controller can determine the display edge, that is, the position of the edge of the display device in the image, based on the relative position information of the image acquisition device and the display device, the size information of the display device, and the parameter information of the image acquisition device.

[0069] The edge extraction method for a display device provided in this application acquires a screen image captured by an image acquisition device, wherein the display device is located within the image acquisition range of the image acquisition device. Based on the relative position information between the image acquisition device and the display device, the size information of the display device, and the parameter information of the image acquisition device, the display edge of the display device is determined from the screen image. The display edge is the position of the edge of the display device in the screen image. Therefore, the edge of the display device can be automatically identified from the screen image based on the relative position information between the image acquisition device and the display device, the size information of the display device, and the parameter information of the image acquisition device, eliminating the need for manual operation by the user, reducing edge extraction errors, improving edge extraction efficiency, and enhancing the user experience.

[0070] In some embodiments of this application, the relative position information includes the installation position of the image acquisition device on the display device and the relative angle between the image acquisition device and the display device.

[0071] In some embodiments of this application, the parameter information of the image acquisition device includes the length of the image acquisition device and the focal length of the image acquisition device. The length of the image acquisition device is the distance between the mounting position of the image acquisition device on the display device and the acquisition lens of the image acquisition device. For example... Figure 1 As shown, the length of the image acquisition device can be the distance between the image mounting part 112 of the image acquisition device 110 and the acquisition lens of the image acquisition part 111.

[0072] In some implementations, the length of the image acquisition device is a known parameter, which can be a fixed parameter of the image acquisition device, a fixed value, or the controller can obtain the specific value of the length of the image acquisition device from the image acquisition device itself. Alternatively, the controller can determine the specific value of the length of the image acquisition device based on its model.

[0073] Furthermore, such as Figure 4 As shown, step 120 may further include steps S210 to S220.

[0074] Step S210: Determine the vertex image position of each vertex of the display device in the image based on the installation location, relative angle, length of the image acquisition device, focal length of the image acquisition device, and size information of the display device.

[0075] like Figure 5As shown, the image acquisition device 120 is mounted at position O on the display device 100, and the image acquisition position of the image acquisition device 110 is G, thus the length of the image acquisition device 120 is GO. The relative angle between the image acquisition device 120 and the display device 200 is ∠GOC. The size information of the display device 200 may include the lengths of the four sides BC, CD, DE, and EB of the display device 120.

[0076] The following explanation will take obtaining the vertex image position of vertex C of display device 200 in the screen image as an example. Figure 5 In the process, the image acquired by the image acquisition device 110 is projected onto the imaging target surface IJ, thereby forming an image on the imaging target surface IJ. The focal length of the image acquisition device 120 is GM, where the focal length GM is perpendicular to the imaging target surface IJ, and point M is the center point of the imaging target surface, which is also the center point of the image acquired by the image acquisition device 110.

[0077] In this embodiment, the relative angle between the image acquisition device 110 and the display device 200 is ∠GOC.

[0078] In order to obtain the vertex image position C' of vertex C in the image, it is necessary to obtain the third distance C'M of image position C' relative to center point M.

[0079] C'M=tan(∠C'GM)*GM (Formula 1)

[0080] Furthermore, since ∠C'GM=∠GCN (Formula 2)

[0081]

[0082] CN = OC - ON (Formula 4)

[0083] ON = GO * cos(∠GON) (Formula 5)

[0084] GN=GO*sin(∠GON)(Formula 6)

[0085]

[0086] Therefore, the third distance C'M of the image position C' relative to point M can be obtained according to formulas one through seven above. It is understood that the image position C can also be determined through other geometric relationships of vertex C, and this application does not impose any restrictions on this.

[0087] Similarly, the vertex image positions of other vertices of the display device 200 in the screen image can be obtained.

[0088] To achieve more accurate edge positioning, in addition to determining the vertex image positions of each vertex of the display device in the screen image, a preset number of discrete points can be selected on each side of the display device. The positional relationships between these discrete points and the vertices are known and can be selected as needed; for example, the midpoints or quarter-division points of each side of the display device can be chosen. Figure 6 As shown, 10 discrete points can be selected on each side of the display device: the first discrete point L1 and the second discrete point L2 between vertex B and vertex C; the third discrete point L3, the fourth discrete point L4, and the fifth discrete point L5 between vertex C and vertex D; the sixth discrete point L6 and the seventh discrete point L7 between vertex D and vertex E; and the eighth discrete point L8, the ninth discrete point L9, and the tenth discrete point L10 between vertex E and vertex B. Then, the discrete image position of each discrete point in the screen image is determined, and the specific method can refer to the method for obtaining vertex image positions described above.

[0089] It is understandable that the more discrete points selected, the more accurate the display edge will be, but the computational load will be relatively larger. The selection can be made according to the accuracy requirements of actual use, and this application does not impose any restrictions on this.

[0090] Step S220: Determine the display edge of the display device based on the vertex image position.

[0091] In some implementations, the line connecting the vertex image positions of all vertices in the image can be used as the display edge of the display device in the image. For example, Figure 5 The four vertices of the display device are B, C, D, and E, and their vertex image positions in the screen image are B', C', D', and E', respectively. Therefore, B'C', C'D', D'E', and E'B' are the display edges. Thus, line segments B'C', C'D', D'E', and E'B in the screen image are the display edges of the display device.

[0092] In some implementations, to obtain a more accurate display edge for the display device, the lines connecting the vertex image positions of each vertex and the discrete image positions of each discrete point in the image can be used as the display edge of the display device in the image. Specifically, the connection relationship between the vertex image positions and the discrete image positions is the same as the connection relationship between the corresponding vertices and discrete points.

[0093] In other embodiments, in order to obtain the display edge of the display device in the image more accurately, image segmentation can be performed based on the grayscale of the image data corresponding to the line connecting the vertex image positions of each vertex in the image, thereby obtaining a more accurate display edge, which will be described in detail below.

[0094] like Figure 7As shown, step S220 may include steps S221 to S223.

[0095] Step S221: Connect the vertex image positions according to the preset connection relationship to form the first edge connection line.

[0096] The preset connection relationship is the connection relationship between the vertex image position and other vertex image positions. The connection relationship between the vertex image position and other vertex image positions is the same as the connection relationship between the vertex corresponding to the vertex image position and other vertices.

[0097] For example, Figure 5 The four vertices of the display device are B, C, D, and E, and their vertex image positions in the image are B', C', D', and E', respectively. In the display device 200, if vertex B is taken as the starting point, the connection relationship between vertex B and the other vertices is: vertex B connects to vertex C, vertex C connects to vertex D, and vertex D connects to vertex B. Therefore, the connection relationship between vertex image position B' and the other image positions is: vertex image position B' connects to vertex image position C', vertex image position C' connects to vertex image position D', and vertex image position D' connects to vertex image position B', thus obtaining the first edge connection line B'C'D'E'.

[0098] Step S222: Increase the width of the first edge connection by a preset number of pixels to obtain the second edge connection.

[0099] In embodiments of this application, in order to accurately obtain the display edge of the display device in the image, the image data around the first edge line B'C'D'E' can be further analyzed. To obtain the image data around the first edge line B'C'D'E', the first edge line B'C'D'E' can be widened to obtain the image data around the first edge line B'C'D'E'.

[0100] Optionally, the width of the first edge line can be increased by a preset number of pixels towards the center of the image to obtain the second edge line.

[0101] Optionally, the width of the first edge line can be increased by a preset number of pixels in a direction away from the center of the image to obtain a second edge line.

[0102] Optionally, the width of the first edge line can be increased by half a preset number of pixels in both the direction closer to the center of the image and the direction farther from the center of the image to obtain the second edge line.

[0103] It is understandable that the number of preset pixels, i.e. the width of the first edge connection line, can be set according to actual needs, and this application does not impose any restrictions on this.

[0104] Step S223: Perform edge extraction processing on the image data corresponding to the second edge connection line to obtain the display edge.

[0105] In the embodiments of this application, the image data of the edge of the display device is different from the image data of the surrounding area. The position of the edge of the display device in the image can be further determined based on the grayscale of the image data corresponding to the second edge line.

[0106] Furthermore, such as Figure 8 As shown, step S223 may include steps S2231 to S2233.

[0107] Step S2231: Convert the YUV color data corresponding to the image data of the second edge connection line into RGB color data.

[0108] Step S2232: Convert RGB color data into grayscale data.

[0109] Step S2233: Use the watershed algorithm to process the grayscale data to extract the display edges.

[0110] The Watershed Algorithm is an image region segmentation method that uses the similarity between neighboring pixels as an important reference during the segmentation process. It connects pixels that are spatially close and have similar grayscale values ​​(gradient calculation) to form a closed contour. Specifically, the display edge can be obtained using the following formula (Equation 8).

[0111] g(x,y)=max(grad(f(x,y)),gθ)(Formula 8)

[0112] Where gθ represents the threshold, and g(x,y) obtains the coordinates (x,y) of the pixel position at the display edge. gθ represents the average of the sum of grayscale values ​​in the region containing the display edge.

[0113] In some embodiments, the display device includes a target display area. After determining the display edge, the screen image corresponding to the display device can be automatically, quickly, and accurately identified from the screen image. Furthermore, based on the distribution of the target display area within the display device, image data corresponding to the target display area can be further determined within a smaller range of the screen image corresponding to the display device. Specifically, such as... Figure 9As shown, after step 120, the edge extraction method of the display device in this application embodiment may further include: steps S310 to S330.

[0114] Step S310: Determine the region edge based on the display edge and region distribution information. The region edge is the position of the edge of the target display area of ​​the display device in the image.

[0115] In embodiments of this application, the edge of the target display area on the display device can be determined based on regional distribution information. For example, when the target display area is the area surrounding the display device, the edge of the target display area on the display device can be an edge formed by being parallel to the four sides of the display device and shifted a preset distance towards the center of the display device. Figure 10 As shown, the edge of the target display area A is formed by the edge connection lines BCDE and K4K3K2K1 of the display device 200. The edge connection lines K4K3K2K1 are formed by shifting the four edges of the display device 200 towards the center of the display device by a predetermined distance d. In other words, the area enclosed by the four edges of the display device 200, the edge connection lines K4K3K2K1, and the edge connection lines BCDE of the display device 200 is the target display area A.

[0116] Furthermore, referring to the method described above for obtaining the vertex image position of the display device A in the image, the image distance d' corresponding to the preset distance d in the image can be obtained. Thus, the inwardly contracted edge line K4'K3'K2'K1' obtained by offsetting the edge line B'C'D'E' of the display device 200 towards the center of the image by the image distance d' can be used as the region edge of the target display area in the image. In the embodiments of this application, the region edge of the target display area also includes the edge line B'C'D'E', that is, the region edge of the target display area includes the edge line B'C'D'E' and the edge line K4'K3'K2'K1', and the area enclosed by the edge line B'C'D'E' and the edge line K4'K3'K2'K1' constitutes the region corresponding to the target display area.

[0117] In other embodiments, the edges of the target display area on the display device can be determined directly based on the area distribution information, namely edge lines BCDE and K4K3K2K1. Then, referring to the method described above for obtaining the vertex image positions of the vertices of display device A in the image, the image positions A', B', C', D', K4', K3', K2', and K1' of points A, B, C, D, K4, K3, K2, and K1 in the image are obtained respectively, thereby obtaining the area edges, namely edge lines B'C'D'E' and K4'K3'K2'K1'.

[0118] Furthermore, in order to obtain more accurate region edges, the YUV color data corresponding to the edge connection line of the region edge can be converted into RGB color data, and then the RGB color data can be converted into grayscale data. Then, the watershed algorithm is used to process the grayscale data to extract the region edges. The specific process can be referred to the detailed explanation of steps S2331 to S2333, which will not be repeated here.

[0119] Step S320: Determine the target image data corresponding to the target display area in the image based on the display edge and the area edge;

[0120] In the embodiments of this application, the image corresponding to the area enclosed by the area edge can be further extracted from the image based on the area edge. Thus, the image to be calculated to determine the target image data is accurate to include only the image of the target display area, which can greatly reduce the amount of calculation, reduce the complexity of lighting effect control, thereby improving the efficiency of lighting effect control and ensuring the synchronization of lighting effect control and display.

[0121] In some implementations, step S320, which determines the target image data, may further include steps S321 to S322.

[0122] Step S321: Extract the region image data corresponding to the target display area from the image based on the region edge.

[0123] Step S322: Perform affine transformation on the region image data to obtain the target image data.

[0124] Understandably, due to the different relative positional information between the image acquisition device and the display device, the position and shape of the display device within the image captured by the image acquisition device will also differ. For example, the image on the display device may be distorted in the actual captured image. This results in the proportion of the same color in the same area of ​​the target display area differing from the proportion of the same area in the image, thus affecting the accuracy of lighting effect control.

[0125] To accurately obtain the color distribution in the target display area, in the embodiments of this application, the region image data can be subjected to affine transformation. An affine transformation is a linear transformation from one two-dimensional coordinate system to another, preserving the relative positions between the two-dimensional images and maintaining the positional order and length ratio of points on the same straight line. Affine transformations can be implemented using a series of basic transformation symbols, including translation, scaling, rotation, flipping, and shearing. Through affine transformation processing, the region image data can be restored to the target image data, ensuring that the proportion of each color in the target display area within the target image data is the same as the proportion of each color in the corresponding area on the display device.

[0126] Step S330: Control the lighting effects of the lighting components corresponding to the target display area based on the target image data.

[0127] In the embodiments of this application, the color value with the highest proportion in the target display area is determined based on the target image data corresponding to the target display area. The lighting effect of the lighting component corresponding to the target display area is controlled according to the color with the highest proportion. Since the image data of the current area is accurately obtained, the accuracy of determining the color with the highest proportion is improved. By controlling the lighting effect of the corresponding lighting component according to the color with the highest proportion, the consistency between the lighting effect of the lighting component and the color of the image currently presented in the target display area of ​​the display device can be guaranteed, thereby improving the user's viewing experience.

[0128] In some embodiments of this application, such as Figure 11 As shown, step S330 may further include steps S410 to S420.

[0129] Step S410: Determine the target color value with the highest proportion in the target image data.

[0130] The target image data includes the position and color value of the pixels in the image corresponding to the target display area. Based on the target image data, the color distribution of the pixels in the image corresponding to the target display area can be determined, and then the color value with the highest proportion in the target image data can be determined as the target color value.

[0131] As one implementation method, such as Figure 12 As shown, the target color value with the highest proportion in the target image data can also be determined through steps S411 to S414.

[0132] Step S411: Obtain the YUV color data of the target image data.

[0133] Step S412: Convert YUV color data to RGB color data.

[0134] Step S413: Convert RGB color data to CIELAB color data.

[0135] Step S414: Perform color difference calculation and color quantization on the CIELAB color data to obtain the target color with the highest proportion.

[0136] Step S420: Control the lighting effect of the lighting components corresponding to the target display area according to the target color.

[0137] In the embodiments of this application, the YUV color data of the target image data is converted into RGB color data, and then the RGB color data is converted into CIELAB color data. Color difference is calculated based on the CIELAB color data to distinguish different colors numerically. Then, color quantization is performed to count the proportion of each color, thereby obtaining the color with the highest proportion as the target color value.

[0138] Alternatively, the CIE 94 color difference formula can be used to calculate the color difference. The CIE 94 color difference formula is currently the standard used in industry for evaluating the color difference of objects under daylight illumination.

[0139] Color quantization leverages the inertia of the human eye regarding color, merging less important similar colors in the original image into a single color, reducing the number of colors in the image, and minimizing the perceptual error between the quantized and unquantized images—that is, minimizing the quantization error. Optionally, the K-means clustering color quantization algorithm can be used for color quantization processing.

[0140] In some embodiments, before step S110, the method may further include the steps of: responding to user input operations and acquiring relative position information between the image acquisition device and the display device, and size information of the display device.

[0141] In some implementations, users can directly input the relative position information between the image acquisition device and the display device, as well as the size information of the display device, through the interactive interface of the mobile terminal. Optionally, after inputting the relative position information and size information through the interactive interface, the user can click the submit control to send the relative position information and size information to the controller.

[0142] In some implementations, the mobile terminal's interface can display images from both the display device and the image acquisition device. The user manually drags the image acquisition device to the corresponding position on the display device, based on its set position. Optionally, after the user inputs the adjusted relative position of the image acquisition device on the interface and clicks the submit control, the mobile terminal can convert the adjusted position of the image acquisition device into actual relative position information between the image acquisition device and the display device according to a preset ratio, and then send this relative position information to the controller. It is understood that the step of converting the data according to a preset ratio can also be performed by the controller, and this application does not limit this.

[0143] Please see Figure 13 This application provides an edge extraction device 400 for a display device, which includes an image acquisition module 410 and an edge extraction module 420.

[0144] The image acquisition module 410 is used to acquire the image captured by the image acquisition device. The display device is located within the image acquisition range of the image acquisition device.

[0145] The edge extraction module is used to determine the display edge of the display device from the image based on the relative position information of the image acquisition device and the display device, the size information of the display device, and the parameter information of the image acquisition device. The display edge is the position of the edge of the display device in the image.

[0146] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For device embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to in the descriptions of the method embodiments. Any processing method described in the method embodiments can be implemented in the device embodiments through corresponding processing modules, and will not be elaborated upon further in the device embodiments.

[0147] Please see Figure 14 This application provides an electronic device 500, which may be a smartphone, tablet computer, e-reader, touch screen, or other electronic device capable of running applications. The electronic device 500 in this application may include one or more of the following components: a processor 510, a memory 520, and one or more applications, wherein the one or more applications may be stored in the memory 520 and configured to be executed by one or more processors 510, and the one or more applications are configured to perform the methods described in the foregoing method embodiments.

[0148] Processor 510 may include one or more processing cores. Processor 510 connects to various parts of the electronic device using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory, and by calling data stored in memory. Optionally, processor 510 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 510 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and Modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor and may be implemented separately using a communication chip.

[0149] The memory 520 may include random access memory (RAM) or read-only memory (ROM). The memory can be used to store instructions, programs, code, code sets, or instruction sets. The memory may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as image acquisition, image extraction, etc.), and instructions for implementing the various method embodiments described below. The data storage area may also store data created by the terminal during use (such as screen images, size information, relative position information, etc.).

[0150] Please refer to Figure 15 This diagram illustrates a structural block diagram of a computer-readable storage medium 600 provided in an embodiment of this application. The computer-readable medium 600 stores program code 610, which can be called by a processor to execute the methods described in the above method embodiments.

[0151] Computer-readable storage medium 600 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium includes a non-transitory computer-readable storage medium. The computer-readable storage medium 600 has storage space for program code that performs any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code may, for example, be compressed in a suitable form.

[0152] In summary, the edge extraction method, system, apparatus, electronic device, and medium provided in this application for a display device determine the display edge from the image by acquiring a screen image captured by an image acquisition device, wherein the display device is located within the image acquisition range of the image acquisition device. The display edge is determined from the image image based on the relative position information between the image acquisition device and the display device, the size information of the display device, and the parameter information of the image acquisition device. The display edge is the position of the edge of the display device in the image image. This allows for automatic identification of the display device's edge from the image image based on the relative position information between the image acquisition device and the display device, the size information of the display device, and the parameter information of the image acquisition device, eliminating the need for manual user operation, reducing edge extraction errors, improving edge extraction efficiency, and enhancing the user experience.

[0153] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for edge extraction in a display device, characterized in that, A controller for an edge extraction system; the edge extraction system also includes an image acquisition device and a lighting device; The lighting device includes at least one lighting component; the edge extraction method includes: The image is acquired by the image acquisition device; wherein the display device is located within the image acquisition range of the image acquisition device; wherein the number of the image acquisition devices is one. The display edge of the display device is determined from the image image based on the relative position information between the image acquisition device and the display device, the size information of the display device, and the parameter information of the image acquisition device. The display edge is the position of the edge of the display device in the image image. The relative position information includes the installation position of the image acquisition device on the display device and the relative angle between the image acquisition device and the display device. The parameter information of the image acquisition device includes the length of the image acquisition device and the focal length of the image acquisition device. The length of the image acquisition device is the distance between the installation position of the image acquisition device on the display device and the acquisition lens of the image acquisition device. The step of determining the display edge of the display device from the image based on the relative position information between the image acquisition device and the display device, the size information of the display device, and the parameter information of the image acquisition device includes: determining the vertex image position of each vertex of the display device in the image based on the installation position, the relative angle, the length of the image acquisition device, the focal length of the image acquisition device, and the size information of the display device; and determining the display edge of the display device based on the vertex image position. The region edge is determined based on the display edge and region distribution information, wherein the region edge is the position of the edge of the target display area of ​​the display device in the image. Based on the display edge and the region edge, the target image data corresponding to the target display area is determined in the image; The lighting effects of the lighting components corresponding to the target display area are controlled based on the target image data.

2. The edge extraction method according to claim 1, characterized in that, Determining the display edge of the display device based on the vertex image position includes: The vertex image positions are connected according to a preset connection relationship to form a first edge line. The preset connection relationship is the connection relationship between the vertex image position and other vertex image positions. The connection relationship between the vertex image position and other vertex image positions is the same as the connection relationship between the vertex corresponding to the vertex image position and other vertices. Increase the width of the first edge connection by a preset number of pixels to obtain the second edge connection; The image data corresponding to the second edge line is subjected to edge extraction processing to obtain the display edge.

3. The edge extraction method according to claim 2, characterized in that, The step of performing edge extraction processing on the image data corresponding to the second edge connection line to obtain the display edge includes: Convert the YUV color data corresponding to the image data of the second edge line into RGB color data; Convert the RGB color data into grayscale data; The grayscale data is processed using the watershed algorithm to extract the display edges.

4. The edge extraction method according to claim 1, characterized in that, The step of determining the target image data corresponding to the target display area in the image based on the display edge and the area edge includes: The image data corresponding to the area enclosed by the display edge and the area edge is used as the area image data corresponding to the target display area; The target image data is obtained by performing an affine transformation on the image data of the region.

5. The edge extraction method according to claim 4, characterized in that, The step of controlling the lighting effect of the lighting component corresponding to the target display area based on the target image data includes: Determine the target color that has the highest proportion in the target image data; The lighting effect of the lighting components corresponding to the target display area is controlled according to the target color.

6. The edge extraction method according to claim 5, characterized in that, Determining the target color with the highest proportion in the target image data includes: Obtain the YUV color data of the target image data; Convert the YUV color data into RGB color data; Convert the RGB color data to CIELAB color data; The CIELAB color data is subjected to color difference calculation and color quantization to obtain the target color with the highest proportion.

7. The edge extraction method according to claim 1, characterized in that, Before acquiring the image captured by the image acquisition device, the method further includes: In response to user input, the relative position information of the image acquisition device and the display device, and the size information of the display device, are obtained.

8. An edge extraction system for a display device, characterized in that, The edge extraction system includes: An image acquisition device is used to acquire screen images, wherein the display device is located within the image acquisition range of the image acquisition device; wherein the number of the image acquisition devices is one. A controller is configured to acquire the image of the display device. The controller is further configured to determine the display edge of the display device from the image of the display device based on the relative position information between the image acquisition device and the display device, the size information of the display device, and the parameter information of the image acquisition device. The display edge is the position of the edge of the display device corresponding to the edge of the display device in the image of the display device. The relative position information includes the installation position of the image acquisition device on the display device and the relative angle between the image acquisition device and the display device. The parameter information of the image acquisition device includes the length of the image acquisition device and the focal length of the image acquisition device. The length of the image acquisition device is the distance between the installation position of the image acquisition device on the display device and the acquisition lens of the image acquisition device. Determining the display edge of the display device from the image of the display device based on the relative position information between the image acquisition device and the display device, the size information of the display device, and the parameter information of the image acquisition device includes: determining the vertex image position of each vertex of the display device in the image of the display device based on the installation position, the relative angle, the length of the image acquisition device, the focal length of the image acquisition device, and the size information of the display device; and determining the display edge of the display device based on the vertex image position. Lighting equipment, including at least one lighting component; The controller is further configured to determine the region edge based on the display edge and region distribution information, wherein the region edge is the position corresponding to the edge of the target display area of ​​the display device in the image; determine the target image data corresponding to the target display area in the image based on the display edge and the region edge; and control the lighting effect of the lighting component corresponding to the target display area based on the target image data.

9. An edge extraction device for a display device, characterized in that, A controller for an edge extraction system; the edge extraction system also includes an image acquisition device and a lighting device; The lighting device includes at least one lighting component; the edge extraction device includes: An image acquisition module is used to acquire images captured by an image acquisition device; wherein, a display device is located within the image acquisition range of the image acquisition device; wherein, the number of the image acquisition devices is one. An edge extraction module is used to determine the display edge of the display device from the image frame based on the relative position information between the image acquisition device and the display device, the size information of the display device, and the parameter information of the image acquisition device. The display edge is the position of the edge of the display device in the image frame. The relative position information includes the installation position of the image acquisition device on the display device and the relative angle between the image acquisition device and the display device. The parameter information of the image acquisition device includes the length of the image acquisition device and the focal length of the image acquisition device. The length of the image acquisition device is the distance between the installation position of the image acquisition device on the display device and the acquisition lens of the image acquisition device. Determining the display edge of the display device from the image frame based on the relative position information between the image acquisition device and the display device, the size information of the display device, and the parameter information of the image acquisition device includes: determining the vertex image position of each vertex of the display device in the image frame based on the installation position, the relative angle, the length of the image acquisition device, the focal length of the image acquisition device, and the size information of the display device; and determining the display edge of the display device based on the vertex image position. The edge extraction module is used to determine the region edge based on the display edge and region distribution information, wherein the region edge is the position of the edge of the target display area of ​​the display device in the image; determine the target image data corresponding to the target display area in the image based on the display edge and the region edge; and control the lighting effect of the lighting component corresponding to the target display area based on the target image data.

10. An electronic device, characterized in that, include: One or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more applications being configured to perform the method as described in any one of claims 1-7.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code that can be invoked by a processor to execute the edge extraction method as described in any one of claims 1-7.

Images