A radar-based cross-screen tracking method and system

Abstract

This application relates to the field of cross-screen interaction technology for displays, and provides a radar-based cross-screen tracking method and system for multiple displays. First, a virtual joint region is created based on the multiple displays. Then, a radar sensing region is determined based on the virtual joint region. After acquiring point clouds of target objects within the radar sensing region using radar, the sensing coordinates of the target objects within the radar sensing region are obtained from the point clouds. Next, the position coordinates of the target objects within the radar sensing region are obtained, and then the position coordinates of the target objects within the virtual joint region are derived from these position coordinates. Finally, cross-screen tracking of the target objects is performed based on these position coordinates within the virtual joint region. This application enables cross-screen tracking of target objects across multiple displays.

Description

Technical Field

[0001] This application relates to the technical field of cross-screen interaction of display screens, and particularly to a cross-screen tracking method and system for multiple display screens based on radar. Background Art

[0002] In recent years, interactive Light Emitting Diode (LED) display screens have been increasingly used. A large screen display composed of multiple LED display screens can provide a larger display area and higher resolution, and is very suitable for large events, exhibitions, advertisements and other places. Radar is an electronic detection system that can radiate electromagnetic waves and use the reflection of these electromagnetic waves by objects to detect targets and determine the positions of targets. The positioning of interactive LED display screens based on radar is more accurate and the interactivity is higher.

[0003] In the prior art, there are many scenarios where radar is used for interactive following sensing of target objects, but they are all based on a single display screen and cannot achieve multi-screen linkage sensing. That is, when multiple LED display screens based on radar are installed side by side on a plane, cross-screen target object tracking between multiple LED display screens cannot be achieved. Summary of the Invention

[0004] To solve the technical problem in the prior art that cross-screen target object tracking between multiple display screens cannot be achieved, this application provides a cross-screen tracking method and system for multiple display screens based on radar.

[0005] The first aspect of this application provides a cross-screen tracking method for multiple display screens based on radar, and the method includes:

[0006] Create a virtual joint area according to the multiple display screens;

[0007] Determine a radar sensing area according to the virtual joint area, and multiple radars are correspondingly installed in the radar sensing area;

[0008] Obtain the point cloud of the target object in the radar sensing area collected by the radar, and obtain the sensing coordinates of the target object in the radar sensing area according to the point cloud;

[0009] Process the sensing coordinates to obtain the position coordinates of the target object in the radar sensing area;

[0010] According to the position coordinates of the target object in the radar sensing area, obtain the position coordinates of the target object in the virtual joint area;

[0011] Based on the position coordinates of the target object in the virtual joint area, perform cross-screen tracking on the target object.

[0012] Optionally, creating a virtual joint area based on the plurality of displays includes:

[0013] Obtain the physical width and physical height of each of the aforementioned displays;

[0014] The physical width of the virtual joint region is determined based on multiple physical widths;

[0015] Determine the maximum physical height among the multiple physical heights, and determine the physical height of the virtual union region based on the maximum physical height;

[0016] The virtual union region is created based on the physical width and the physical height.

[0017] Optionally, obtaining the sensing coordinates of the target object in the radar sensing area based on the point cloud includes:

[0018] Obtain the installation position and angle of the radar relative to the radar sensing area;

[0019] Generate a point cloud radar coordinate transformation matrix based on the installation location and the installation angle;

[0020] Based on the point cloud radar coordinate transformation matrix and the point cloud, the sensing coordinates of the target object in the radar sensing area are obtained.

[0021] Optionally, processing the sensed coordinates to obtain the position coordinates of the target object within the radar sensing area includes:

[0022] The point cloud outside the radar sensing area is filtered out to obtain the target point cloud;

[0023] Based on the sensed coordinates of the target point cloud, the target point cloud is subjected to dimensionality reduction processing to obtain a two-dimensional point cloud;

[0024] The two-dimensional point cloud data is rasterized and saved.

[0025] Based on the preset refresh cycle, cluster analysis is performed on the saved two-dimensional point clouds to obtain multiple types of two-dimensional point clouds;

[0026] The position coordinates of each type of target object in the radar sensing area are obtained based on the sensing coordinates of the two-dimensional point cloud for each type.

[0027] Optionally, the step of performing cross-screen tracking of the target object based on its position coordinates within the virtual joint region includes:

[0028] Based on the location coordinates of the virtual joint region, calculate the distance between each type of target object in the previous refresh cycle and the target object in the current refresh cycle;

[0029] Obtain the target distance that is less than a preset distance threshold from among the multiple distances;

[0030] The target object corresponding to the previous refresh cycle and the target object corresponding to the current refresh cycle are determined to be the same object.

[0031] Optionally, the method further includes:

[0032] Based on the location coordinates of the target object in the virtual joint area, cross-screen image fusion is performed on the target object.

[0033] Optionally, the step of performing cross-screen image fusion on the target object based on its position coordinates in the virtual joint area includes:

[0034] The pixel size of the virtual joint region is determined based on the region size of the virtual joint region;

[0035] Create a texture object with the pixel size of the virtual union region:

[0036] The texture object is rendered to obtain a rendered texture object;

[0037] Based on the display area corresponding to the rendered texture object in each of the display screens, cross-screen image fusion is performed on the target object.

[0038] Optionally, the step of performing cross-screen image fusion on the target object based on the display area corresponding to the rendered texture object in each of the display screens includes:

[0039] Convert the position coordinates of the rendered texture object from the virtual joint region to pixel coordinates within the virtual joint region; based on the pixel coordinates of the rendered texture object within the virtual joint region, move the rendered texture object to the coordinate position corresponding to the desktop output area;

[0040] The rendered texture object in the desktop output area is scaled according to the pixel size of the virtual union region and the physical size of the desktop output area.

[0041] Optionally, converting the position coordinates of the rendered texture object from the virtual union region to pixel coordinates within the virtual union region includes:

[0042] The pixel coordinates of each display screen in the virtual joint area are obtained by dividing the position coordinates of each display screen in the virtual joint area by the smallest pixel pitch among the multiple display screens;

[0043] The pixel coordinates of the rendered texture object in the virtual joint region are obtained based on the pixel coordinates of each display screen in the virtual joint region.

[0044] A second aspect of this application provides a radar-based multi-display cross-screen tracking system, the system comprising:

[0045] Multiple displays mounted on the same plane;

[0046] An electronic device that connects the plurality of displays;

[0047] And multiple radars connected to the electronic device;

[0048] The electronic device is used to perform the steps of the radar-based multi-display cross-screen tracking method.

[0049] This application provides a radar-based cross-screen tracking method and system for multiple displays. First, a virtual joint region is created based on the multiple displays. Then, a radar sensing region is determined based on the virtual joint region. After acquiring point clouds of target objects within the radar sensing region using radar, the sensing coordinates of the target objects within the radar sensing region are obtained from the point clouds. Next, the position coordinates of the target objects within the radar sensing region are obtained. Based on these position coordinates, the position coordinates of the target objects within the virtual joint region are derived. Finally, cross-screen tracking of the target objects is performed based on their position coordinates within the virtual joint region. This application can accurately obtain the coordinates of the target position within the radar sensing region and convert them into position coordinates within the virtual joint region, thus accurately achieving cross-screen tracking of target objects across multiple displays. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a hardware architecture diagram of a radar-based multi-display cross-screen tracking system provided in Embodiment 1 of this application;

[0052] Figure 2 This is a flowchart of the radar-based multi-display cross-screen tracking method provided in Embodiment 2 of this application;

[0053] Figure 3 This is a schematic diagram of the three-dimensional coordinate system of the virtual joint region provided in the embodiments of this application;

[0054] Figure 4 This is a schematic diagram of the three-dimensional coordinate system of the radar sensing area provided in the embodiments of this application;

[0055] Figure 5 This is a flowchart of a radar-based multi-display cross-screen image fusion method provided in Embodiment 3 of this application.

[0056] Explanation of reference numerals in the attached figures:

[0057] 1. First display screen; 2. Second display screen; 3. Third display screen; 4. Synchronous display screen control system; 5. Broadcast control computer; 6. Radar sensing area; 7. Radar; 8. Virtual joint area. Detailed Implementation

[0058] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0059] Figure 1 This is a hardware architecture diagram of a radar-based multi-display cross-screen tracking system provided in Embodiment 1 of this application.

[0060] A radar-based multi-display cross-screen tracking system may include, but is not limited to, multiple displays (e.g., Figure 1 The system includes a first display screen 1, a second display screen 2, and a third display screen 3; a synchronous display screen control system 4 connecting multiple display screens; a broadcast control computer 5 connected to the synchronous display screen control system 4; and two radars 7 connected to the broadcast control computer 5.

[0061] The multiple displays need to be mounted on the same plane. In some embodiments, the displays can be light-emitting diode (LED) displays.

[0062] In this embodiment, the synchronized display screen control system 4 can be installed in an electronic device, and the broadcast control computer 5 can also be the electronic device. That is, the synchronized display screen control system 4 runs in the broadcast control computer 5. In other embodiments, the synchronized display screen control system 4 can be installed in an electronic device, and the broadcast control computer 5 can also be a device other than the electronic device.

[0063] like Figure 2 The diagram shows a flowchart of a radar-based multi-display cross-screen tracking method provided in an embodiment of this application. The radar-based multi-display cross-screen tracking method specifically includes the following steps. Depending on different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted.

[0064] S110: Create a virtual joint area based on the multiple displays.

[0065] A virtual syndicated area created from multiple displays mounted on the same plane can include multiple displays, which are then simulated and displayed in the broadcast control computer.

[0066] In an optional implementation, creating a virtual joint region based on the plurality of displays includes:

[0067] Obtain the physical width and physical height of each of the aforementioned displays;

[0068] The physical width of the virtual joint region is determined based on multiple physical widths;

[0069] Determine the maximum physical height among the multiple physical heights, and determine the physical height of the virtual union region based on the maximum physical height;

[0070] The virtual union region is created based on the physical width and the physical height.

[0071] The following example using three LED displays illustrates how to create a virtual joint area. The physical parameters of the three LED displays are shown below:

[0072]

[0073] The physical width and physical height of the display screen refer to the actual size of the display screen.

[0074] The pixel pitch refers to the center-to-center distance between the LED beads in the display screen, measured in millimeters (mm). For the same physical dimensions, a smaller pixel pitch results in a higher resolution LED display, while a larger pixel pitch results in a lower resolution. Pixel pitch is also related to the optimal viewing distance; a larger pixel pitch allows for a farther optimal viewing distance, while a smaller pixel pitch allows for a closer optimal viewing distance.

[0075] The desktop output area refers to the actual display area of ​​the LED display screen. The desktop output width is calculated by dividing the physical width of the LED display screen by its pixel pitch, and the desktop output height is calculated by dividing the physical height of the LED display screen by its pixel pitch. For example, the desktop output width of the first LED display screen is 4500 / 2.60417 = 1728, and the desktop output height of the first LED display screen is 2500 / 2.60417 = 960.

[0076] Since the three LED displays are mounted on the same plane, their desktop output positions Y are all the same, for example, all are 0. The desktop output position X of the first LED display is 0, the desktop output position X of the second LED display is determined based on the desktop output width of the first LED display, and the desktop output position X of the third LED display is determined by the combined desktop output widths of the first and second LED displays.

[0077] The outermost width of the multiple displays can be determined by their physical widths, thus defining the physical width of the virtual joint area. Similarly, the outermost height of the multiple displays can be determined by their maximum and minimum physical heights, thus defining the physical height of the virtual joint area. It should be understood that determining the physical height of the virtual joint area requires considering not only the physical height of each display but also its installation height.

[0078] After creating a virtual joint region, the coordinate system of the virtual joint region can be established simultaneously, such as... Figure 3 As shown, the rightward direction of the screen width is the X-axis, the downward direction of the screen height is the Y-axis, and the outward direction perpendicular to the screen is the Z-axis.

[0079] For example, taking the upper left corner of each LED display screen as a point, the corresponding LED display screen is mapped onto the virtual joint area. Assuming that the XY position of the first LED display screen in the virtual joint area is (0.0 m, 1.0 m), the XY position of the second LED display screen in the virtual joint area is (4.75 m, 0.0 m), and the XY position of the third LED display screen in the virtual joint area is (6.5 m, 0.75 m), then the physical width of the virtual joint area 8 created based on the first LED display screen, the second LED display screen, and the third LED display screen can be 6.5 + 3.5 = 10.0 m, and the physical height of the virtual joint area 8 can be 3 + 0.75 = 3.75 m.

[0080] In the broadcast control computer 5, a virtual unified area 8 corresponding to the display desktop is created. This virtual unified area is a simulated display program generated in the broadcast control computer, defaulting to a single large screen. However, in reality, cross-screen functionality is required, so the display area can only be divided based on the coordinates of each LED screen within the virtual unified area.

[0081] Coordinates are established for the virtual joint region 8 to simulate the display of the rendered texture object in order to be compatible with the various parameter differences of the LED display screen (including, but not limited to, installation location, pixel pitch of the display screen, etc.).

[0082] S120: Determine the radar sensing area based on the virtual joint area, wherein multiple radars are installed in the radar sensing area.

[0083] The radar sensing area is set according to the site space, pedestrian flow, and actual user needs, and is used to track target objects in the desktop output area. Preferably, a virtual cuboid area is defined as the radar sensing area 6 directly in front of the virtual combined area 8. The physical width of the radar sensing area is set according to the physical width of the virtual combined area, the physical height of the radar sensing area is set according to the physical height of the virtual combined area, and the physical thickness of the radar sensing area is freely set according to the site space.

[0084] For example, the physical width of the radar sensing area is set to 10.0 meters, the physical height of the radar sensing area is set to 2.5 meters, and the physical thickness (positive Z-axis direction) of the radar sensing area 6 is set to 2.0 meters. That is, the radar sensing area 6 is approximately 0.5 meters away from the Z-axis of the virtual combined area 8 and approximately 0.25 meters away from the bottom edge of the virtual combined area 8.

[0085] After determining the radar sensing area, a coordinate system for that radar sensing area can be established simultaneously, such as... Figure 4 As shown, the rightward direction of the screen width is the X-axis, the downward direction of the screen height is the Y-axis, and the outward direction perpendicular to the screen is the Z-axis.

[0086] In this embodiment, two radars can be installed corresponding to the radar sensing area, for example, one radar can be installed at the upper left and one at the upper right of the radar sensing area. In other embodiments, radars can also be installed at other locations, as long as the field of view of the installed radars can cover the radar sensing area. This application does not impose any restrictions here.

[0087] S130: Obtain the point cloud of the target object within the radar sensing area collected by the radar, and obtain the sensing coordinates of the target object within the radar sensing area based on the point cloud.

[0088] The target object refers to the subject that needs to be tracked across multiple displays, such as pedestrians or puppies.

[0089] When a target object enters the radar sensing area 6, the radar will collect a point cloud of the target object in real time and transmit the collected point cloud to the broadcast control computer 5. The broadcast control computer 5 obtains the sensing coordinates of the target object in the radar sensing area based on the point cloud collected by the radar 7.

[0090] In an optional implementation, obtaining the sensing coordinates of the target object in the radar sensing area based on the point cloud includes:

[0091] Obtain the installation position and angle of the radar relative to the radar sensing area;

[0092] Generate a point cloud radar coordinate transformation matrix based on the installation location and the installation angle;

[0093] Based on the point cloud radar coordinate transformation matrix and the point cloud, the sensing coordinates of the target object in the radar sensing area are obtained.

[0094] First, a point cloud radar coordinate transformation matrix is ​​generated based on the installation position and angle of the radar relative to the radar sensing area. Then, the product between the point cloud and the point cloud radar coordinate transformation matrix is ​​calculated to obtain the sensing coordinates of the target object in the radar sensing area.

[0095] S140: Process the sensed coordinates to obtain the position coordinates of the target object in the radar sensing area.

[0096] After obtaining the sensing coordinates of the target object, it is necessary to classify the sensing coordinates to obtain the position coordinates of multiple types of target objects in the radar sensing area.

[0097] In an optional implementation, processing the sensed coordinates to obtain the position coordinates of the target object within the radar sensing area includes:

[0098] The point cloud outside the radar sensing area is filtered out to obtain the target point cloud;

[0099] Based on the sensed coordinates of the target point cloud, the target point cloud is subjected to dimensionality reduction processing to obtain a two-dimensional point cloud;

[0100] The two-dimensional point cloud data is rasterized and saved.

[0101] Cluster analysis is performed on the saved two-dimensional point clouds according to the preset refresh cycle to obtain multiple types of two-dimensional point clouds;

[0102] The position coordinates of each type of target object in the radar sensing area are obtained based on the sensing coordinates of the two-dimensional point cloud for each type.

[0103] Filtering out point clouds outside the radar sensing area and retaining point clouds within the radar sensing area can effectively reduce the amount of point cloud data, thereby reducing the amount of computation.

[0104] The Z-axis coordinates of the point cloud can be directly removed, thereby achieving dimensionality reduction of the point cloud.

[0105] After reducing the dimensionality of the target point cloud to obtain a two-dimensional point cloud, the two-dimensional point cloud can be rasterized and saved according to a preset size (e.g., 10 cm * 10 cm).

[0106] A refresh cycle (e.g., 33 milliseconds) is set to periodically perform cluster analysis on the saved 2D point clouds, resulting in multiple categories of 2D point clouds. For each category, the average value is calculated, then multiplied by the grid size (10 cm * 10 cm), and then added to 0.5 times the grid size (5 cm * 5 cm) to obtain the position coordinates of the target object of that category within the radar sensing area. Here, 0.5 times represents the offset of the grid center position from the top-left corner of the grid.

[0107] It should be noted that after performing cluster analysis, the saved 2D point cloud needs to be zeroed out after a preset delay (e.g., 200 milliseconds). Since the radar only returns the coordinates of points with obstacles, the original coordinates will not be updated after the target object leaves. Zeroing out the coordinates ensures that the original coordinates are updated after the target object leaves.

[0108] S150: Based on the position coordinates of the target object in the radar sensing area, obtain the position coordinates of the target object in the virtual joint area.

[0109] Combination Figure 3 The coordinate system of the virtual joint region shown Figure 4 As shown in the coordinate system of the radar sensing area, the XYZ axes of the radar sensing area and the coordinate system of the virtual combined area are aligned. The radar sensing area is essentially shifted downwards along the Z-axis of the virtual combined area. Therefore, the position coordinates of the target object in the virtual combined area = the position coordinates of the target object in the radar sensing area + the distance between the origin of the radar sensing area's coordinate system and the origin of the virtual combined area's coordinate system.

[0110] S160: Based on the position coordinates of the target object in the virtual joint area, perform cross-screen tracking of the target object.

[0111] When the target object moves within the radar sensing area 6, the position of the target object relative to each display screen can be calculated based on the position coordinates of the target object in the virtual joint area 8 and the position of each display screen in the virtual joint area, thereby realizing cross-screen tracking of the target object.

[0112] In an optional implementation, the cross-screen tracking of the target object based on its position coordinates within the virtual union region includes:

[0113] Based on the location coordinates of the virtual joint region, calculate the distance between each type of target object in the previous refresh cycle and the target object in the current refresh cycle;

[0114] Obtain the target distance that is less than a preset distance threshold from among the multiple distances;

[0115] The target object corresponding to the previous refresh cycle and the target object corresponding to the current refresh cycle are determined to be the same object.

[0116] In the above optional implementation, after determining the position coordinates of each type of target object in the virtual joint area, the distance between each type of target object corresponding to two adjacent refresh cycles can be calculated, thereby determining whether the target objects corresponding to two adjacent refresh cycles are the same based on the distance.

[0117] For every two adjacent refresh cycles, multiple distances are calculated. When the calculated distance is greater than a preset distance threshold, it indicates that the two target objects corresponding to that distance are not the same. When the calculated distance is less than the preset distance threshold, it indicates that the two target objects corresponding to that distance are the same.

[0118] For example, the preset distance threshold can be 20 centimeters.

[0119] If the target object corresponding to two adjacent refresh cycles is determined to be the same object, then the session ID of the target object in the current refresh cycle can be set to the same session ID as the target object in the previous refresh cycle. If the target object corresponding to two adjacent refresh cycles is determined to be different, then a new session ID can be created for the target object in the current refresh cycle.

[0120] The above optional implementation method identifies the same target object in two adjacent refresh cycles by finding the nearest point of the target object in the current refresh cycle among the target objects in the previous refresh cycle, thereby realizing cross-screen tracking of the target object.

[0121] When a target object moves within the radar sensing area, the position of the target object relative to each display screen can be calculated based on the target object's position coordinates in the virtual joint area and the position of each display screen in the virtual joint area, thereby achieving cross-screen target object tracking.

[0122] In an optional implementation, the method further includes:

[0123] Based on the location coordinates of the target object in the virtual joint area, cross-screen image fusion is performed on the target object.

[0124] Building upon the achievement of cross-screen tracking of the target object across multiple displays, it is possible to further realize cross-screen image fusion of the target object across multiple displays.

[0125] Figure 5 This is a flowchart of a radar-based multi-display cross-screen image fusion method provided in Embodiment 3 of this application. The radar-based multi-display cross-screen image fusion method specifically includes the following steps. Depending on different requirements, the order of the steps in this flowchart can be changed, and some steps can be omitted.

[0126] S210: Determine the pixel size within the virtual joint region based on the region size of the virtual joint region.

[0127] After creating a virtual joint region based on multiple displays, the minimum pixel pitch in the multiple displays is determined. Then, the pixel size within the virtual joint region is obtained based on the physical size (physical width and physical height) of the virtual joint region and the minimum pixel pitch.

[0128] For example, assuming the physical width of the virtual joint area created based on multiple displays is 10.0 meters and the physical height is 3.75 meters, and the minimum pixel pitch in the multiple displays is 1.953125 millimeters, since the physical width 10.0 meters / 1.953125 millimeters = 5120 and the physical height 3.75 meters / 1.953125 millimeters = 1920, the pixel size within the virtual joint area is 5120 * 1920.

[0129] In the above embodiments, the pixel size within the virtual joint region is obtained based on the physical size of the virtual joint region and the minimum pixel pitch. This ensures that the physical size of the images displayed on the screen is consistent. The final output image ensures that the display screen with the minimum pixel pitch achieves pixel-to-pixel accuracy, while the display screens with other pixel pitches achieve fusion by scaling down the rendered image. For example, when a deer spans two display screens with different pixel pitches, the minimum pixel pitch allows the viewer to see that the deer's physical size is consistent on both screens, and the two screens can be seamlessly stitched together.

[0130] In this embodiment, the following table of data is calculated based on the pixel size of the virtual joint region:

[0131]

[0132] S220: Create a texture object with the pixel size of the virtual union region.

[0133] For example, a render texture object is created with the pixel size of the virtual union region (5120*1920).

[0134] S230: Render the texture object to obtain a rendered texture object.

[0135] Different texture objects have different rendering processes.

[0136] Taking the example of a puppy following a human body, the rendering of the texture object includes:

[0137] Create an animation list with 2 puppies, where each puppy animation object can be bound to a session ID representing a human's coordinates;

[0138] During initialization, the puppies are distributed on the left and right sides of the virtual union area, and the session ID of the bound human coordinate point is 0;

[0139] When a human coordinate point is received, the animation list is searched for a dog that is bound to the session ID of the human coordinate point. If a dog is found to be bound to the session ID of the human coordinate point, the dog is moved to the human coordinate point. If no dog is found to be bound to the session ID of the human coordinate point, the animation list is searched for a session ID that is not bound to a dog animation object and bound to it, and the dog is moved to the human coordinate point.

[0140] S240: Perform cross-screen image fusion on the target object according to the display area corresponding to the rendered texture object in each of the display screens.

[0141] The display area corresponding to the rendered texture object in each display screen can be extracted and copied to the output area of ​​the display screen on the desktop.

[0142] In an optional implementation, the step of performing cross-screen image fusion on the target object based on the display area corresponding to the rendered texture object in each of the display screens includes:

[0143] Convert the position coordinates of the rendered texture object from the virtual joint region to pixel coordinates within the virtual joint region; based on the pixel coordinates of the rendered texture object within the virtual joint region, move the rendered texture object to the coordinate position corresponding to the desktop output area;

[0144] The rendered texture object in the desktop output area is scaled according to the pixel size of the virtual union region and the physical size of the desktop output area.

[0145] The pixel coordinates of each display screen in the virtual joint area can be obtained by dividing the position coordinates of each display screen in the virtual joint area by the minimum pixel pitch in the display screen. Thus, the pixel coordinates of the rendered texture object in the virtual joint area can be obtained based on the pixel coordinates of the display screen displaying the rendered texture object in the virtual joint area.

[0146] The rendered texture object can be scaled up or down proportionally based on the ratio of the pixel size of the virtual joint region to the physical size of the desktop output region.

[0147] For example, the display area corresponding to the rendered texture object in the first LED display screen [(0, 512) = (pixel coordinates within the joint area), (2304 x 1280) = (pixel width x pixel height of the joint area)] is copied to the output area of ​​the first LED display screen on the desktop [(0, 0) = (corresponding desktop output position coordinates), (1728 x 960) = (desktop output width x output height)], and the copying process completes the screen scaling down;

[0148] Copy the display area [(2432, 0), (768x 1792)] corresponding to the rendered texture object in the second LED display to the output area [(1728, 0), (768x 1792)] of the second LED display on the desktop;

[0149] Copy the display area [(3328, 384), (1792x1536)] corresponding to the rendered texture object in the third LED display to the output area [(2496, 0), (1344x1152)] of the third LED display on the desktop. The copying process reduces the size of the image.

[0150] The synchronous LED control system sends the output area of ​​the LED display screen to the corresponding desktop to complete the display of the LED screen image.

[0151] It should be noted that the image size reduction is based on the pixel pitch; for example, (1792x 1536) is reduced to (1344x 1152).

[0152] The cross-screen image fusion method provided in this application first creates a virtual joint region, and assumes that all displays have the same pixel pitch. All displays are rendered on the same texture object at once, and then each display is displayed point-to-point by image matting and scaling, thereby achieving cross-screen image fusion.

[0153] The embodiments of the present application achieve cross-screen tracking of multiple displays, solving the limitation of only single-screen interaction in the past. At the same time, the embodiments of the present application also achieve cross-screen image fusion of multiple displays, enabling linkage between multiple displays, and the interaction effects can be seamlessly connected between multiple displays, creating a more vivid display method for the displays.

[0154] The above-disclosed are only the preferred embodiments of the present application. Of course, the scope of the rights of the present application cannot be limited thereby. Those of ordinary skill in the art can understand all or part of the processes of implementing the above embodiments, and the equivalent changes made according to the present application still fall within the scope covered by the invention.

Claims

1. A radar-based multi-display cross-screen tracking method applied to an electronic device, comprising: The method comprises: creating a virtual joint area according to a plurality of display screens; determining a radar sensing area according to the virtual joint area, the radar sensing area corresponding to a plurality of radars installed; obtaining point clouds of a target object in the radar sensing area collected by the radars, and obtaining sensing coordinates of the target object in the radar sensing area according to the point clouds; processing the sensing coordinates to obtain position coordinates of the target object in the radar sensing area; obtaining position coordinates of the target object in the virtual joint area according to the position coordinates of the target object in the radar sensing area; tracking the target object across screens based on the position coordinates of the target object in the virtual joint area.

2. The radar-based multi-display cross-screen tracking method of claim 1, wherein, The creating of the virtual joint area according to the plurality of display screens comprises: obtaining physical width and physical height of each display screen; determining the physical width of the virtual joint area according to a plurality of physical widths; determining the maximum physical height among a plurality of physical heights, and determining the physical height of the virtual joint area according to the maximum physical height; creating the virtual joint area according to the physical width and the physical height.

3. The radar-based multi-display cross-screen tracking method of claim 1, wherein, The obtaining of the sensing coordinates of the target object in the radar sensing area according to the point clouds comprises: obtaining the installation position and installation angle of the radars relative to the radar sensing area; generating a point cloud radar coordinate conversion matrix according to the installation position and the installation angle; obtaining the sensing coordinates of the target object in the radar sensing area according to the point cloud radar coordinate conversion matrix and the point clouds.

4. The radar-based multi-display cross-screen tracking method of claim 1, wherein, The processing of the sensing coordinates to obtain the position coordinates of the target object in the radar sensing area comprises: filtering out point clouds outside the radar sensing area to obtain target point clouds; performing dimension reduction processing on the target point clouds according to the sensing coordinates of the target point clouds to obtain two-dimensional point clouds; performing rasterization processing on the two-dimensional point cloud data and saving the data; performing clustering analysis on the saved two-dimensional point clouds according to a preset refresh period to obtain a plurality of categories of two-dimensional point clouds; obtaining the position coordinates of each category of target object in the radar sensing area according to the sensing coordinates of each category of two-dimensional point cloud.

5. The radar-based multi-display cross-screen tracking method of claim 1, wherein, The tracking of the target object across screens based on the position coordinates of the target object in the virtual joint area comprises: calculating the distance between each category of target object corresponding to a previous refresh period and the target object corresponding to a current refresh period based on the position coordinates of the virtual joint area; obtaining target distances smaller than a preset distance threshold among a plurality of distances; determining the target object corresponding to the previous refresh period corresponding to the target distance and the target object corresponding to the current refresh period as the same object.

6. The radar-based multi-display cross-screen tracking method according to any one of claims 1 to 5, characterized in that, The method further comprises: performing cross-screen picture fusion on the target object according to the position coordinates of the target object in the virtual joint area.

7. The radar-based multi-display cross-screen tracking method of claim 6, wherein, The cross-screen picture fusion on the target object according to the position coordinates of the target object in the virtual joint area comprises: determining the pixel size of the virtual joint area according to the area size of the virtual joint area; creating a texture object with the virtual joint area pixel size: rendering the texture object to obtain a rendered texture object; performing cross-screen picture fusion of the target object according to the display area corresponding to the rendered texture object in each display screen.

8. The radar-based multi-display cross-screen tracking method of claim 7, wherein, The cross-screen picture fusion of the target object according to the display area corresponding to the rendered texture object in each display screen includes: converting the position coordinates of the rendered texture object in the virtual joint area to pixel coordinates in the virtual joint area; moving the rendered texture object to the coordinate position corresponding to the desktop output area according to the pixel coordinates of the rendered texture object in the virtual joint area; scaling the rendered texture object in the desktop output area according to the pixel size of the virtual joint area and the physical size of the desktop output area.

9. The radar-based multi-display cross-screen tracking method of claim 8, wherein, The conversion of the position coordinates of the rendered texture object in the virtual joint area to pixel coordinates in the virtual joint area includes: obtaining the pixel coordinates of each display screen in the virtual joint area by dividing the position coordinates of each display screen in the virtual joint area by the smallest point spacing in the plurality of display screens; obtaining the pixel coordinates of the rendered texture object in the virtual joint area according to the pixel coordinates of each display screen in the virtual joint area.

10. A radar-based multi-display cross-screen tracking system, comprising: The system includes: a plurality of display screens installed on the same plane; an electronic device connected to the plurality of display screens; and a plurality of radars connected to the electronic device; wherein the electronic device is configured to perform the steps of the radar-based multi-display screen cross-screen tracking method according to any one of claims 1 to 9.

Images