A method and device for forming an image matrix in a live environment

Abstract

The purpose of the present application is to provide a method and device for forming an image matrix in a live environment, the method comprising: obtaining the position information of each user device among a plurality of user devices participating in forming the image matrix in the live environment; determining a target image, and dividing the target image into a plurality of target image units according to the position information of each user device, wherein each target image unit corresponds to a different user device among the plurality of user devices; for each target image unit, generating an image presentation instruction corresponding to the target image unit, and sending the target image unit and the image presentation instruction to the user device corresponding to the target image unit, so that the plurality of target image units simultaneously presented in the plurality of user devices form an image matrix corresponding to the target image.

Description

Technical Field

[0001] The present application relates to the field of communications, and in particular to a technology for forming an image matrix in a live environment. Background Art

[0002] In the prior art, in some performance venues, the light-emitting props such as glow sticks held by the audience can be uniformly controlled to display a complete image by manually changing the lights. Alternatively, the complete image can be displayed through multiple display devices by manually segmenting the image or manually assigning different colors. Summary of the invention

[0003] An object of the present application is to provide a method and device for forming an image matrix in a live environment.

[0004] According to one aspect of the present application, a method for controlling a device to form an image matrix in a live environment is provided, the method comprising:

[0005] Acquire location information of each user device among a plurality of user devices participating in forming an image matrix in a live environment;

[0006] Determine a target image, and divide the target image into a plurality of target image units according to the location information of each user device, wherein each target image unit corresponds to a different user device among the plurality of user devices;

[0007] For each target image unit, an image presentation instruction corresponding to the target image unit is generated, and the target image unit and the image presentation instruction are sent to a user device corresponding to the target image unit, so that the multiple target image units simultaneously presented in the multiple user devices constitute an image matrix corresponding to the target image.

[0008] According to another aspect of the present application, a method for forming an image matrix in a live environment by a user device is provided, the method comprising:

[0009] Receiving target image units obtained by segmenting a target image and image presentation instructions corresponding to the target image units sent by a control device;

[0010] The target image unit is presented according to the image presentation instruction, wherein the target image unit and other target image units obtained by segmenting the target image and presented simultaneously in other multiple user devices constitute an image matrix corresponding to the target image.

[0011] According to one aspect of the present application, a control device for forming an image matrix in a live environment is provided, the device comprising:

[0012] A module for obtaining location information of each user device among a plurality of user devices participating in forming an image matrix in a live environment;

[0013] A module one or two, configured to determine a target image, and divide the target image into a plurality of target image units according to the location information of each user device, wherein each target image unit corresponds to a different user device among the plurality of user devices;

[0014] A module 13 is used to generate an image presentation instruction corresponding to each target image unit, and send the target image unit and the image presentation instruction to the user device corresponding to the target image unit, so that the multiple target image units simultaneously presented in the multiple user devices constitute an image matrix corresponding to the target image.

[0015] According to another aspect of the present application, there is provided a user device for forming an image matrix in a live environment, the device comprising:

[0016] A module 21 is used to receive target image units obtained by segmenting a target image and image presentation instructions corresponding to the target image units sent by a control device;

[0017] Module 22 is used to present the target image unit according to the image presentation instruction, wherein the target image unit and other target image units obtained by segmenting the target image and presented simultaneously in other multiple user devices constitute an image matrix corresponding to the target image.

[0018] According to one aspect of the present application, a device for forming an image matrix in a live environment is provided, wherein the device comprises:

[0019] Processor; and

[0020] A memory arranged to store computer executable instructions which, when executed, cause the processor to perform the following operations:

[0021] Acquire location information of each user device among a plurality of user devices participating in forming an image matrix in a live environment;

[0022] Determine a target image, and divide the target image into a plurality of target image units according to the location information of each user device, wherein each target image unit corresponds to a different user device among the plurality of user devices;

[0023] For each target image unit, an image presentation instruction corresponding to the target image unit is generated, and the target image unit and the image presentation instruction are sent to a user device corresponding to the target image unit, so that the multiple target image units simultaneously presented in the multiple user devices constitute an image matrix corresponding to the target image.

[0024] According to another aspect of the present application, a device for forming an image matrix in a live environment is provided, wherein the device comprises:

[0025] Processor; and

[0026] A memory arranged to store computer executable instructions which, when executed, cause the processor to perform the following operations:

[0027] Receiving target image units obtained by segmenting a target image and image presentation instructions corresponding to the target image units sent by a control device;

[0028] The target image unit is presented according to the image presentation instruction, wherein the target image unit and other target image units obtained by segmenting the target image and presented simultaneously in other multiple user devices constitute an image matrix corresponding to the target image.

[0029] According to one aspect of the present application, a computer readable medium storing instructions is provided, wherein when the instructions are executed, the system performs the following operations:

[0030] Acquire location information of each user device among a plurality of user devices participating in forming an image matrix in a live environment;

[0031] Determine a target image, and divide the target image into a plurality of target image units according to the location information of each user device, wherein each target image unit corresponds to a different user device among the plurality of user devices;

[0032] For each target image unit, an image presentation instruction corresponding to the target image unit is generated, and the target image unit and the image presentation instruction are sent to a user device corresponding to the target image unit, so that the multiple target image units simultaneously presented in the multiple user devices constitute an image matrix corresponding to the target image.

[0033] According to another aspect of the present application, a computer readable medium storing instructions is provided, wherein when the instructions are executed, the system performs the following operations:

[0034] Receiving target image units obtained by segmenting a target image and image presentation instructions corresponding to the target image units sent by a control device;

[0035] The target image unit is presented according to the image presentation instruction, wherein the target image unit and other target image units obtained by segmenting the target image and presented simultaneously in other multiple user devices constitute an image matrix corresponding to the target image.

[0036] Compared with the prior art, the present application can divide the target image into multiple target image units according to the acquired position information by acquiring the position information of each user device among the multiple user devices participating in forming the image matrix in the on-site environment. By presenting the corresponding target image unit in each user device, the multiple user devices can form an image matrix corresponding to the target image, which will make people feel that a huge complete image is presented in the on-site environment as a whole, so that a visual effect of presenting a complete image in the on-site environment can be created by forming the image matrix, which can enhance the user's visual experience; further, by uniformly controlling what target image units should be presented on the screen of each user device participating in forming the image matrix and how to present the target image units, the efficiency and flexibility of presenting the complete image by forming the image matrix in the on-site environment can be improved, and there is no need to provide additional display devices in the on-site environment. The image matrix can be presented by the user devices carried by the on-site users themselves, which can greatly save the cost of hardware resources and improve the participation and sense of involvement of the on-site users. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0038] Figure 1 A flow chart of a method for forming an image matrix in a field environment applied to a control device according to an embodiment of the present application is shown;

[0039] Figure 2 A flow chart of a method for forming an image matrix in a live environment applied to a user device according to an embodiment of the present application is shown;

[0040] Figure 3 A flow chart of a system method for forming an image matrix in a live environment according to an embodiment of the present application is shown;

[0041] Figure 4 A structural diagram of a control device for forming an image matrix in a field environment according to an embodiment of the present application is shown;

[0042] Figure 5 A structural diagram of a user device forming an image matrix in a field environment according to an embodiment of the present application is shown;

[0043] Figure 6 An exemplary system is shown that can be used to implement the various embodiments described in this application.

[0044] The same or similar reference numerals in the drawings represent the same or similar components. DETAILED DESCRIPTION

[0045] The present application is described in further detail below in conjunction with the accompanying drawings.

[0046] In a typical configuration of the present application, the terminal, the device of the service network and the trusted party all include one or more processors (eg, a central processing unit (CPU)), an input / output interface, a network interface and a memory.

[0047] Memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash memory. Memory is an example of a computer-readable medium.

[0048] Computer readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer readable instructions, data structures, modules of programs or other data. Examples of computer storage media include, but are not limited to, Phase-Change Memory (PCM), Programmable Random Access Memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of random access memory (RAM), Read-Only Memory (ROM), Electrically-Erasable Programmable Read-Only Memory (EEPROM), Flash memory or other memory technology, Compact Disc Read-Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0049] The devices referred to in this application include but are not limited to user devices, network devices, or devices formed by integrating user devices and network devices through a network. The user devices include but are not limited to any mobile electronic products that can interact with users (for example, interact with users through a touchpad), such as smart phones, tablet computers, etc. The mobile electronic products can use any operating system, such as Android operating system, iOS operating system, etc. Among them, the network device includes an electronic device that can automatically perform numerical calculations and information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable logic devices (Programmable Logic Device, PLD), field programmable gate arrays (Field Programmable Gate Array, FPGA), digital signal processors (Digital Signal Processor, DSP), embedded devices, etc. The network devices include but are not limited to computers, network hosts, single network servers, multiple network server sets or multiple servers. The cloud is composed of a large number of computers or network servers based on cloud computing (Cloud Computing), wherein cloud computing is a type of distributed computing, a virtual supercomputer composed of a group of loosely coupled computer sets. The network includes but is not limited to the Internet, a wide area network, a metropolitan area network, a local area network, a VPN network, a wireless self-organizing network (Ad Hoc network), etc. Preferably, the device may also be a program running on the user device, the network device, or a device formed by integrating the user device and the network device, the network device, the touch terminal, or the network device and the touch terminal through a network.

[0050] Of course, those skilled in the art should understand that the above-mentioned devices are only examples, and other existing or future devices that are applicable to the present application should also be included in the scope of protection of the present application and are included here by reference.

[0051] In the description of the present application, “plurality” means two or more, unless otherwise clearly and specifically defined.

[0052] Figure 1A flow chart of a method for forming an image matrix in a live environment for a control device according to an embodiment of the present application is shown, and the method includes step S11, step S12 and step S13. In step S11, the control device obtains the position information of each user device in the live environment that participates in forming the image matrix; in step S12, the control device determines a target image, and divides the target image into a plurality of target image units according to the position information of each user device, wherein each target image unit corresponds to a different user device in the plurality of user devices; in step S13, the control device generates an image presentation instruction corresponding to each target image unit, and sends the target image unit and the image presentation instruction to the user device corresponding to the target image unit, so that the plurality of target image units simultaneously presented in the plurality of user devices form an image matrix corresponding to the target image.

[0053] In step S11, the control device obtains the location information of each user device in the multiple user devices participating in forming the image matrix in the scene environment. In some embodiments, the scene environment can be an indoor environment or an outdoor environment. In some embodiments, the target image is divided into multiple target image units, and different target image units are simultaneously presented on the screen of each user device held by the user participating in forming the image matrix in the scene environment. The different target image units obtained by segmenting the target image and simultaneously presented on the screens of multiple user devices constitute an image matrix, which makes people feel that a huge complete image is presented in the scene environment as a whole, so that a visual effect of presenting a complete image in the scene environment can be created by forming the image matrix, which can enhance the user's visual experience. In some embodiments, it is necessary to obtain the position information of each user device in the multiple user devices participating in the image matrix in the on-site environment, so as to assign the target image unit matching the position information of each user device to the user device for display. The position information of each user device can be the relative position coordinates of the user device in the on-site environment coordinate system, or a point or an area corresponding to the user device in the on-site environment position map, or the relative position information of the user device in all the multiple user devices participating in the image matrix (for example, all the multiple user devices participating in the image matrix have a total of 4 rows and 5 columns, and the user device D1 is in the upper left corner). In some embodiments, the control device can be a special user device that does not participate in the image matrix, or a network device, which is used to uniformly control what target image unit should be presented on the screen of each user device participating in the image matrix and how to present the target image unit, which can improve the efficiency and flexibility of presenting a complete image by forming an image matrix in the on-site environment, and does not require the on-site environment to provide additional display devices. The image matrix can be presented by the user device (for example, a mobile phone) carried by the on-site user himself, which can greatly save the cost of hardware resources and improve the participation and sense of immersion of the on-site user.

[0054] In step S12, the control device determines the target image, and divides the target image into a plurality of target image units according to the location information of each user device, wherein each target image unit corresponds to a different user device among the plurality of user devices. In some embodiments, according to the location information of each user device, the target image unit corresponding to the location information is found in the target image, so that according to the location information of the plurality of user devices, the target image is divided into a plurality of target image units, each user device corresponds to a different target image unit, and each target image unit corresponds to a different user device. In some embodiments, for each user device, according to the location information of the device, the user device is mapped to a corresponding pixel area on the target image, each user device is mapped to a different pixel area, and the different pixel areas do not overlap with each other, and the target image is divided according to the plurality of pixel areas mapped by the plurality of user devices, and the target image unit corresponding to each pixel area of ​​the target image is obtained, thereby achieving the division of the target image into a plurality of target image units, each target image unit corresponding to a different user device. For example, the location information of the user device D1 is "all the multiple user devices participating in forming the image matrix have a total of 4 rows and 5 columns, and the user device D1 is in the 1st row and 3rd column in the upper left corner", so the target image is equally divided into 4 rows and 5 columns, a total of 20 different pixel areas, and according to "the user device D1 is in the 1st row and 3rd column in the upper left corner", the user device is mapped to the pixel area Area1 located in the 1st row and 3rd column in the upper left corner of the 20 pixel areas, and the target image unit corresponding to the target image in the pixel area Area1 is determined as the target image unit corresponding to the user device D1.

[0055] In step S13, the control device generates an image presentation instruction corresponding to each target image unit, and sends the target image unit and the image presentation instruction to the user device corresponding to the target image unit, so that the multiple target image units simultaneously presented in the multiple user devices constitute the image matrix corresponding to the target image. In some embodiments, the image presentation instruction corresponding to each target image unit may include but is not limited to a start presentation time, an end presentation time, screen setting information, and presentation configuration information, wherein the start presentation time is used to indicate when the user device corresponding to the target image unit starts to present the target image unit, the end presentation time is used to indicate when the user device starts to end presenting the target image unit, the screen setting information is used to indicate how the user device sets the screen of the user device, a part of the screen setting information can be directly set automatically by the user device, for example, screen brightness, and a part of the screen setting information needs to be presented on the user device screen to prompt the user to manually set, for example, horizontal screen or vertical screen, and the presentation configuration information is determined according to the device posture information of each user device (for example, the rotation angle of the mobile phone), and is used to indicate the configuration information of the user device when rendering and presenting the target image unit (for example, the rotation matrix of the target image unit). In some embodiments, the target image unit and the image presentation instruction corresponding to each user device may be sent to the user device together, or the target image unit and the image presentation instruction may be sent to the user device separately.

[0056] In some embodiments, the method further includes before step S11: the control device sends a request message for requesting to participate in forming the image matrix to each of the multiple pending user devices in the on-site environment; and determines the multiple user devices participating in forming the image matrix according to the feedback information of at least one of the multiple pending user devices in response to the request message, wherein the feedback information is used to indicate whether the at least one pending user device participates in forming the image matrix. In some embodiments, the control device sends a request message for requesting to participate in forming the image matrix to each pending user device in the on-site environment, and each user can perform a corresponding feedback operation for the request information, generate feedback information (agree to participate in forming the image matrix or refuse to participate in forming the image matrix) and return it to the control device. In some embodiments, there is a predetermined feedback period for the feedback operation. After the feedback period is exceeded, the user can no longer perform the feedback operation for the request information, or the user can continue to perform the feedback operation for the request information, but the user device will not generate feedback information and send it to the control device, or the control device will no longer process the feedback information received outside the feedback period. In some embodiments, after a predetermined feedback deadline is reached, the control device determines the pending user device that has fed back consent among at least one pending user device that has currently fed back as a user device participating in forming the image matrix, and for one or more pending user devices that have not currently fed back, they can be processed by default as agreeing to participate in forming the image matrix, or they can be processed by default as refusing to participate in forming the image matrix.

[0057] In some embodiments, the step S11 includes: for each user device, the control device receives the location identification information sent by the user device, and determines the location information corresponding to the user device according to the location identification information and the preset location mapping relationship information, wherein the preset location mapping relationship information is used to indicate the mapping relationship between the predetermined location identification and the location information. In some embodiments, for each user device, the location identification information of the user device is obtained and sent to the control device, the location identification information is used to uniquely identify the location of each user device, and the location identification information of each user device is different. For example, the user device can prompt the user to enter the seat number of each seat into the user device and send it to the control device, where the seat number is used as the location identification information of each user device. In some embodiments, the control device receives the location identification information sent by each user device, presets the location mapping relationship information, and determines the location information corresponding to the location identification information, wherein the preset location mapping relationship information is used to map the location identification information to the location information, for example, the preset location mapping relationship information can be a location map of the on-site environment, and each seat number in the on-site environment is marked in the location map. Through the seat number sent by the user device, the location information corresponding to the user device can be found in the location map.

[0058] In some embodiments, the method further includes: the control device generates device identification information corresponding to each user device for each user device, and sends the device identification information to the user device. In some embodiments, the control device assigns a different device identification information to each user device, and the device identification information is used to uniquely identify each user device. For example, the device identification information can be an integer starting from 0 and increasing in step 1. The device identification information of user device D1 is 0, the device identification information of user device D2 is 1, and so on. In some embodiments, after receiving the device identification information assigned by the control device to itself, each user device will present the device identification information on the screen. Users can lift up their respective user devices and keep the screen in a lit state so that the device identification information on the screen of each user device can be seen in the on-site photos taken by drones or other shooting equipment.

[0059] In some embodiments, the step S11 includes: obtaining a live photo corresponding to the live environment, and determining the location information of each user device by identifying the device identification information corresponding to each user device in the live photo. In some embodiments, a drone or other shooting device takes a live photo corresponding to the live environment and sends it to a control device. The control device identifies the device identification information on the screen of each user device in the live photo, and according to the mapping relationship between the user device and the device identification information, the location information of each user device can be determined. For example, from the live photo, it can be obtained that all the multiple user devices participating in the image matrix have a total of 4 rows and 5 columns, and the device identification information on the screen of the user device in the 1st row and 3rd column in the upper left corner is "D1". Since the device identification information "D1" corresponds to the user device D1, it can be determined that the location information corresponding to the user device D1 is "the 1st row and 3rd column in the upper left corner".

[0060] In some embodiments, the method further includes: the control device generates screen flickering rule information corresponding to each user device, and sends the screen flickering rule information to the user device. In some embodiments, the control device assigns a unique and different screen flickering rule information to each user device, and the screen flickering rule information is used to define the screen flickering rule of each user device. For example, the screen flickering rule information of the user device D1 is "on, off, on, on, off, and the time interval between each screen flickering state switch is 0.2 seconds, and this cycle is repeated". In some embodiments, after receiving the screen flickering rule information assigned by the control device to itself, each user device will flicker its own screen according to the screen flickering rule information. Continuing with the above example, the user device D1 flickers the screen according to the screen flickering rule information according to "on-interval 0.2 seconds-off-interval 0.2 seconds-on-interval 0.2 seconds-on-interval 0.2 seconds-off, and this cycle". Users can lift up their respective user devices so that the screen flickering rule information of each user device screen can be distinguished in the live video captured by the drone or other shooting equipment.

[0061] In some embodiments, the step S11 includes: the control device obtains the on-site video corresponding to the on-site environment, and determines the location information corresponding to each user device by identifying the screen flickering rule information corresponding to each user device in the on-site video. In some embodiments, the drone or other shooting device obtains the on-site video corresponding to the on-site environment and sends it to the control device. The control device can determine the location information of each user device by identifying the screen flickering rule information of each user device screen in the on-site photo, and according to the mapping relationship between the user device and the screen flickering rule information. For example, it can be obtained from the on-site video that all the multiple user devices participating in the formation of the image matrix have a total of 4 rows and 5 columns, and the screen flickering rule information of the user device in the 1st row and 3rd column in the upper left corner is "on, off, on, off, the time interval of each screen flickering state switching is 0.2 seconds, and this cycle", and since the screen flickering rule information corresponds to the user device D1, it can be determined that the location information corresponding to the user device D1 is "the 1st row and the 3rd column in the upper left corner".

[0062] In some embodiments, the determining of the target image includes: determining the device arrangement information corresponding to the multiple user devices according to the location information of each user device; and determining the target image matching the device arrangement information according to the device arrangement information. In some embodiments, the device arrangement information is used to indicate the overall layout information of the multiple user devices participating in forming the image matrix, including but not limited to any information related to the overall layout, such as layout shape information (e.g., rectangle, circle, triangle), layout aspect ratio information (e.g., the aspect ratio of the overall rectangular layout is 1:2), layout angle information (e.g., the angles of the overall triangle layout are 30 degrees, 60 degrees, 90 degrees respectively), layout arrangement information (e.g., how many user devices are there in the horizontal arrangement direction, how many user devices are there in the vertical arrangement direction), etc. In some embodiments, according to the device arrangement information of the multiple user devices, the target image matching the device arrangement information is determined, so as to improve the overall visual experience of presenting the target image by forming the image matrix. For example, if the device arrangement information includes layout shape information of a square, then the target image matching the device arrangement information is determined to be a square image; if the device arrangement information includes layout aspect ratio information of 1:2, then the target image matching the device arrangement information is determined to be a rectangular image with an aspect ratio of 1:2.

[0063] In some embodiments, the target image is segmented into a plurality of target image units according to the location information of each user device, including step S14 (not shown). In step S14, the control device maps each user device to a pixel area on the target image according to the location information of each user device, and segments the target image according to the multiple pixel areas obtained by mapping to obtain a plurality of target image units, wherein each user device is mapped to a different pixel area, each pixel area corresponds to a target image unit, and different pixel areas do not overlap with each other. In some embodiments, for each user device, according to the location information of the device, the user device is mapped to a corresponding pixel area on the target image, each user device is mapped to a different pixel area, and different pixel areas do not overlap with each other, and the target image is segmented according to the multiple pixel areas mapped by the multiple user devices to obtain the target image unit corresponding to each pixel area of ​​the target image, thereby achieving segmentation of the target image into a plurality of target image units, each target image unit corresponding to a different user device. For example, the location information of the user device D1 is "all the multiple user devices participating in forming the image matrix have a total of 4 rows and 5 columns, and the user device D1 is in the 1st row and 3rd column in the upper left corner", so the target image is equally divided into 4 rows and 5 columns, a total of 20 different pixel areas, and according to "the user device D1 is in the 1st row and 3rd column in the upper left corner", the user device is mapped to the pixel area Area1 located in the 1st row and 3rd column in the upper left corner of the 20 pixel areas, and the target image unit corresponding to the target image in the pixel area Area1 is determined as the target image unit corresponding to the user device D1.

[0064] In some embodiments, step S14 includes step S15 (not shown). In step S15, the control device maps each user device to a pixel area on the target image according to the location information of each user device and the pixel area division rule, and divides the target image according to the mapped multiple pixel areas to obtain multiple target image units; wherein the method further includes: the control device determines the pixel area division rule corresponding to the target image according to the device arrangement information corresponding to the multiple user devices. In some embodiments, the pixel area division rule is used to indicate how to divide the pixel area for the target image. The pixel area division rule can be determined according to the overall layout information of the multiple user devices participating in the image display matrix indicated by the device arrangement information. For example, if the device arrangement information indicates that the layout shape of the multiple user devices is a rectangle, there are M user devices in the horizontal arrangement direction and N user devices in the vertical arrangement direction, and according to the aspect ratio X:Y of the target image, the pixel area division rule corresponding to the target image can be to divide the pixel area using a rectangle with an aspect ratio of X / M:Y / N, and the target image is divided into M pixel areas in the horizontal direction and N pixel areas in the vertical direction. In some embodiments, a user device is uniquely mapped to a pixel area in the target image according to its position information, and the relative position of the user device among the multiple user devices is consistent with the relative position of the pixel area to which the user device is mapped in the target image. For example, a user device arranged at the upper left vertex among the multiple user devices is mapped to a pixel area arranged at the upper left vertex of the target image. In some embodiments, the number of pixel regions obtained by dividing the target image in the horizontal direction is equal to the number of user devices in the horizontal arrangement direction in the device arrangement information. Similarly, the number of pixel regions obtained by dividing the target image in the vertical direction is equal to the number of user devices in the vertical arrangement direction in the device arrangement information. For example, the device arrangement information indicates that the layout shape of the multiple user devices is a rectangle, there are 20 user devices in the horizontal arrangement direction, and there are 30 user devices in the vertical arrangement direction. Therefore, the target image is divided into 20 pixel regions in the horizontal direction, and the target image is divided into 30 pixel regions in the vertical direction. According to the aspect ratio of the target image of 1:2, it can be determined that each pixel region is a rectangular region with an aspect ratio of 1 / 20:2 / 30 (i.e., 3:4). The position information of the user device D1 is the 2nd row and 3rd column in the upper left corner. Therefore, the pixel region on the target image mapped by the user device D1 is the second rectangular region in the horizontal direction and the third rectangular region in the vertical direction in the upper left corner with an aspect ratio of 3:4. The target image unit corresponding to the target image in the rectangular region is determined as the target image unit corresponding to the user device D1.

[0065] In some embodiments, the step S15 includes: the control device maps each user device to a pixel area on the target image according to the location information of each user device and the pixel area division rule, wherein, for each user device, the area size of the pixel area mapped by the user device matches the screen size information corresponding to the user device; the target image is segmented according to the multiple pixel areas obtained by mapping to obtain multiple target image units; wherein, the method further includes: obtaining the screen size information of each user device. In some embodiments, each user device can obtain the screen size information of the user device by reading the setting information of the user device and send it to the control device. In some embodiments, the size of the pixel area on the target image mapped by user devices with different screen sizes is proportional to the screen size of the user device. For example, the larger the screen size of the user device, the larger the size of the pixel area on the target image mapped by the user device, and the larger the size of the target image unit corresponding to the pixel area of ​​the target image, thereby presenting more image content on the screen of the user device, thereby fully utilizing the screen display space of each user device; further, the aspect ratio of the pixel area on the target image mapped by user devices with different screen aspect ratios is equal to or close to the screen aspect ratio of the user device, and the image aspect ratio of the target image unit corresponding to the pixel area of ​​the target image is also equal to or close to the screen aspect ratio of the user device, thereby presenting image content that is more in line with the screen aspect ratio of the user device on the screen of the user device, thereby avoiding or reducing the image distortion problem caused by unequal scaling and stretching of the target image unit on the screen of the user device.

[0066] In some embodiments, the image presentation instruction corresponding to each target image unit includes start presentation time information and end presentation time information of the target image unit. In some embodiments, the start presentation time is used to indicate when the user device corresponding to the target image unit starts presenting the target image unit, and the end presentation time is used to indicate when the user device starts and ends presenting the target image unit.

[0067] In some embodiments, the graphic rendering instructions corresponding to each target image unit also include rendering configuration information of the target image unit; wherein the method further includes: receiving device posture information sent by a user device corresponding to the target image unit; and determining, based on the device posture information, rendering configuration information of the user device corresponding to the target image unit when presenting the target image unit. In some embodiments, each user device obtains the device posture information of the user device based on a sensor module on the user device, such as a gyroscope, and sends it to the control device, such as the rotation angle of the mobile phone, including the pitch angle (Pitch), the yaw angle (Yaw), and the roll angle (Roll). Then the control device determines the presentation configuration information of the user device when presenting the target image unit corresponding to the user device based on the device posture information of each user device, wherein the presentation configuration information is used to indicate the configuration information of each user device when rendering the corresponding target image unit. For example, the user device can determine the rotation matrix (Rotation Matrix) corresponding to the target image unit based on the rotation angle of the mobile phone in the device posture information, so that when rendering the target image unit, the target image unit can be rotated and transformed according to the corresponding rotation matrix, thereby avoiding deformation and distortion of the overall presentation effect of the target image caused by the device postures of each user device participating in forming the image matrix, and at the same time enabling the image matrix to present a three-dimensional effect.

[0068] In some embodiments, the graphic presentation instruction corresponding to each target image unit also includes screen setting information of the user device corresponding to the target image unit. In some embodiments, the screen setting information is used to instruct the user device how to set the screen of the user device. A part of the screen setting information can be automatically set by the user device, such as screen brightness, and a part of the screen setting information needs to be presented on the screen of the user device to prompt the user to set it manually, such as horizontal or vertical screen.

[0069] Figure 2 A flow chart of a method for forming an image matrix in a live environment applied to a user device according to an embodiment of the present application is shown, the method comprising step S21 and step S22. In step S21, the user device receives a target image unit obtained by segmenting a target image and an image presentation instruction corresponding to the target image unit sent by a control device; in step S22, the user device presents the target image unit according to the image presentation instruction, wherein the target image unit and other target image units obtained by segmenting the target image presented simultaneously in other multiple user devices form an image matrix corresponding to the target image.

[0070] In step S21, the user device receives the target image unit obtained by segmenting the target image and the image presentation instruction corresponding to the target image unit sent by the control device. In some embodiments, the control device finds the target image unit corresponding to the position information in the target image according to the position information of each user device, thereby segmenting the target image into multiple target image units according to the position information of multiple user devices, each user device corresponds to a different target image unit, and each target image unit corresponds to a different user device. In some embodiments, the image presentation instruction corresponding to the target image unit may include but is not limited to the start presentation time, the end presentation time, the screen setting information and the presentation configuration information, wherein the start presentation time is used to indicate when the user device starts to present the target image unit, the end presentation time is used to indicate when the user device starts to end presenting the target image unit, the screen setting information is used to indicate how the user device sets the screen, part of the screen setting information can be directly set automatically by the user device, for example, the screen brightness, and part of the screen setting information needs to be presented on the user device screen to prompt the user to manually set it, for example, horizontal screen or vertical screen, and the presentation configuration information is determined by the control device according to the device posture information of the user device (for example, the rotation angle of the mobile phone), and is used to indicate the configuration information of the user device when rendering and presenting the target image unit (for example, the rotation matrix of the target image unit). In some embodiments, the configuration information in the image presentation instruction can also be determined by the user device according to its own device posture information, without sending its own device posture information to the control device for the control device to determine the presentation configuration information and send it to the user device. In some embodiments, the control device can send the target image unit corresponding to the user device and the image presentation instruction together to the user device, and can also send the target image unit and the image presentation instruction separately to the user device.

[0071] In step S22, the user device presents the target image unit according to the image presentation instruction, wherein the target image unit and other target image units obtained by segmenting the target image and presented simultaneously in other multiple user devices constitute an image matrix corresponding to the target image. In some embodiments, the user device renders and presents the target image unit on the screen of the user device according to the image presentation instruction. In some embodiments, by simultaneously presenting different target image units on the screen of each user device held by a user participating in forming the image matrix in the on-site environment, the different target image units obtained by segmenting the target image and presented simultaneously on the screens of multiple user devices constitute an image matrix, which makes people feel as if a huge complete image is presented in the on-site environment as a whole, thereby being able to visually create an effect of presenting a complete image in the on-site environment by forming an image matrix, and can enhance the user's visual experience. In some embodiments, a control device is used to uniformly control what target image units should be presented on the screen of each user device participating in forming an image matrix and how to present the target image units. This can improve the efficiency and flexibility of presenting a complete image by forming an image matrix in a live environment. There is no need to provide additional display devices in the live environment. The image matrix can be presented through user devices (e.g., mobile phones) carried by the live users themselves, which can greatly save hardware resource costs and improve the participation and sense of involvement of live users.

[0072] In some embodiments, the method further includes: the user device receives the request information sent by the control device for requesting to participate in forming the image matrix; in response to the feedback trigger operation performed by the user on the request information, feedback information is generated and sent to the control device, wherein the feedback information is used to indicate whether the user device participates in forming the image matrix. In some embodiments, the control device sends a request information for requesting to participate in forming the image matrix to each pending user device in the on-site environment, and each user can perform a corresponding feedback operation on the request information, generate feedback information (agree to participate in forming the image matrix or refuse to participate in forming the image matrix) and return it to the control device. In some embodiments, there is a predetermined feedback period for the feedback operation. After the feedback period is exceeded, the user can no longer perform the feedback operation on the request information, or the user can continue to perform the feedback operation on the request information, but the user device will not generate feedback information and send it to the control device, or the control device will no longer process the feedback information received outside the feedback period.

[0073] In some embodiments, the method further includes: the user device obtains location identification information corresponding to the user device, and sends the location identification information to the control device. In some embodiments, the location identification information is used to uniquely identify the location of each user device, and the location identification information of each user device is different. For example, the user device can prompt the user to enter the seat number of each seat into the user device and send it to the control device. Here, the seat number is used as the location identification information of the user device.

[0074] In some embodiments, the method further includes: the user device receives the device identification information corresponding to the user device and sent by the control device, and presents the device identification information. In some embodiments, the control device assigns a different device identification information to each user device, and the device identification information is used to uniquely identify each user device. For example, the device identification information can be an integer starting from 0 and increasing in step 1. The device identification information of user device D1 is 0, the device identification information of user device D2 is 1, and so on. In some embodiments, after receiving the device identification information assigned by the control device to itself, the user device will present the device identification information on the screen. Users can lift up their respective user devices and keep the screen in a lit state so that the device identification information on the screen of each user device can be seen in the on-site photos taken by drones or other shooting equipment.

[0075] In some embodiments, the method further includes: the user device receives the screen flickering rule information corresponding to the user device and sent by the control device, and controls the screen of the user device to flicker according to the screen flickering rule information. In some embodiments, the control device assigns a unique and different screen flickering rule information to each user device, and the screen flickering rule information is used to define the screen flickering rule of each user device. For example, the screen flickering rule information is "on, off, on, on, off, and the time interval between each screen flickering state switching is 0.2 seconds, and this cycle is repeated". In some embodiments, after receiving the screen flickering rule information assigned by the control device to itself, the user device will flicker its respective screen according to the screen flickering rule information. Continuing with the above example, the user device flickers the screen according to the screen flickering rule information according to "on-interval 0.2 seconds-off-interval 0.2 seconds-on-interval 0.2 seconds-on-interval 0.2 seconds-off, and this cycle". Users can lift up their respective user devices so that the screen flickering rule information of each user device screen can be distinguished in the live video captured by the drone or other shooting equipment.

[0076] In some embodiments, the method further includes: the user device acquires screen size information corresponding to the user device, and sends the screen size information to the control device. In some embodiments, the user device can acquire the screen size information of the user device and send it to the control device by reading setting information.

[0077] In some embodiments, the image presentation instruction includes the start presentation time information and the end presentation time information of the target image unit; wherein the presenting the target image unit includes: starting to present the target image unit according to the start presentation time, and ending to present the target image unit according to the end presentation time. In some embodiments, the start presentation time is used to indicate when the user device corresponding to the target image unit starts to present the target image unit, and the end presentation time is used to indicate when the user device starts to end presenting the target image unit.

[0078] In some embodiments, the graphic presentation instruction also includes presentation configuration information of the target image unit; wherein, presenting the target image unit includes: presenting the target image unit according to the presentation configuration information; wherein, the method also includes: obtaining the device posture information corresponding to the user device, and sending the device posture information to the control device. In some embodiments, the user device obtains the device posture information of the user device according to the sensor module on the user device, such as a gyroscope, and sends it to the control device, and the control device determines the presentation configuration information and sends it to the user device, such as the rotation angle of the mobile phone, including the pitch angle (Pitch), the yaw angle (Yaw), and the roll angle (Roll). In some embodiments, the presentation configuration information is used to indicate the configuration information of each user device when rendering and presenting the corresponding target image unit. For example, the user device can determine the rotation matrix (Rotation Matrix) corresponding to the target image unit according to the rotation angle of the mobile phone in the device posture information, so that when rendering and presenting the target image unit, the target image unit can be rotated and transformed according to the corresponding rotation matrix, so as to avoid the deformation and distortion of the overall presentation effect of the target image caused by the device postures of each user device participating in forming the image matrix, and at the same time, the image matrix can present a three-dimensional effect. In some embodiments, the presentation configuration information may also be determined by the user device according to its own device posture information, without sending its own device posture information to the control device, so that the control device determines the presentation configuration information and sends it to the user device.

[0079] In some embodiments, the graphic presentation instruction further includes screen setting information corresponding to the target image unit; wherein the method further includes: setting the screen of the user device according to the screen setting information. In some embodiments, the screen setting information is used to instruct the user device how to set the screen, and part of the screen setting information can be directly set automatically by the user device, such as screen brightness, and part of the screen setting information needs to be presented on the user device screen to prompt the user to set it manually, such as horizontal or vertical screen.

[0080] Figure 3 A flow chart of a system method for forming an image matrix in a live environment according to an embodiment of the present application is shown.

[0081] like Figure 3 As shown, in step S31, the control device obtains the position information of each user device among the multiple user devices participating in forming the image matrix in the field environment. Step S31 is the same or similar to the aforementioned step S11 and will not be described in detail here. In step S32, the control device determines the target image, and divides the target image into multiple target image units according to the position information of each user device, wherein each target image unit corresponds to a different user device among the multiple user devices. Step S32 is the same or similar to the aforementioned step S12 and will not be described in detail here. In step S33, the control device generates an image presentation instruction corresponding to each target image unit, and sends the target image unit and the image presentation instruction to the target image unit. The user device corresponding to the unit, step S33 is the same or similar to the aforementioned step S13, which will not be repeated here; in step S34, the first user device receives the first target image unit obtained by segmenting the target image and the first image presentation instruction corresponding to the first target image unit sent by the control device, step S34 is the same or similar to the aforementioned step S21, which will not be repeated here; in step S35, the first user device presents the first target image unit according to the first image presentation instruction, wherein the first target image unit and other target image units obtained by segmenting the target image presented simultaneously in other multiple user devices constitute an image matrix corresponding to the target image, step S35 is the same or similar to the aforementioned step S22, which will not be repeated here.

[0082] Figure 4A structural diagram of a control device for forming an image matrix in a field environment according to an embodiment of the present application is shown, and the device includes a first module 11, a second module 12, and a third module 13. The first module 11 is used to obtain the position information of each user device in the field environment that participates in forming the image matrix; the first module 12 is used to determine a target image, and according to the position information of each user device, divide the target image into a plurality of target image units, wherein each target image unit corresponds to a different user device in the plurality of user devices; the first module 13 is used to generate an image presentation instruction corresponding to each target image unit, and send the target image unit and the image presentation instruction to the user device corresponding to the target image unit, so that the plurality of target image units simultaneously presented in the plurality of user devices form an image matrix corresponding to the target image.

[0083] A module 11 is used to obtain the location information of each user device in the multiple user devices participating in forming the image matrix in the scene environment. In some embodiments, the scene environment can be an indoor environment or an outdoor environment. In some embodiments, the target image is divided into multiple target image units, and different target image units are simultaneously presented on the screen of each user device held by each user participating in forming the image matrix in the scene environment. The different target image units obtained by segmenting the target image and simultaneously presented on the screens of multiple user devices constitute an image matrix, which makes people feel that a huge complete image is presented in the scene environment as a whole, so that a visual experience of presenting a complete image in the scene environment can be visually created by forming the image matrix. In some embodiments, it is necessary to obtain the position information of each user device in the multiple user devices participating in the image matrix in the on-site environment, so as to assign the target image unit matching the position information of each user device to the user device for display. The position information of each user device can be the relative position coordinates of the user device in the on-site environment coordinate system, or a point or an area corresponding to the user device in the on-site environment position map, or the relative position information of the user device in all the multiple user devices participating in the image matrix (for example, all the multiple user devices participating in the image matrix have a total of 4 rows and 5 columns, and the user device D1 is in the upper left corner). In some embodiments, the control device can be a special user device that does not participate in the image matrix, or a network device, which is used to control what target image unit should be presented on the screen of each user device participating in the image matrix and how to present the target image unit, which can improve the efficiency and flexibility of presenting a complete image by forming an image matrix in the on-site environment, and does not require the on-site environment to provide additional display devices. The image matrix can be presented by the user device (for example, a mobile phone) carried by the on-site user himself, which can greatly save the cost of hardware resources and improve the participation and sense of involvement of the on-site user.

[0084] The first and second modules 12 are used to determine the target image and divide the target image into a plurality of target image units according to the location information of each user device, wherein each target image unit corresponds to a different user device among the plurality of user devices. In some embodiments, according to the location information of each user device, the target image unit corresponding to the location information is found in the target image, so that according to the location information of the plurality of user devices, the target image is divided into a plurality of target image units, each user device corresponds to a different target image unit, and each target image unit corresponds to a different user device. In some embodiments, for each user device, according to the location information of the device, the user device is mapped to a corresponding pixel area on the target image, each user device is mapped to a different pixel area, and the different pixel areas do not overlap with each other, and the target image is divided according to the plurality of pixel areas mapped by the plurality of user devices, and the target image unit corresponding to each pixel area of ​​the target image is obtained, thereby achieving the division of the target image into a plurality of target image units, each target image unit corresponding to a different user device. For example, the location information of the user device D1 is "all the multiple user devices participating in forming the image matrix have a total of 4 rows and 5 columns, and the user device D1 is in the 1st row and 3rd column in the upper left corner", so the target image is equally divided into 4 rows and 5 columns, a total of 20 different pixel areas, and according to "the user device D1 is in the 1st row and 3rd column in the upper left corner", the user device is mapped to the pixel area Area1 located in the 1st row and 3rd column in the upper left corner of the 20 pixel areas, and the target image unit corresponding to the target image in the pixel area Area1 is determined as the target image unit corresponding to the user device D1.

[0085] Module 13 is used to generate an image presentation instruction corresponding to each target image unit, and send the target image unit and the image presentation instruction to the user device corresponding to the target image unit, so that the multiple target image units simultaneously presented in the multiple user devices constitute the image matrix corresponding to the target image. In some embodiments, the image presentation instruction corresponding to each target image unit may include but is not limited to a start presentation time, an end presentation time, screen setting information, and presentation configuration information, wherein the start presentation time is used to indicate when the user device corresponding to the target image unit starts to present the target image unit, the end presentation time is used to indicate when the user device starts to end presenting the target image unit, the screen setting information is used to indicate how the user device sets the screen of the user device, part of the screen setting information can be directly set automatically by the user device, for example, screen brightness, and part of the screen setting information needs to be presented on the user device screen to prompt the user to set it manually, for example, horizontal screen or vertical screen, and the presentation configuration information is determined according to the device posture information of each user device (for example, the rotation angle of the mobile phone), and is used to indicate the configuration information of the user device when rendering and presenting the target image unit (for example, the rotation matrix of the target image unit). In some embodiments, the target image unit and the image presentation instruction corresponding to each user device may be sent to the user device together, or the target image unit and the image presentation instruction may be sent to the user device separately.

[0086] In some embodiments, the device further includes a method for: sending a request message for requesting to participate in forming an image matrix to each of the plurality of pending user devices in the on-site environment; determining the plurality of user devices participating in forming the image matrix according to feedback information of at least one of the plurality of pending user devices in response to the request message, wherein the feedback information is used to indicate whether the at least one pending user device participates in forming the image matrix. Here, the related operations are similar to Figure 1 The embodiments shown are the same or similar, so they are not described again and are incorporated herein by reference.

[0087] In some embodiments, the module 11 is used to: for each user device, control the device to receive the location identification information sent by the user device, and determine the location information corresponding to the user device according to the location identification information and the preset location mapping relationship information, wherein the preset location mapping relationship information is used to indicate the mapping relationship between the predetermined location identification and the location information. Figure 1 The embodiments shown are the same or similar, so they are not described again and are incorporated herein by reference.

[0088] In some embodiments, the device is further configured to: for each user device, generate device identification information corresponding to the user device, and send the device identification information to the user device. Figure 1 The embodiments shown are the same or similar, so they are not described again and are incorporated herein by reference.

[0089] In some embodiments, the module 11 is used to obtain a scene photo corresponding to the scene environment, and determine the location information of each user device by identifying the device identification information corresponding to each user device in the scene photo. Figure 1 The embodiments shown are the same or similar, so they are not described again and are incorporated herein by reference.

[0090] In some embodiments, the device is further configured to: for each user device, generate screen flicker rule information corresponding to the user device, and send the screen flicker rule information to the user device. Figure 1 The embodiments shown are the same or similar, so they are not described again and are incorporated herein by reference.

[0091] In some embodiments, the module 11 is used to: obtain a live video corresponding to the live environment, and determine the location information corresponding to each user device by identifying the screen flashing rule information corresponding to each user device in the live video. Figure 1 The embodiments shown are the same or similar, so they are not described again and are incorporated herein by reference.

[0092] In some embodiments, the determining the target image includes: determining the device arrangement information corresponding to the plurality of user devices according to the location information of each user device; and determining the target image matching the device arrangement information according to the device arrangement information. Figure 1 The embodiments shown are the same or similar, so they are not described again and are incorporated herein by reference.

[0093] In some embodiments, the target image is segmented into a plurality of target image units according to the location information of each user device, including a four-module 14 (not shown). The four-module 14 is used to map each user device to a pixel area on the target image according to the location information of each user device, and segment the target image according to the multiple pixel areas obtained by mapping to obtain multiple target image units, wherein each user device is mapped to a different pixel area, each pixel area corresponds to a target image unit, and different pixel areas do not overlap with each other. Here, the specific implementation method of the four-module 14 is the same as Figure 1The embodiments of step S14 are the same or similar, so they are not described again and are included here by reference.

[0094] In some embodiments, the one-fourth module 14 includes a one-fiveth module 15 (not shown). The one-fiveth module 15 is used to map each user device to a pixel area on the target image according to the location information of each user device and the pixel area division rule, and segment the target image according to the multiple pixel areas obtained by mapping to obtain multiple target image units; wherein the method further includes: controlling the device to determine the pixel area division rule corresponding to the target image according to the device arrangement information corresponding to the multiple user devices. Here, the specific implementation method of the one-fiveth module 15 is the same as Figure 1 The embodiments of step S15 are the same or similar, so they are not described again and are included here by reference.

[0095] In some embodiments, the one-five module 15 is used to: map each user device to a pixel area on the target image according to the location information of each user device and the pixel area division rule, wherein, for each user device, the area size of the pixel area mapped by the user device matches the screen size information corresponding to the user device; segment the target image according to the multiple pixel areas obtained by mapping to obtain multiple target image units; wherein the method further includes: obtaining the screen size information of each user device. Here, the related operations are similar to Figure 1 The embodiments shown are the same or similar, so they are not described again and are incorporated herein by reference.

[0096] In some embodiments, the image presentation instruction corresponding to each target image unit includes the start presentation time information and the end presentation time information of the target image unit. Figure 1 The embodiments shown are the same or similar, so they are not described again and are incorporated herein by reference.

[0097] In some embodiments, the graphic rendering instruction corresponding to each target image unit also includes rendering configuration information of the target image unit; wherein the device is further used to: receive device posture information sent by the user device corresponding to the target image unit; and determine, according to the device posture information, the rendering configuration information of the user device corresponding to the target image unit when rendering the target image unit. Figure 1 The embodiments shown are the same or similar, so they are not described again and are incorporated herein by reference.

[0098] In some embodiments, the graphic presentation instruction corresponding to each target image unit also includes screen setting information of the user device corresponding to the target image unit. Figure 1 The embodiments shown are the same or similar, so they are not described again and are incorporated herein by reference.

[0099] Figure 5 A user device structure diagram for forming an image matrix in a field environment according to an embodiment of the present application is shown, and the device includes a 21 module 21 and a 22 module 22. The 21 module 21 is used to receive a target image unit obtained by segmenting a target image and an image presentation instruction corresponding to the target image unit sent by a control device; the 22 module 22 is used to present the target image unit according to the image presentation instruction, wherein the target image unit and other target image units obtained by segmenting the target image and presented simultaneously in other multiple user devices constitute an image matrix corresponding to the target image.

[0100] The second module 21 is used to receive the target image unit obtained by segmenting the target image and the image presentation instruction corresponding to the target image unit sent by the control device. In some embodiments, the control device finds the target image unit corresponding to the position information in the target image according to the position information of each user device, and thus segments the target image into multiple target image units according to the position information of multiple user devices, each user device corresponds to a different target image unit, and each target image unit corresponds to a different user device. In some embodiments, the image presentation instruction corresponding to the target image unit may include but is not limited to the start presentation time, the end presentation time, the screen setting information and the presentation configuration information, wherein the start presentation time is used to indicate when the user device starts to present the target image unit, the end presentation time is used to indicate when the user device starts to end presenting the target image unit, the screen setting information is used to indicate how the user device sets the screen, part of the screen setting information can be directly set automatically by the user device, for example, the screen brightness, and part of the screen setting information needs to be presented on the user device screen to prompt the user to manually set it, for example, horizontal screen or vertical screen, and the presentation configuration information is determined by the control device according to the device posture information of the user device (for example, the rotation angle of the mobile phone), and is used to indicate the configuration information of the user device when rendering and presenting the target image unit (for example, the rotation matrix of the target image unit). In some embodiments, the configuration information in the image presentation instruction can also be determined by the user device according to its own device posture information, without sending its own device posture information to the control device for the control device to determine the presentation configuration information and send it to the user device. In some embodiments, the control device can send the target image unit corresponding to the user device and the image presentation instruction together to the user device, and can also send the target image unit and the image presentation instruction separately to the user device.

[0101] Module 22 is used to present the target image unit according to the image presentation instruction, wherein the target image unit and other target image units obtained by segmenting the target image and presented simultaneously in other multiple user devices constitute an image matrix corresponding to the target image. In some embodiments, the user device renders and presents the target image unit on the user device screen according to the image presentation instruction. In some embodiments, by simultaneously presenting different target image units on the screen of each user device holding the user device participating in forming the image matrix in the on-site environment, the different target image units obtained by segmenting the target image and presented simultaneously on the screens of multiple user devices constitute an image matrix, which makes people feel that a huge complete image is presented in the on-site environment as a whole, so that a visual experience of presenting a complete image in the on-site environment can be visually created by forming an image matrix. In some embodiments, by uniformly controlling what target image units should be presented on the screen of each user device participating in forming the image matrix and how to present the target image units, the efficiency and flexibility of presenting the complete image by forming the image matrix in the on-site environment can be improved, and no additional display device is required in the on-site environment. The presentation of the image matrix can be completed by the user device (for example, a mobile phone) carried by the on-site user himself, which can greatly save the hardware resource cost and improve the participation and sense of substitution of the on-site user.

[0102] In some embodiments, the device is further used to: receive request information sent by the control device for requesting to participate in forming an image matrix; in response to a feedback trigger operation performed by the user on the request information, generate feedback information and send it to the control device, wherein the feedback information is used to indicate whether the user device participates in forming the image matrix. Figure 1 The embodiments shown are the same or similar, so they are not described again and are incorporated herein by reference.

[0103] In some embodiments, the device is further used to: obtain location identification information corresponding to the user equipment, and send the location identification information to the control device. Figure 1 The embodiments shown are the same or similar, so they are not described again and are incorporated herein by reference.

[0104] In some embodiments, the device is further configured to: receive device identification information corresponding to the user device sent by the control device, and present the device identification information. Figure 1 The embodiments shown are the same or similar, so they are not described again and are incorporated herein by reference.

[0105] In some embodiments, the device is further used to: receive screen flicker rule information corresponding to the user device sent by the control device, and control the screen of the user device to flicker according to the screen flicker rule information. Figure 1 The embodiments shown are the same or similar, so they are not described again and are incorporated herein by reference.

[0106] In some embodiments, the device is further used to: obtain screen size information corresponding to the user device, and send the screen size information to the control device. Figure 1 The embodiments shown are the same or similar, so they are not described again and are incorporated herein by reference.

[0107] In some embodiments, the image presentation instruction includes the start presentation time information and the end presentation time information of the target image unit; wherein the presenting the target image unit includes: starting to present the target image unit according to the start presentation time, and ending to present the target image unit according to the end presentation time. Figure 1 The embodiments shown are the same or similar, so they are not described again and are incorporated herein by reference.

[0108] In some embodiments, the graphic presentation instruction further includes presentation configuration information of the target image unit; wherein presenting the target image unit includes: presenting the target image unit according to the presentation configuration information; wherein the device is further used to: obtain device posture information corresponding to the user device, and send the device posture information to the control device. Here, the relevant operations are similar to Figure 1 The embodiments shown are the same or similar, so they are not described again and are incorporated herein by reference.

[0109] In some embodiments, the graphic presentation instruction further includes screen setting information corresponding to the target image unit; wherein the device is further configured to: set the screen of the user device according to the screen setting information. Figure 1 The embodiments shown are the same or similar, so they are not described again and are incorporated herein by reference.

[0110] Figure 6 An exemplary system is shown that can be used to implement the various embodiments described in this application.

[0111] like Figure 6In some embodiments shown, the system 300 can be used as any of the devices in the various described embodiments. In some embodiments, the system 300 may include one or more computer-readable media (e.g., system memory or NVM / storage device 320) with instructions and one or more processors (e.g., (one or more) processors 305) coupled to the one or more computer-readable media and configured to execute instructions to implement modules to perform the actions described in this application.

[0112] For one embodiment, system control module 310 may include any suitable interface controller to provide any suitable interface to at least one of processor(s) 305 and / or any suitable device or component in communication with system control module 310 .

[0113] The system control module 310 may include a memory controller module 330 to provide an interface to the system memory 315. The memory controller module 330 may be a hardware module, a software module, and / or a firmware module.

[0114] The system memory 315 may be used, for example, to load and store data and / or instructions for the system 300. For one embodiment, the system memory 315 may include any suitable volatile memory, such as a suitable DRAM. In some embodiments, the system memory 315 may include double data rate type four synchronous dynamic random access memory (DDR4 SDRAM).

[0115] For one embodiment, system control module 310 may include one or more input / output (I / O) controllers to provide interfaces to NVM / storage device 320 and communication interface(s) 325 .

[0116] For example, NVM / storage device 320 may be used to store data and / or instructions. NVM / storage device 320 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable non-volatile storage device(s) (e.g., one or more hard disk drives (HDDs), one or more compact disk (CD) drives, and / or one or more digital versatile disk (DVD) drives).

[0117] NVM / storage device 320 may include storage resources that are physically part of the device on which system 300 is installed, or it may be accessible to the device without being part of the device. For example, NVM / storage device 320 may be accessed over a network via communication interface(s) 325.

[0118] Communication interface(s) 325 may provide an interface for system 300 to communicate over one or more networks and / or with any other suitable devices. System 300 may wirelessly communicate with one or more components of a wireless network in accordance with any of one or more wireless network standards and / or protocols.

[0119] For one embodiment, at least one of the processor(s) 305 may be packaged together with the logic of one or more controllers of the system control module 310 (e.g., the memory controller module 330). For one embodiment, at least one of the processor(s) 305 may be packaged together with the logic of one or more controllers of the system control module 310 to form a system-in-package (SiP). For one embodiment, at least one of the processor(s) 305 may be integrated on the same die with the logic of one or more controllers of the system control module 310. For one embodiment, at least one of the processor(s) 305 may be integrated on the same die with the logic of one or more controllers of the system control module 310 to form a system on chip (SoC).

[0120] In various embodiments, the system 300 may be, but is not limited to: a server, a workstation, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet computer, a netbook, etc.). In various embodiments, the system 300 may have more or fewer components and / or a different architecture. For example, in some embodiments, the system 300 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touch screen display), a non-volatile memory port, multiple antennas, a graphics chip, an application specific integrated circuit (ASIC), and a speaker.

[0121] The present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer code. When the computer code is executed, the method described in any of the preceding items is executed.

[0122] The present application also provides a computer program product. When the computer program product is executed by a computer device, the method described in any of the preceding items is executed.

[0123] The present application also provides a computer device, the computer device comprising:

[0124] one or more processors;

[0125] a memory for storing one or more computer programs;

[0126] When the one or more computer programs are executed by the one or more processors, the one or more processors are caused to implement the method as described in any of the preceding items.

[0127] It should be noted that the present application can be implemented in software and / or a combination of software and hardware, for example, can be implemented using an application specific integrated circuit (ASIC), a general purpose computer or any other similar hardware device. In one embodiment, the software program of the present application can be executed by a processor to implement the steps or functions described above. Similarly, the software program of the present application (including relevant data structures) can be stored in a computer-readable recording medium, for example, a RAM memory, a magnetic or optical drive or a floppy disk and similar devices. In addition, some steps or functions of the present application can be implemented using hardware, for example, as a circuit that cooperates with a processor to perform each step or function.

[0128] In addition, a part of the present application may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present application through the operation of the computer. Those skilled in the art should understand that the existence of computer program instructions in computer-readable media includes but is not limited to source files, executable files, installation package files, etc., and accordingly, the way in which computer program instructions are executed by a computer includes but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to the computer.

[0129] Communication media include media by which communication signals containing, for example, computer readable instructions, data structures, program modules, or other data are transmitted from one system to another. Communication media may include guided transmission media such as cables and wires (e.g., fiber optic, coaxial, etc.) and wireless (unguided transmission) media that can propagate energy waves, such as acoustic, electromagnetic, RF, microwave, and infrared. Computer readable instructions, data structures, program modules, or other data may be embodied as a modulated data signal in, for example, a wireless medium such as a carrier wave or similar mechanism such as embodied as part of spread spectrum technology. The term "modulated data signal" refers to a signal whose one or more characteristics are changed or set in such a manner as to encode information in the signal. Modulation may be analog, digital, or a hybrid modulation technique.

[0130] By way of example and not limitation, computer-readable storage media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. For example, computer-readable storage media include, but are not limited to, volatile memory, such as random access memory (RAM, DRAM, SRAM); and non-volatile memory, such as flash memory, various read-only memories (ROM, PROM, EPROM, EEPROM), magnetic and ferromagnetic / ferroelectric memories (MRAM, FeRAM); and magnetic and optical storage devices (hard disks, magnetic tapes, CDs, DVDs); or other media now known or later developed that can store computer-readable information / data for use by a computer system.

[0131] Here, according to an embodiment of the present application, a device is included, which includes a memory for storing computer program instructions and a processor for executing the program instructions, wherein, when the computer program instructions are executed by the processor, the device is triggered to run the methods and / or technical solutions based on the aforementioned multiple embodiments of the present application.

[0132] It is obvious to those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and that the present application can be implemented in other specific forms without departing from the spirit or basic features of the present application. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the present application is limited by the attached claims rather than the above description, so it is intended to include all changes that fall within the meaning and scope of the equivalent elements of the claims in the present application. Any figure mark in the claims should not be regarded as limiting the claims involved. In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the device claim can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

Claims

1. A method for forming an image matrix in a live environment, applied to a control device, wherein: The method comprises: Acquire location information of each user device among a plurality of user devices participating in forming an image matrix in a live environment; Determine, according to the location information of each user device, device arrangement information corresponding to the multiple user devices, the device arrangement information is used to indicate the overall layout information of the multiple user devices; determine, according to the device arrangement information, a target image matching the device arrangement information; map, according to the location information of each user device, each user device to a pixel area on the target image, and segment the target image according to the mapped pixel areas to obtain a plurality of target image units, wherein each target image unit corresponds to a different user device among the multiple user devices, each user device is mapped to a different pixel area, each pixel area corresponds to a target image unit, different pixel areas do not overlap with each other, and a relative position of a user device among the multiple user devices is consistent with a relative position of the pixel area to which the user device is mapped in the target image; For each target image unit, an image presentation instruction corresponding to the target image unit is generated, and the target image unit and the image presentation instruction are sent to a user device corresponding to the target image unit, so that the multiple target image units simultaneously presented in the multiple user devices constitute an image matrix corresponding to the target image.

2. The method according to claim 1, wherein: The method further comprises, before acquiring the position information of each user device among a plurality of user devices participating in forming the image matrix in the live environment: Sending request information for requesting to participate in forming an image matrix to each of the plurality of pending user devices in the live environment; A plurality of user devices participating in forming an image matrix are determined according to feedback information of at least one of the plurality of pending user devices in response to the request information, wherein the feedback information is used to indicate whether the at least one pending user device participates in forming the image matrix.

3. The method according to claim 1, wherein: The obtaining of the location information of each user device among the plurality of user devices participating in forming the image matrix in the on-site environment includes: For each user device, receive the location identification information sent by the user device, and determine the location information corresponding to the user device based on the location identification information and preset location mapping relationship information, wherein the preset location mapping relationship information is used to indicate the mapping relationship between the predetermined location identifier and the location information.

4. The method according to claim 1, wherein: The method further comprises: For each user equipment, device identification information corresponding to the user equipment is generated, and the device identification information is sent to the user equipment.

5. The method according to claim 4, wherein: The obtaining of the location information of each user device among the plurality of user devices participating in forming the image matrix in the on-site environment includes: An on-site photo corresponding to the on-site environment is obtained, and the location information of each user device is determined by identifying the device identification information corresponding to each user device in the on-site photo.

6. The method according to claim 1, wherein: The method further comprises: For each user equipment, screen flicker rule information corresponding to the user equipment is generated, and the screen flicker rule information is sent to the user equipment.

7. The method according to claim 6, wherein: The obtaining of the location information of each user device among the plurality of user devices participating in forming the image matrix in the on-site environment includes: A live video corresponding to the live environment is obtained, and the location information corresponding to each user device is determined by identifying the screen flashing rule information corresponding to each user device in the live video.

8. The method according to claim 1, wherein: The method of mapping each user device to a pixel region on the target image according to the location information of each user device, and segmenting the target image according to the multiple pixel regions obtained by mapping to obtain multiple target image units includes: According to the location information of each user device and the pixel area division rule, each user device is mapped to a pixel area on the target image, and the target image is segmented according to the multiple pixel areas obtained by mapping to obtain multiple target image units; Wherein, the method further comprises: A pixel area division rule corresponding to the target image is determined according to device arrangement information corresponding to the multiple user devices.

9. The method according to claim 8, wherein: The method of mapping each user device to a pixel region on the target image according to the location information of each user device and the pixel region division rule, and segmenting the target image according to the multiple pixel regions obtained by mapping to obtain multiple target image units includes: According to the location information of each user device and the pixel area division rule, each user device is mapped to a pixel area on the target image, wherein, for each user device, the area size of the pixel area mapped by the user device matches the screen size information corresponding to the user device; Segmenting the target image according to the multiple pixel regions obtained by mapping to obtain multiple target image units; Wherein, the method further comprises: The screen size information of each user device is obtained.

10. The method according to claim 1, wherein: The image presentation instruction corresponding to each target image unit includes the start presentation time information and the end presentation time information of the target image unit.

11. The method according to claim 10, wherein: The image presentation instruction corresponding to each target image unit also includes presentation configuration information of the target image unit; Wherein, the method further comprises: Receiving device posture information sent by a user device corresponding to the target image unit; According to the device posture information, presentation configuration information of the user device corresponding to the target image unit when presenting the target image unit is determined.

12. The method according to claim 10, wherein: The image presentation instruction corresponding to each target image unit also includes screen setting information of the user equipment corresponding to the target image unit.

13. A method for forming an image matrix in a live environment, applied to a user device, wherein: The method comprises: Receive a target image unit obtained by segmenting a target image and an image presentation instruction corresponding to the target image unit, sent by a control device, wherein the control device obtains the position information of each user device among multiple user devices participating in forming an image matrix in a live environment; determines device arrangement information corresponding to the multiple user devices according to the position information of each user device, wherein the device arrangement information is used to indicate overall layout information of the multiple user devices; determines a target image matching the device arrangement information according to the device arrangement information; maps each user device to a pixel area on the target image according to the position information of each user device, and displays the target image according to the mapping information. The target image is segmented into multiple pixel areas to obtain multiple target image units, wherein each target image unit corresponds to a different user device among the multiple user devices, each user device is mapped to a different pixel area, each pixel area corresponds to a target image unit, different pixel areas do not overlap with each other, and the relative position of a user device among the multiple user devices is consistent with the relative position of the pixel area mapped to the user device in the target image; for each target image unit, an image presentation instruction corresponding to the target image unit is generated, and the target image unit and the image presentation instruction are sent to the user device corresponding to the target image unit; The target image unit is presented according to the image presentation instruction, wherein the target image unit and other target image units obtained by segmenting the target image and presented simultaneously in other multiple user devices constitute an image matrix corresponding to the target image.

14. The method according to claim 13, further comprising: receiving request information sent by the control device for requesting to participate in forming an image matrix; In response to a feedback triggering operation performed by the user on the request information, feedback information is generated and sent to the control device, wherein the feedback information is used to indicate whether the user device participates in forming an image matrix.

15. The method according to claim 13, wherein: The method further comprises: Acquire location identification information corresponding to the user equipment, and send the location identification information to the control device.

16. The method according to claim 13, wherein: The method further comprises: Receive device identification information corresponding to the user equipment and sent by the control device, and present the device identification information.

17. The method according to claim 13, wherein: The method further comprises: Receive screen flickering rule information corresponding to the user equipment and sent by the control device, and control the screen of the user equipment to flicker according to the screen flickering rule information.

18. The method according to claim 13, wherein: The method further comprises: Acquire screen size information corresponding to the user equipment, and send the screen size information to the control device.

19. The method according to claim 13, wherein: The image presentation instruction includes the start presentation time information and the end presentation time information of the target image unit; Wherein, presenting the target image unit includes: The target image unit starts to be presented according to the presentation start time, and the target image unit ends to be presented according to the presentation end time.

20. The method according to claim 19, wherein: The image presentation instruction also includes presentation configuration information of the target image unit; Wherein, presenting the target image unit includes: presenting the target image unit according to the presentation configuration information; Wherein, the method further comprises: Acquire device posture information corresponding to the user device, and send the device posture information to the control device.

21. The method according to claim 19, wherein: The image presentation instruction also includes screen setting information corresponding to the target image unit; Wherein, the method further comprises: The screen of the user equipment is set according to the screen setting information.

22. A method for forming an image matrix in a live environment, wherein: The method comprises: The control device obtains the position information of each user device among the multiple user devices participating in forming the image matrix in the scene environment; The control device determines, according to the location information of each user device, device arrangement information corresponding to the multiple user devices, wherein the device arrangement information is used to indicate the overall layout information of the multiple user devices; determines, according to the device arrangement information, a target image matching the device arrangement information; maps each user device to a pixel area on the target image according to the location information of each user device, and segments the target image according to the mapped pixel areas to obtain a plurality of target image units, wherein each target image unit corresponds to a different user device among the multiple user devices, each user device is mapped to a different pixel area, each pixel area corresponds to a target image unit, different pixel areas do not overlap with each other, and a relative position of a user device among the multiple user devices is consistent with a relative position of the pixel area to which the user device is mapped in the target image; The control device generates, for each target image unit, an image presentation instruction corresponding to the target image unit, and sends the target image unit and the image presentation instruction to a user device corresponding to the target image unit; The first user equipment receives, sent by the control device, a first target image unit obtained by segmenting the target image and a first image presentation instruction corresponding to the first target image unit; The first user device presents the first target image unit according to the first image presentation instruction, wherein the first target image unit and other target image units obtained by segmenting the target image and presented simultaneously in other multiple user devices constitute an image matrix corresponding to the target image.

23. A device for forming an image matrix in a live environment, characterized in that: The device comprises: Processor; and A memory arranged to store computer executable instructions which, when executed, cause the processor to perform a method as claimed in any one of claims 1 to 12.

24. A device for forming an image matrix in a live environment, characterized in that: The device comprises: Processor; and A memory arranged to store computer executable instructions which, when executed, cause the processor to perform a method as claimed in any one of claims 13 to 21.

25. A computer readable medium storing instructions which, when executed, cause a system to perform the operations of the method of any one of claims 1 to 21.

Images