Image transmission and reception system, transmitter, receiver, computer program, and image transmission and reception method

Abstract

To improve the efficiency of transmitting image content. [Solution] The image transmission and reception system comprises a data acquisition unit that acquires text data representing the subject in each region of a raw image in which the raw image in which the subject is captured has been divided into multiple regions; a transmission unit that transmits transmission data representing the text data of each region; a reception unit that receives the transmission data; and an image acquisition unit that acquires an image generated based on the text data of each region represented by the transmission data.

Description

Technical Field

[0001] The present invention relates to an image transmission / reception system, a transmitter, a receiver, a computer program, and an image transmission / reception method.

Background Art

[0002] In Patent Document 1, for example, in a system that performs video streaming transmission / reception to view moving image content via a communication network such as the Internet, reduction of the transmission rate (bit rate) of moving image content (data compression of moving image content) and maintenance of the image quality of moving image content (maintenance of details of moving image content) are achieved. The image transmission / reception technology described in Patent Document 1 generates model data for generating an improved image closer to the original image from a low-bitrate encoded image obtained by encoding the original image at a low bit rate by machine learning. The transmitter transmits the low-bitrate encoded image and the model data, and the receiver generates an improved image of the low-bitrate encoded image from the received low-bitrate encoded image and the model data.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the image transmission / reception technology described in Patent Document 1 mentioned above, the reduction of the transmission bit rate of image content may be insufficient. For example, in a satellite communication network that covers a wide area, the bandwidth per person is narrow, and further reduction of the transmission bit rate of image content is desired.

[0005] Furthermore, in recent years, there has been an increase in viewers seeking to improve their time performance by shortening viewing time through summaries of video content. For such viewers, the details of video content are not necessarily important, and it is considered sufficient if they can understand the general content of the video content.

[0006] Therefore, an image transmission and reception technology that can improve the transmission efficiency of image content by further reducing the transmission bitrate of image content while maintaining the outline of the image content is preferable.

[0007] This invention has been made in consideration of these circumstances, and its purpose is to provide an image transmission and reception technology that can improve the transmission efficiency of image content. [Means for solving the problem]

[0008] One aspect of the present invention includes a data acquisition unit that acquires text data representing the subject in each region of a raw image in which a subject is captured, the raw image being divided into multiple regions, a transmission unit that transmits transmission data representing the text data in each region, a receiving unit that receives the transmission data, and an image acquisition unit that acquires an image generated based on the text data in each region represented by the transmission data. The system includes a division setting unit for setting the division method of the original image, wherein the division setting unit changes the number of pixels per region into which the original image is divided according to the load state of the communication network used for transmitting the transmission data. It is an image transmission and reception system. One aspect of the present invention is an image transmission and reception system in which the text data is text data representing a part of the subject. One aspect of the present invention is an image transmission and reception system in which the data acquisition unit further acquires metadata representing the state of the subject captured in each region within the original image, the transmission data further includes the metadata, and the image acquired by the image acquisition unit is an image generated based on the metadata as well. One aspect of the present invention is an image transmission and reception system having an identifier assigned to each piece of text data and corresponding correspondence data, wherein the transmitted data includes the identifier assigned to each piece of text data in each of the regions. One aspect of the present invention is an image transmission and reception system in which the transmitted data further includes the corresponding data. One aspect of the present invention is an image transmission and reception system in which the identifier included in the transmitted data is compressed using a predetermined data compression method.

[0009] One aspect of the present invention includes a data acquisition unit that acquires text data representing the subject in each region of a raw image in which a subject is captured, and a transmission unit that transmits transmission data representing the text data for each of the regions. The system includes a division setting unit for setting the division method of the original image, wherein the division setting unit changes the number of pixels per region into which the original image is divided according to the load state of the communication network used for transmitting the transmission data. It is a transmitter. One aspect of the present invention is a transmitter further comprising an image recognition unit that generates the text data from the original image.

[0010] One aspect of the present invention is a receiver comprising a receiving unit that receives the transmission data transmitted from the above-mentioned transmitter, and an image acquisition unit that acquires an image generated based on the text data of each of the regions represented by the transmission data. One aspect of the present invention is a receiver further comprising an image generation unit that generates an image from the text data of each region represented by the transmitted data.

[0011] One aspect of the present invention provides a data acquisition step to a computer of a transmitter, in which the original image in which the subject is captured is divided into multiple regions, and for each region within the original image, text data representing the subject captured in that region is acquired; and a transmission step to transmit transmission data representing the text data for each of the regions. The system executes a division setting step, which sets a method for dividing the original image, and the division setting step changes the number of pixels per region into which the original image is divided according to the load state of the communication network used for transmitting the transmission data. It is a computer program.

[0012] One aspect of the present invention is a computer program for causing a computer of a receiver to execute a receiving step of receiving the transmission data transmitted from the above transmitter, and an image acquisition step of acquiring an image generated based on the text data of each of the regions represented by the transmission data.

[0013] One aspect of the present invention is an image transmission / reception method executed by an image transmission / reception system, including a data acquisition step of acquiring text data representing a subject captured in a region in a source image in which a source image of a subject is divided into a plurality of parts, a transmission step of transmitting transmission data representing the text data of each of the regions, a reception step of receiving the transmission data, and an image acquisition step of acquiring an image generated based on the text data of each of the regions represented by the transmission data. The division setting step includes setting a method for dividing the original image, wherein the division setting step changes the number of pixels per region into which the original image is divided according to the load state of the communication network used for transmitting the transmission data. It is an image transmission / reception method.

Advantages of the Invention

[0014] According to the present invention, an effect can be obtained that the transmission efficiency of image content can be improved.

Brief Description of the Drawings

[0015] [Figure 1] It is a diagram showing a configuration example of an image transmission / reception system according to an embodiment. [Figure 2] It is a diagram showing an example of a transmission procedure of an image transmission / reception method according to an embodiment. [Figure 3] It is a diagram showing an example of a reception procedure of an image transmission / reception method according to an embodiment. [Figure 4] It is a diagram for explaining an image recognition process according to an embodiment. [Figure 5] It is a diagram showing a configuration example of association data according to an embodiment. [Figure 6] It is a diagram showing a configuration example of identifier data according to an embodiment. [Figure 7] It is a diagram showing an example of compressed identifier data according to an embodiment. [Figure 8]This is a diagram for explaining image generation processing according to an embodiment. [Figure 9] This is a diagram showing a modification example of a transmitter and a receiver according to an embodiment.

Embodiments for Carrying Out the Invention

[0016] Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a diagram showing a configuration example of an image transmission / reception system according to an embodiment. In FIG. 1, the image transmission / reception system includes a transmitter 10 and a receiver 30. The transmitter 10 transmits transmission data SD of image content via a communication network NW. The receiver 30 receives the transmission data SD of image content via the communication network NW. The image content may be content of a moving image or content of a still image. The communication network NW may be a wireless network, a wired network, or a combination of a wireless network and a wired network. For example, the communication network NW may include a satellite communication network.

[0017] The transmitter 10 includes a data acquisition unit 11, a transmission unit 12, and a division setting unit 13. The transmitter 10 accesses an image recognition device 50. In the configuration example of FIG. 1, the image recognition device 50 is provided separately from the transmitter 10, but the transmitter 10 may have the function (image recognition function) of the image recognition device 50. The image recognition device 50 may be configured using an artificial intelligence (image recognition AI) used for image recognition. The image recognition AI may utilize, for example, semantic segmentation technology.

[0018] Each function of the transmitter 10 is realized by the transmitter 10 being equipped with computer hardware such as a processor (CPU, Central Processing Unit, GPU, Graphics Processing Unit, etc.) and memory, with the CPU executing a computer program stored in memory. The transmitter 10 may be configured using a general-purpose computer device, or it may be configured as a dedicated hardware device. For example, the transmitter 10 may be configured using a server computer connected to a communication network such as the Internet. Furthermore, each function of the transmitter 10 may be realized by cloud computing. The transmitter 10 may also be configured using a mobile terminal device such as a smartphone or a tablet computer (tablet PC). Furthermore, the transmitter 10 may be configured using a camera connected to a communication network. The transmitter 10 may be implemented by a single computer, or its functions may be distributed among multiple computers. Furthermore, the transmitter 10 may be configured to launch a website using, for example, a WWW system.

[0019] The data acquisition unit 11 acquires text data 102 representing the subject in each region of the original image 101, which has been divided into multiple parts. The data acquisition unit 11 acquires text data 102 from the original image 101 using the image recognition device 50. The transmission unit 12 transmits transmission data SD representing the text data 102 for each region. The division setting unit 13 sets the division method for the original image 101.

[0020] The receiver 30 comprises a receiving unit 31 and an image acquisition unit 32. The receiver 30 accesses the image generation device 60. In the example configuration shown in Figure 1, the image generation device 60 is provided separately from the receiver 30, but the receiver 30 may also have the functions of the image generation device 60 (image generation function). The image generation device 60 may be configured using artificial intelligence (image generation AI) used for image generation. For example, a diffusion model may be used as the image generation AI.

[0021] Each function of the receiver 30 is realized by the receiver 30 being equipped with computer hardware such as a processor (CPU, GPU, etc.) and memory, with the CPU executing a computer program stored in memory. The receiver 30 may be configured using a general-purpose computer device, or it may be configured as a dedicated hardware device. For example, the receiver 30 may be configured using a server computer connected to a communication network such as the Internet. Furthermore, each function of the receiver 30 may be realized by cloud computing. The receiver 30 may also be configured using a mobile terminal device such as a smartphone or a tablet computer (tablet PC). Furthermore, the receiver 30 may be implemented by a single computer, or its functions may be distributed among multiple computers. Furthermore, the receiver 30 may be configured to launch a website using, for example, a WWW system.

[0022] The receiving unit 31 receives the transmission data SD. The image acquisition unit 32 acquires an image 302 generated based on the text data 102 of each region represented by the transmission data SD. The image acquisition unit 32 acquires the image 302 using the image generation device 60. The image acquisition unit 32 creates a prompt 301 to supply to the image generation device 60 from the text data 102 of each region represented by the transmission data SD.

[0023] Next, the image transmission and reception method according to this embodiment will be described.

[0024] [Sending Procedure] The transmission procedure in the transmitter 10 will be explained with reference to Figure 2. Figure 2 is a diagram showing an example of the transmission procedure of the image transmission and reception method according to this embodiment.

[0025] (Step S11) The division setting unit 13 sets the method for dividing the original image 101. The division method may be fixed or variable. For example, the number of pixels per region in which the original image 101 is divided (region pixel count) may be predetermined, and the division setting unit 13 sets how many regions to divide the original image 101 into according to the size (number of pixels) of the original image 101. A default value for the region pixel count may be set in advance. Furthermore, the system may be configured to allow the user to arbitrarily change the region pixel count.

[0026] Furthermore, the division setting unit 13 may acquire information indicating the load status of the communication network NW and change the number of region pixels according to the load status of the communication network NW. The division setting unit 13 increases the number of region pixels when the load of the communication network NW is high. When the number of region pixels increases, the number of divisions (regions) of the original image 101 decreases, so the amount of information of the original image 101 included in the transmitted data SD decreases, and the amount of data (transmission bitrate) of the transmitted data SD decreases. On the other hand, the division setting unit 13 decreases the number of region pixels when the load of the communication network NW is low. When the number of region pixels decreases, the number of divisions (regions) of the original image 101 increases, so the amount of information of the original image 101 included in the transmitted data SD increases, and the amount of data (transmission bitrate) of the transmitted data SD increases. For example, by improving the fidelity of the subject position within the image, effects such as the image 302 generated on the receiver side becoming closer to the original image 101 can be obtained.

[0027] (Step S12) The data acquisition unit 11 uses the image recognition device 50 to acquire text data 102 from the original image 101. Specifically, the data acquisition unit 11 transmits the original image 101 to the image recognition device 50 and requests the image recognition device 50 to perform image recognition according to the division method set in step S11. The image recognition device 50 performs image recognition of the original image 101 according to the division method in response to the request and sends the text data 102 of the image recognition result back to the transmitter 10. The data acquisition unit 11 acquires the text data 102 returned from the image recognition device 50.

[0028] Figure 4 is a diagram illustrating the image recognition process according to this embodiment. In the example in Figure 4, the original image 101 (hereinafter referred to as the target original image 101) containing the subject "cat" is divided into 20 regions in a 4x5 array. The data acquisition unit 11 transmits the target original image 101 to the image recognition device 50 and requests the image recognition device 50 to perform image recognition using the division method "divide into 20 regions in a 4x5 array" set in step S11. In response to this request, the image recognition device 50 performs image recognition of the target original image 101 using the division method "divide into 20 regions in a 4x5 array". Specifically, the image recognition device 50 recognizes the subject in each of the 20 regions of the target original image 101 and generates text data 102 of the recognition result.

[0029] For example, the image recognition device 50 recognizes that a "cat's tail" is visible in a 1x1 area and generates text data 102 "cat's tail" for that area. For example, the image recognition device 50 recognizes that a "cat's right paw" is visible in a 2x1 area and generates text data 102 "cat's right paw" for that area. For example, the image recognition device 50 recognizes that a "cat's nose" and a "cat's paw" are visible in a 4x2 area and generates text data 102 "cat's nose, cat's paw" for that area. Text data 102 is text data representing the part of the subject "cat". For each area, there may be one or more recognized elements of the subject.

[0030] The image recognition device 50 responds to the transmitter 10 with text data 102 for each region of the 4x5 array of the target original image 101. The text data 102 for each region of the target original image 101 contains information indicating which region of the 4x5 array of the target original image 101 it belongs to. For example, as shown in Figure 4, the text data 102 for each region of the target original image 101 may be array data stored in each element of the 4x5 array. The data acquisition unit 11 acquires the text data 102 for each region of the 4x5 array of the target original image 101 that has been responded to by the image recognition device 50.

[0031] (Step S13) The transmission unit 12 creates correspondence data in which an identifier assigned to each text data 102 of the image recognition result of the original image 101 is linked to the text data 102. Figure 5 shows an example of the configuration of correspondence data 110 for the target original image 101 shown in Figure 4. As illustrated in Figure 5, correspondence data 110 is data in which an identifier (ID) "1,2,···,12" assigned to each text data 102 related to the target original image 101 is linked to the text data 102.

[0032] The ID of each text data 102 may be determined, for example, according to the frequency of occurrence of each text data 102. For example, IDs may be assigned in ascending order from the text data 102 with the highest frequency of occurrence. In the example configuration of the correspondence data 110 in Figure 5, for example, the text data 102 "cat's left cheek" is assigned the ID "1". For example, the text data 102 "cat's tail" is assigned the ID "4".

[0033] (Step S14) The transmission unit 12 uses the mapping data to create identifier data that stores the IDs of the text data 102 for each region of the original image 101. Figure 6 shows an example of the configuration of the identifier data 103 for the target original image 101 shown in Figure 4. The identifier data 103 stores the IDs of the text data 102 for each region of the 4x5 array of the target original image 101.

[0034] For example, as shown in Figure 6, the identifier data 103 may be array data in which the IDs of the text data 102 of each region in a 4x5 array of the target original image 101 are stored in each element of the 4x5 array. In the array data of identifier data 103 in Figure 6, for example, the element in 1x1 stores the ID "4" obtained from the text data 102 "cat's tail" in the 1x1 region of the target original image 101 in Figure 4 using the mapping data 110 in Figure 5. For example, the element in 2x1 stores the ID "7" obtained from the text data 102 "cat's right paw" in the 2x1 region of the target original image 101 in Figure 4 using the mapping data 110 in Figure 5. For example, the element in 4x2 stores the IDs "11" and "2" obtained from the text data 102 "cat's nose, cat's paw" in the 4x2 region of the target original image 101 in Figure 4 using the mapping data 110 in Figure 5.

[0035] (Step S15) The transmitting unit 12 performs data compression on the identifier data of the original image 101. The data compression method is not limited. For example, the Huffman coding method may be used.

[0036] Figure 7 shows an example of compressed data (compressed identifier data) related to the identifier data 103 in Figure 6. The compressed identifier data 104 in Figure 7 is the compressed data resulting from the data compression of the identifier data 103 in Figure 6 related to the target original image 101 in Figure 4 using the Huffman coding method.

[0037] (Step S16) The transmitting unit 12 transmits transmission data SD, which includes compressed identifier data and mapping data related to the original image, via the communication network NW. In the example of the target original image 101 in Figure 4, the transmitter 10 transmits transmission data SD, which includes compressed identifier data 104 in Figure 7 and mapping data in Figure 5 related to the target original image 101, via the communication network NW.

[0038] The transmitted data SD from the transmitter 10 is received by the receiver 30 via the communication network NW. The receiver 30 then obtains the compressed identifier data and mapping data related to the original image contained in the transmitted data SD. In the example of the target original image 101 in Figure 4, the receiver 30 obtains the compressed identifier data 104 in Figure 7 and the mapping data in Figure 5 related to the target original image 101, which are contained in the transmitted data SD.

[0039] [Receiving Procedure] The receiving procedure in the receiver 30 will be explained with reference to Figure 3. Figure 3 is a diagram showing an example of the receiving procedure of the image transmission and reception method according to this embodiment.

[0040] (Step S21) The receiving unit 31 receives the transmission data SD transmitted from the transmitter 10. The transmission data SD is data that includes compressed identifier data and mapping data related to the original image. In the example of the target original image 101 in Figure 4, the transmission data SD is data that includes the compressed identifier data 104 in Figure 7 and the mapping data in Figure 5 related to the target original image 101.

[0041] (Step S22) The image acquisition unit 32 decodes the compressed identifier data relating to the original image and restores the identifier data. In the example of the target original image 101 in Figure 4, the image acquisition unit 32 decodes the compressed identifier data 104 relating to the target original image 101 using the Huffman coding compression method and restores the identifier data 103 in Figure 6.

[0042] (Step S23) The image acquisition unit 32 uses the correspondence data for the original image to reconstruct the text data 102 for each region from the reconstructed identifier data. In the example of the target original image 101 in Figure 4, the image acquisition unit 32 uses the correspondence data 110 in Figure 5 for the target original image 101 to reconstruct the text data 102 for each region of the 4x5 array of the target original image 101 in Figure 4 from the reconstructed identifier data 103 (Figure 6).

[0043] (Step S24) The image acquisition unit 32 uses the image generation device 60 to generate images from the text data 102 of each restored region. Figure 8 is a diagram illustrating the image generation process according to this embodiment. The example in Figure 8 is for image generation related to the target original image 101 in Figure 4, and the image generation device 60 is configured using an image generation AI.

[0044] As shown in Figure 8, the image acquisition unit 32 creates a prompt 301 to supply to the image generation device 60 (image generation AI) from the text data 102 in each region of the 4x5 array of the restored target original image 101 in Figure 4. The image acquisition unit 32 sends the created prompt 301 to the image generation device 60 and requests image generation. In response to this request, the image generation device 60 uses the prompt 301 and generates an image 302 using the image generation AI. The image generation device 60 responds to the receiver 30 with the generated image 302. The image acquisition unit 32 acquires the image 302 responded to from the image generation device 60. The receiver 30 displays the image 302 acquired by the image acquisition unit 32 on its display screen. The viewer sees the image 302 displayed on the display screen of the receiver 30.

[0045] Image 302 is an image generated based on the text data 102 of each region in the 4x5 array of the original image 101 in Figure 4. Therefore, the subject in image 302 is a "cat," and it is expected that each region in the 4x5 array of image 302 will contain a portion of the cat generated based on the text data 102 of that region. However, the reproducibility of the original image 101 in image 302 may be low. However, although the details of the subject "cat" in the original image 101 may be lost, the approximate state of the subject "cat" in the original image 101 can be represented in image 302. Therefore, for viewers who do not necessarily need to know the details of the image content and are satisfied as long as they can understand the approximate content of the image, the reproducibility of the original image 101 in image 302 is considered sufficient.

[0046] On the other hand, regarding the transmission bitrate, actual experiments have shown that while the data volume of the target original image 101 in Figure 4 is approximately 504 kilobits, the data volume can be reduced to less than 100 bits under certain conditions, making a significant reduction in the transmission bitrate possible.

[0047] Thus, according to this embodiment, it is possible to significantly reduce the transmission bitrate of image content while maintaining the outline of the image content, thereby improving the transmission efficiency of image content.

[0048] Next, some modified examples of the embodiments described above will be explained.

[0049] (Variation 1) In the embodiment described above, the mapping data is created by the transmitter 10 for each original image and transmitted from the transmitter 10 to the receiver 30. However, the method by which the transmitter 10 and the receiver 30 share the mapping data is not limited to this. For example, one mapping data set may be created for multiple different original images. For example, the mapping data may be created in advance and shared between the transmitter 10 and the receiver 30.

[0050] Furthermore, the mapping data may be compressed as a bit string. The transmission unit 12 includes the compressed mapping data, which is the result of the mapping data being compressed as a bit string, in the transmission data SD and transmits it. The image acquisition unit 32 decodes the compressed mapping data and restores the mapping data.

[0051] (Modification 2) The data acquisition unit 11 further acquires metadata representing the state of the subject in each region of the original image. The transmission unit 12 includes the acquired metadata for each region in the transmission data SD. The image acquisition unit 32 further supplies the metadata for each region included in the transmission data SD to the image generation device 60. The image generation device 60 generates an image based on the supplied metadata for each region.

[0052] The metadata for each region may include information about the subject, such as information about the direction and speed of movement, the direction and speed of rotation, the direction and speed of scaling, the direction and speed of hue changes, the direction and speed of brightness changes, the direction and speed of saturation changes, and the direction and speed of changes in the components of each of the three primary colors.

[0053] Furthermore, the metadata for each region may include information that describes the actions of the subject or situation in the original image, such as walking, running, laughing, waving, or the sun setting, depending on the type of subject.

[0054] (Variation 3) When the transmission unit 12 transmits multiple text data 102 acquired from multiple original images in succession, it transmits only the difference information between the consecutively transmitted text data 102. This makes it possible to further reduce the transmission bitrate.

[0055] (Modification 4) The transmitter 10 transmits video content composed of multiple source images. Each source image corresponds to one frame that makes up the video content (video frame type distribution). The transmitter 10 includes frame time information in the transmitted data SD. The frame time information may be added to the text data 102, or it may be separate information from the text data 102.

[0056] The receiver 30 uses the image generation device 60 to generate multiple frames from the text data 102 and frame time information of multiple frames, and generates video content by playing the generated multiple frames in sequence.

[0057] Furthermore, video frame-type distribution may also be video frame representative distribution, which transmits only one frame (representative frame) that represents multiple frames. In the case of video frame representative distribution, the transmitter 10 includes frame start time information and frame end time information in the transmitted data SD. The frame start time information and frame end time information may be added to the text data 102, or they may be separate information from the text data 102. The transmitter 10 may also include information representing the temporal behavior of the subject (behavior information) in the transmitted data SD. The behavior information may be added to the text data 102, or they may be separate information from the text data 102. The receiver 30 may use the image generation device 60 to generate multiple frames from the text data 102 and the behavior information to cover the period from the frame start time of the frame start time information to the frame end time of the frame end time information (between consecutive representative frames). Furthermore, the image generation device 60 may be configured using video generation AI (e.g., a video diffusion model). Furthermore, the image generation device 60 may be configured using a time-series interpolation generation AI (for example, a temporal diffusion model) which is an AI that generates a video from a single image.

[0058] (Variation 5) The transmitter 10 transmits 3D image content composed of multiple original images. Each original image corresponds to one slice that makes up the 3D image content (3D slice type distribution). The transmitter 10 includes slice coordinate information in the transmitted data SD in a direction perpendicular to the slice plane. The slice coordinate information may be added to the text data 102, or it may be separate information from the text data 102.

[0059] The receiver 30 uses the image generation device 60 to generate multiple slices from the text data 102 and slice coordinate information of multiple slices, and generates a 3D image content composed of the generated multiple slices. The receiver 30 may display the 3D image content on a 2D display using perspective. The receiver 30 may also use parallax 3D display as a method for displaying the 3D image content, which uses parallax to create a 3D representation by inputting separate images to the viewer's two eyes. The image generation device 60 may be configured using a 3D (spatial) information generation AI (for example, a 3D diffusion model).

[0060] The transmitter 10 may also transmit 3D video content composed of multiple 3D image contents. Each 3D image content corresponds to a 3D frame that constitutes the 3D video content (3D video frame type distribution). The transmitter 10 includes 3D frame time information in the transmitted data SD. The 3D frame time information may be added to the text data 102, or it may be separate information from the text data 102.

[0061] The receiver 30 uses the image generation device 60 to generate multiple 3D frames from the text data 102 and time information of multiple 3D frames, and generates 3D video content by continuously playing back the generated multiple 3D frames. The image generation device 60 may be configured using a stereoscopic video generation AI (for example, a 4D diffusion model).

[0062] Furthermore, 3D video frame type distribution may also be 3D video frame representative type distribution, which transmits only one 3D frame (representative 3D frame) that represents multiple 3D frames. In the case of 3D video frame representative type distribution, the transmitter 10 includes 3D frame start time information and 3D frame end time information in the transmitted data SD. The 3D frame start time information and 3D frame end time information may be added to the text data 102, or they may be separate information from the text data 102. The transmitter 10 may also include information representing the temporal behavior of the subject (3D behavior information) in the transmitted data SD. The 3D behavior information may be added to the text data 102, or they may be separate information from the text data 102. The receiver 30 may use the image generation device 60 to generate multiple 3D frames from the text data 102 and the 3D behavior information to cover the period from the 3D frame start time information of the 3D frame start time to the 3D frame end time information of the 3D frame end time (between consecutive representative 3D frames). The image generation device 60 may also be configured using a time-series interpolation generation AI (for example, a temporal diffusion model) for three-dimensional spatial information.

[0063] (Experimental variation 6) Figure 9 shows a modified example of a transmitter and receiver according to one embodiment. In Figure 9, parts corresponding to parts in Figure 1 are given the same reference numerals, and their descriptions are omitted. The transmitter 10a shown in Figure 9 further includes an image recognition unit 50a in addition to the transmitter 10 in Figure 1. The image recognition unit 50a has an image recognition function similar to the image recognition device 50. The image recognition unit 50a may be configured using image recognition AI (e.g., semantic segmentation technology). The receiver 30a shown in Figure 9 further includes an image generation unit 60a in addition to the receiver 30 in Figure 1. The image generation unit 60a may be configured using image generation AI (e.g., diffusion model).

[0064] According to Modification 6, since the transmitter 10a has an internal image recognition function, it is no longer necessary to communicate with an external image recognition device 50 for image recognition. Also, since the receiver 30a has an internal image generation function, it is no longer necessary to communicate with an external image generation device 60 for image generation.

[0065] The above is an explanation of the modified form.

[0066] Furthermore, the above-described embodiment makes it possible to improve the overall service quality in, for example, video streaming services, thereby contributing to Goal 9 of the United Nations-led Sustainable Development Goals (SDGs), "Build resilient infrastructure, promote sustainable industrialization and foster innovation."

[0067] Although embodiments of the present invention have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments, and design modifications and the like are also included within the scope of the gist of the present invention.

[0068] For example, a computer program to implement the functions of each of the above-mentioned devices may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be loaded into a computer system and executed. The term "computer system" here may include hardware such as an operating system and peripheral devices. Furthermore, "computer system" shall also include the homepage provisioning environment (or display environment) if a WWW system is being used. Furthermore, "computer-readable recording media" refers to writable non-volatile memory such as flexible disks, magneto-optical disks, ROMs, and flash memory, portable media such as DVDs (Digital Versatile Discs), and storage devices such as hard disks built into computer systems.

[0069] Furthermore, "computer-readable recording media" also includes volatile memory (such as DRAM (Dynamic Random Access Memory)) within computer systems that act as servers or clients when programs are transmitted via networks such as the Internet or communication lines such as telephone lines, which retain programs for a certain period of time. Furthermore, the above program may be transmitted from a computer system that stores the program in a memory device or the like to another computer system via a transmission medium or by transmission waves within the transmission medium. Here, the "transmission medium" used to transmit the program refers to a medium that has the function of transmitting information, such as a network (communication network) like the Internet or a communication line (communication line) like a telephone line. Furthermore, the above program may be intended to implement some of the functions described above. It may also be a so-called differential file (differential program) that can implement the aforementioned functions in combination with programs already recorded in the computer system. [Explanation of Symbols]

[0070] 10...Transmitter, 11...Data acquisition unit, 12...Transmission unit, 13...Split setting unit, 30...Receiver, 31...Receiving unit, 32...Image acquisition unit, 50...Image recognition device, 50a...Image recognition unit, 60...Image generation device, 60a...Image generation unit, NW...Communication network

Claims

1. A data acquisition unit acquires text data representing the subject in each region of the original image, which is divided into multiple regions. A transmission unit that transmits transmission data representing the text data for each of the aforementioned regions, A receiving unit that receives the aforementioned transmitted data, An image acquisition unit that acquires an image generated based on the text data of each region represented by the transmitted data, The system includes a division setting unit for setting the division method of the original image, The division setting unit changes the number of pixels per region into which the original image is divided, according to the load state of the communication network used for transmitting the transmission data. Image transmission and reception system.

2. The aforementioned text data is text data representing the part of the subject. The image transmission and reception system according to claim 1.

3. The data acquisition unit further acquires metadata representing the state of the subject captured in each region within the original image, The transmitted data further includes the metadata, The image acquired by the aforementioned image acquisition unit is an image generated based on the aforementioned metadata. The image transmission and reception system according to claim 1.

4. Each of the aforementioned text data has an identifier assigned to it, and the text data has an associated identifier. The transmitted data includes the identifier assigned to each of the text data in each of the regions, The image transmission and reception system according to any one of claims 1 to 3.

5. The transmitted data further includes the corresponding data, The image transmission and reception system according to claim 4.

6. The identifier included in the transmitted data is compressed using a predetermined data compression method. The image transmission and reception system according to claim 4.

7. A data acquisition unit acquires text data representing the subject in each region of the original image, which is divided into multiple regions. A transmission unit that transmits transmission data representing the text data for each of the aforementioned regions, The system includes a division setting unit for setting the division method of the original image, The division setting unit changes the number of pixels per region into which the original image is divided, according to the load state of the communication network used for transmitting the transmission data. Transmitter.

8. Image recognition unit that generates the text data from the original image, The transmitter according to claim 7, further comprising:

9. A receiving unit that receives the transmission data transmitted from the transmitter described in any one of claims 7 or 8, An image acquisition unit that acquires an image generated based on the text data of each region represented by the transmitted data, A receiver equipped with the following features.

10. An image generation unit that generates the image from the text data of each region represented by the transmitted data, The receiver according to claim 9, further comprising:

11. In the transmitter's computer, A data acquisition step in which, for each region within the original image in which the subject is captured is obtained, the original image in which the subject is captured is divided into multiple regions, and text data representing the subject captured in the region is obtained. A transmission step of transmitting transmission data representing the text data for each of the aforementioned regions, The division setting step, which sets the division method for the original image, is executed, The division setting step changes the number of pixels per region into which the original image is divided, according to the load state of the communication network used for transmitting the transmitted data. Computer program.

12. In the receiver's computer, A receiving step of receiving the transmission data transmitted from the transmitter described in any one of claims 7 or 8, Image acquisition step: acquires an image generated based on the text data of each region represented by the transmitted data, A computer program designed to execute something.

13. An image transmission and reception method performed by an image transmission and reception system, A data acquisition step in which, for each region within the original image in which the subject is captured is obtained, the original image in which the subject is captured is divided into multiple regions, and text data representing the subject captured in the region is obtained. A transmission step of transmitting transmission data representing the text data for each of the aforementioned regions, A receiving step in which the aforementioned transmission data is received, Image acquisition step: acquires an image generated based on the text data of each region represented by the transmitted data, The division setting step includes setting a method for dividing the original image, The division setting step changes the number of pixels per region into which the original image is divided, according to the load state of the communication network used for transmitting the transmitted data. Image transmission and reception methods.

Images