Image generation methods and apparatus

Abstract

This application provides an image generation method and apparatus applicable to devices with projection functions, such as televisions, vehicle lights, and projection lamps. The method includes: acquiring a first parameter set, which includes a first original image, a second original image, the distance from a first projector to a first plane, the distance to a second plane, the desired position of a first desired image, and the desired position of a second desired image; determining a first mapping relationship and a second mapping relationship for the two planes; and determining the first desired image and the second desired image for the two planes based on the parameters to obtain a first target image finally presented on the two planes. Therefore, because different mapping relationships between the desired image and the original image are used for the two different planes, the first target image will not suffer from image distortion or other problems, improving the display quality of the image projected onto the plane and expanding the display area of ​​the image projected onto different planes.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence (AI), and more particularly to a method and apparatus for image generation. Background Technology

[0002] Currently, with the development of projection technology, the application of digital projection lighting is becoming increasingly widespread. For example, car lights are no longer limited to illumination; in the field of driver assistance, intelligent car lights can project more complex graphics such as text or traffic signs; in entertainment scenarios, intelligent car lights can project videos and other images.

[0003] However, during the projection process, when there are at least two projection planes, if the distance between the projection device and the at least two planes changes, it is not possible to quickly and accurately complete the projection on the at least two planes and ensure image quality. This may result in image distortion due to the presence of at least two projection planes. Summary of the Invention

[0004] This application provides an image generation method and apparatus that can improve the display quality of images projected onto different planes and avoid image distortion caused by different planes.

[0005] In a first aspect, a method for image generation is provided, the method comprising: acquiring a first parameter set, the first parameter set including: a first original image displayed on a first plane, a second original image displayed on a second plane, a distance from a first projector to the first plane, a distance from the first projector to the second plane, a desired position of a first desired image displayed on the first plane, and a desired position of a second desired image displayed on the second plane, wherein the first original image and the second original image belong to original images projected by the first projector, and the first desired image and the second desired image belong to desired images projected by the first projector. The first plane is connected to the second plane; a first mapping relationship and a second mapping relationship are determined, wherein the first mapping relationship is the mapping relationship between the first desired image and the first original image, and the second mapping relationship is the mapping relationship between the second original image; the first desired image is determined based on the first original image and the first mapping relationship, and the second desired image is determined based on the second original image and the second mapping relationship; a first target image is obtained based on the first desired image, the desired position of the first desired image, the second desired image, and the desired position of the second desired image, and the first target image is displayed on the first plane and the second plane by the first projector.

[0006] For example, let the coordinate position of the first desired image be Pos. imagedst1According to the formula Pos mapdst1 ·M′ src→dst1 =Pos imagedst1 Obtain the coordinates of the first desired image mapped from the first original image, where Pos mapdst1 Let M′ be the coordinates of the first original image. src→dst1 This represents the first mapping relationship output by the first neural network model. Similarly, according to the formula Pos... mapdst2 ·M′ src→dst2 =Pos imagedst2 Obtain the coordinates of the second desired image mapped from the second original image, where Pos mapdst2 M′ represents the coordinates of the second original image. src→dst2 This is the second mapping relationship output by the first neural network model.

[0007] Furthermore, the first desired image and the second desired image are fused to obtain the first target image. For example, the desired positions of the first and second desired images are represented by coordinates. The region formed by the desired positions of the first and second desired images is the effective region. Let the horizontal coordinate range of the effective region be [-x, x] and the vertical coordinate range be [-y, y], then the mapping is valid. The first and second desired images are then fused into a single image, namely, the first target image. The first target image is the image projected onto the first and second planes by the first projector.

[0008] Among them, acquiring the first original image and the second original image refers to acquiring parameters such as image resolution, image size, and image color of the first original image and the second original image. The expected position of the first expected image and the expected position of the second expected image can be represented by the expected coordinates of the first expected image and the second expected image. Optionally, parameters such as the brightness of the expected image may also be included. The first plane and the second plane are connected, which means that the first plane and the second plane are seamlessly connected and the angle formed is not equal to 0°, 180°, or 360°.

[0009] Based on the above scheme, after obtaining the first parameter set, the mapping relationship between the first desired image and the first original image corresponding to the distance from the first projector to the first plane, and the mapping relationship between the second desired image and the second original image corresponding to the distance from the first projector to the second plane are determined respectively. Then, based on the above mapping relationship, the first original image, and the second original image, the first desired image and the second desired image are determined respectively. Finally, based on the first desired image, the desired position of the first desired image, the second desired image, and the desired position of the second desired image, a continuous first target image that can be formed and presented on two different planes is obtained. Furthermore, since different mapping relationships are used for different planes, the first target image will not suffer from image distortion or other problems, thus improving the display quality of the image projected onto the plane.

[0010] In conjunction with the first aspect, in some implementations of the first aspect, determining the first mapping relationship and the second mapping relationship includes: inputting the first original image and the distance from the first projector to the first plane into a first neural network model to obtain the first mapping relationship; inputting the second original image and the distance from the first projector to the second plane into the first neural network model to obtain the second mapping relationship; the first mapping relationship corresponds to the distance from the first projector to the first plane; and the second mapping relationship corresponds to the distance from the first projector to the second plane.

[0011] Based on the above scheme, by training a neural network model, the neural network model can predict or obtain the mapping relationship for different distances and different planes. This eliminates the need for manual calibration for each projection at different distances, simplifies the image projection process, and improves the flexibility of image projection.

[0012] In conjunction with the first aspect, in some implementations of the first aspect, the first neural network model is trained based on a second set of parameters, which includes: the distance from the first projector to the first reference plane, a first training mapping relationship, the distance from the first projector to the second reference plane, and a second training mapping relationship. The first training mapping relationship is a mapping relationship between a first training desired image and a first training original image, and the second training mapping relationship is a mapping relationship between a second training desired image and a second training original image. The first training original image and the second training original image are training original images projected by the first projector. The first training desired image is displayed on the first reference plane, the second training desired image is displayed on the second reference plane, and the first reference plane and the second reference plane are connected.

[0013] It should be understood that the first plane and the first reference plane can be the same plane or different planes; the second plane and the second reference plane can also be the same plane or different planes. When the first plane and the first reference plane are the same plane, and the second plane and the second reference plane are the same plane, the distance from the first projector to the first plane can include the distance from the first projector to the first reference plane, and the distance from the first projector to the second plane can include the distance from the first projector to the second reference plane. The number of distances from the first projector to the first reference plane is equal to or less than the number of distances from the first projector to the first plane, and the number of distances from the first projector to the second reference plane is equal to or less than the number of distances from the first projector to the second plane.

[0014] In conjunction with the first aspect, in some implementations of the first aspect, the first training mapping relationship is determined based on the parameters of the first training projection image displayed on the first reference plane, the parameters of the first training original image, and the parameters of the first training desired image; the second training mapping relationship is determined based on the parameters of the second training projection image displayed on the second reference plane, the parameters of the second training original image, and the parameters of the second training desired image. The first training projection image is the image of the first training original image projected onto the first reference plane by the first projector; the second training projection image is the image of the second training original image projected onto the second reference plane by the first projector; the first training mapping relationship corresponds to the distance from the first projector to the first reference plane; and the second training mapping relationship corresponds to the distance from the first projector to the second reference plane.

[0015] For example, the coordinate position of the calibration point in the first training image is set to Pos. srcl1 The first training original image is projected onto the first reference plane through the first projector to obtain the first training projected image, and the coordinates of the obtained calibration points are set as Pos. map1 Set the coordinates of the first training desired image to Pos. mapdst1 The first training mapping relationship can be obtained through the following method:

[0016] 1. Establish the mapping relationship between the first training original image and the first training projected image.

[0017] Let M be the mapping coefficient between the first training projected image and the first training original image. dstI→srcI1 According to Pos map1 ·M map1→srcI1 =Pos srcl1 By solving using the least squares method, we obtain M. map1→srcI1 .

[0018] 2. Calculate the coordinates of the first training desired image mapped to the first training original image.

[0019] Let Pos be the coordinates of the first training desired image mapped to the first training original image. dst1 According to Pos dst1 =Pos mapdst1 ·M map→srcI1 By solving using the least squares method, Pos is obtained. dst1 .

[0020] 3. Calculate the mapping relationship between the first training original image and the first training expected image.

[0021] Let M be the mapping coefficient between the first training original image and the first training desired image. scr1→dst1 According to Pos dst1 =Pos srcl1 ·M scr1→dst1 By solving using the least squares method, we obtain M. scr1→dst1 .

[0022] It should be understood that the calculation method for the second training mapping relationship is the same as that for the first training mapping relationship. Only the first original training image, the first projected training image, and the first expected training image need to be replaced with the second original training image, the first projected training image, and the second expected training image. For the sake of brevity, this application will not elaborate further.

[0023] The first training projection image is the image of the first training original image projected onto the first reference plane by the first projector, and the second training projection image is the image of the second training original image projected onto the second reference plane by the first projector. Furthermore, the first training projection image and the second training projection image are images obtained after calibration, and the parameters of the training projection image include the position parameters of the training projection image.

[0024] It should be understood that the calculation method for the second training mapping relationship is the same as that for the first training mapping relationship. Only the first original training image, the first projected training image, and the first expected training image need to be replaced with the second original training image, the first projected training image, and the second expected training image. For the sake of brevity, this application will not elaborate further.

[0025] It should also be understood that the training projection image can be an image projected by a projector, a light projected by a projector, or another carrier projected by a projector that can obtain coordinate positions; this application does not limit this.

[0026] In conjunction with the first aspect, in certain implementations of the first aspect, when the first original image and the second original image constitute part of an image, a third parameter set is obtained. This third parameter set includes: a third original image displayed on the first plane, a fourth original image displayed on the second plane, the distance from the second projector to the first plane, the desired position of the third desired image displayed on the first plane, and the desired position of the fourth desired image displayed on the second plane. The third original image and the fourth original image belong to the original images projected by the second projector, and the third desired image and the fourth desired image belong to the desired images projected by the second projector. The original image projected by the first projector and the second original image constitute part of an image. The original image projected by the projector constitutes the image; a third mapping relationship and a fourth mapping relationship are determined, wherein the third mapping relationship is the mapping relationship between the third expected image and the third original image, and the fourth mapping relationship is the mapping relationship between the fourth expected image and the fourth original image; the third expected image is determined based on the third original image and the third mapping relationship, and the fourth expected image is determined based on the fourth original image and the fourth mapping relationship; a second target image is obtained based on the third expected image, the expected position of the third expected image, the fourth expected image, and the expected position of the fourth expected image, and the second target image is displayed on the first plane and the second plane by the second projector, wherein the first target image and the second target image respectively constitute a part of the projected target image.

[0027] The second projector and the first projector are devices with the same projection function. The two projectors can project part or all of the content of an image, and the two projectors are located on the same plane.

[0028] Based on the above scheme, two projected images can be obtained by projecting images from the first projector and the second projector respectively, thereby increasing the display area of ​​the projected images displayed on the first plane and the second plane, or improving their display quality.

[0029] In conjunction with the first aspect, in some implementations of the first aspect, determining the third mapping relationship and the fourth mapping relationship includes: inputting the third original image and the distance from the second projector to the first plane into a second neural network model to obtain the third mapping relationship; inputting the fourth original image and the distance from the second projector to the second plane into the second neural network model; the third mapping relationship corresponds to the distance from the second projector to the first plane; and the fourth mapping relationship corresponds to the distance from the second projector to the second plane.

[0030] It should be understood that the second neural network model and the first neural network model can be different neural network models or the same neural network model, and this application does not limit this.

[0031] In conjunction with the first aspect, in some implementations of the first aspect, the second neural network model is trained based on a fourth parameter set, which includes: the distance from the second projector to the third reference plane, a third training mapping relationship, the distance from the second projector to the fourth reference plane, and a fourth training mapping relationship. The third training mapping relationship is a mapping relationship between the third training expected image and the third training original image, and the fourth training mapping relationship is a mapping relationship between the fourth training expected image and the fourth training original image. The third training original image and the fourth training original image are training original images projected by the second projector. The third training expected image is displayed on the third reference plane, the fourth training expected image is displayed on the fourth reference plane, and the third reference plane is connected to the fourth reference plane.

[0032] In conjunction with the first aspect, in some implementations of the first aspect, the third training mapping relationship is determined based on the parameters of the third training projection image displayed on the third reference plane, the third training original image, and the expected position of the third training expected image; the fourth training mapping relationship is determined based on the parameters of the fourth training projection image displayed on the fourth reference plane, the fourth training original image, and the expected position of the fourth training expected image; wherein the third training projection image is the image of the third training original image projected onto the second reference plane by the first projector, and the fourth training projection image is the image of the fourth training original image projected onto the second reference plane by the second projector.

[0033] In conjunction with the first aspect, in some implementations of the first aspect, the projected target image is obtained based on the first target image and the second target image, wherein the first target image and the second target image are aligned on the horizontal plane.

[0034] In conjunction with the first aspect, in some implementations of the first aspect, when the area of ​​the projected target image is equal to the sum of the areas of the first target image and the second target image, the first target image and the second target image have no overlapping portion; or when the area of ​​the projected target image is equal to the area of ​​either the first target image or the second target image, the first target image and the second target image completely overlap, and the area of ​​the first target image is equal to the area of ​​the second target image; or when the area of ​​the projected target image is less than the sum of the first target image and the second target image, but greater than the area of ​​either the first target image or the second target image, the first target image and the second target image partially overlap, and the area of ​​the first target image is equal to the area of ​​the second target image.

[0035] Based on the above scheme, a second projector, using the same method as the first projector, applies different mapping relationships to different planes, ultimately displaying the second target image on both the first and second planes. Finally, the first and second target images are fused to obtain the final projected target image. This projected target image, compared to the first and second target images, has the advantages of a larger display area and / or better display quality.

[0036] In a second aspect, an image generation apparatus is provided, comprising: an acquisition unit configured to acquire a first parameter set, the first parameter set including: a first original image displayed on a first plane, a second original image displayed on a second plane, a desired position of a first desired image displayed on the first plane, and a desired position of a second desired image displayed on the second plane, wherein the first original image and the second original image belong to original images projected by a first projector, and the first desired image and the second desired image belong to desired images projected by the first projector, and the first plane and the second plane are connected; the acquisition unit is further configured to acquire the distance from the first projector to the first plane, and the distance from the first projector to the second plane. The processing unit is configured to determine a first mapping relationship and a second mapping relationship, wherein the first mapping relationship is a mapping relationship between the first desired image and the first original image, and the second mapping relationship is a mapping relationship between the second desired image and the second original image; the processing unit is further configured to determine the first desired image based on the first original image and the first mapping relationship, and to determine the second desired image based on the second original image and the second mapping relationship; the processing unit is further configured to obtain a first target image based on the first desired image, the desired position of the first desired image, the second desired image, and the desired position of the second desired image, wherein the first target image is used to be displayed on the first plane and the second plane by the first projector.

[0037] Based on the above scheme, after obtaining the first parameter set, the mapping relationship between the first desired image and the first original image corresponding to the distance from the first projector to the first plane, and the mapping relationship between the second desired image and the second original image corresponding to the distance from the first projector to the second plane are determined respectively. Then, based on the above mapping relationship, the first original image, and the second original image, the first desired image and the second desired image are determined respectively. Finally, based on the first desired image, the desired position of the first desired image, the second desired image, and the desired position of the second desired image, a continuous first target image that can be formed and presented on two different planes is obtained. Furthermore, since different mapping relationships are used for different planes, the first target image will not suffer from image distortion or other problems, thus improving the display quality of the image projected onto the plane.

[0038] In conjunction with the second aspect, in some implementations of the second aspect, the processing unit is specifically used to input the first original image and the distance from the first projector to the first plane into the first neural network model to obtain the first mapping relationship, and to input the second original image and the distance from the first projector to the second plane into the first neural network model to obtain the second mapping relationship.

[0039] Based on the above scheme, by training a neural network model, the neural network model can predict or obtain the mapping relationship for different distances and different planes. This eliminates the need for manual calibration for each projection at different distances, simplifies the image projection process, and improves the flexibility of image projection.

[0040] In conjunction with the second aspect, in some implementations of the second aspect, the first neural network model is trained based on a second set of parameters, which includes: the distance from the first projector to the first reference plane, a first training mapping relationship, the distance from the first projector to the second reference plane, and a second training mapping relationship. The first training mapping relationship is a mapping relationship between a first training desired image and a first training original image, and the second training mapping relationship is a mapping relationship between a second training desired image and a second training original image. The first training original image and the second training original image are training original images projected by the first projector. The first training desired image is displayed on the first reference plane, the second training desired image is displayed on the second reference plane, and the first reference plane and the second reference plane are connected.

[0041] In conjunction with the second aspect, in some implementations of the second aspect, the processing unit is further configured to determine the first training mapping relationship based on the parameters of the first training projection image displayed on the first reference plane, the first training original image, and the expected position of the first training expected image, and to determine the second training mapping relationship based on the parameters of the second training projection image displayed on the second reference plane, the second training original image, and the expected position of the second training expected image, wherein the first training projection image is an image of the first training original image projected onto the first reference plane by the first projector, the second training projection image is an image of the second training original image projected onto the second reference plane by the first projector, the first training mapping relationship corresponds to the distance from the first projector to the first reference plane, and the second training mapping relationship corresponds to the distance from the first projector to the second reference plane.

[0042] In conjunction with the second aspect, in certain implementations of the second aspect, when the first original image and the second original image constitute part of an image, the acquisition unit is configured to acquire the desired positions of the third original image displayed on the first plane, the fourth original image displayed on the second plane, the third desired image displayed on the first plane, and the fourth desired image displayed on the second plane, wherein the third original image and the fourth original image belong to the original images projected by the second projector, the third desired image and the fourth desired image belong to the desired images projected by the second projector, and the original images projected by the first projector and the original images projected by the second projector constitute the image; the acquisition unit is further configured to acquire the distance from the second projector to the first plane, and the distance from the second projector to the first plane. The processing unit is further configured to determine a third mapping relationship and a fourth mapping relationship, wherein the third mapping relationship is a mapping relationship between the third expected image and the third original image, and the fourth mapping relationship is a mapping relationship between the fourth expected image and the fourth original image; the processing unit is further configured to determine the third expected image based on the third original image and the third mapping relationship, and to determine the fourth expected image based on the fourth original image and the fourth mapping relationship; the processing unit is further configured to obtain a second target image based on the third expected image, the expected position of the third expected image, the fourth expected image, and the expected position of the fourth expected image, wherein the second target image is displayed on the first plane and the second plane through the second projector, and the first target image and the second target image constitute an image.

[0043] Based on the above scheme, two projected images can be obtained by projecting images from the first projector and the second projector respectively, thereby increasing the display area of ​​the projected images displayed on the first plane and the second plane, or improving their display quality.

[0044] In conjunction with the second aspect, in some implementations of the second aspect, the processing unit is further specifically configured to input the third original image and the distance from the second projector to the first plane into the second neural network model to obtain the third mapping relationship, and input the fourth original image and the distance from the second projector to the second plane into the second neural network model to obtain the fourth mapping relationship, wherein the third mapping relationship corresponds to the distance from the second projector to the first plane, and the fourth mapping relationship corresponds to the distance from the second projector to the second plane.

[0045] In conjunction with the second aspect, in some implementations of the second aspect, the second neural network model is trained based on a fourth parameter set, which includes: the distance from the second projector to the third reference plane, a third training mapping relationship, the distance from the second projector to the fourth reference plane, and a fourth training mapping relationship. The third training mapping relationship is a mapping relationship between the third training desired image and the third training original image; the fourth training mapping relationship is a mapping relationship between the fourth training desired image and the fourth training original image; the third training original image and the fourth training original image are training original images projected by the second projector; the third training desired image is displayed on the third reference plane; the fourth training desired image is displayed on the fourth reference plane; and the third reference plane is connected to the fourth reference plane.

[0046] In conjunction with the second aspect, in some implementations of the second aspect, the processing unit is further configured to determine the third training mapping relationship based on the parameters of the third training projection image displayed on the third reference plane, the third training original image, and the expected position of the third training expected image, and to determine the fourth training mapping relationship based on the parameters of the fourth training projection image displayed on the fourth reference plane, the fourth training original image, and the expected position of the fourth training expected image, wherein the third training projection image is an image of the third training original image projected onto the second reference plane by the first projector, the fourth training projection image is an image of the fourth training original image projected onto the second reference plane by the second projector, the third training mapping relationship corresponds to the distance from the second projector to the third reference plane, and the fourth training mapping relationship corresponds to the distance from the second projector to the second reference plane.

[0047] In conjunction with the second aspect, in some implementations of the second aspect, the processing unit is further configured to obtain the projected target image based on the first target image and the second target image, wherein the first target image and the second target image are aligned on a horizontal plane.

[0048] In conjunction with the second aspect, in some implementations of the second aspect, when the area of ​​the projected target image is equal to the sum of the areas of the first target image and the second target image, the first target image and the second target image have no overlapping portion; or when the area of ​​the projected target image is equal to the area of ​​either the first target image or the second target image, the first target image and the second target image completely overlap, and the area of ​​the first target image is equal to the area of ​​the second target image; or when the area of ​​the projected target image is less than the sum of the first target image and the second target image, but greater than the area of ​​either the first target image or the second target image, the first target image and the second target image partially overlap, and the area of ​​the first target image is equal to the area of ​​the second target image.

[0049] Based on the above scheme, a second projector, using the same method as the first projector, applies different mapping relationships to different planes, ultimately displaying the second target image on both the first and second planes. Finally, the first and second target images are fused to obtain the final projected target image. This projected target image, compared to the first and second target images, has the advantages of a larger display area and / or better display quality.

[0050] Thirdly, an image generation apparatus is provided, the apparatus comprising: a memory for storing a program; and at least one processor for executing the computer program or instructions stored in the memory to perform the method provided in the first aspect or any of the above-described implementations of the first aspect.

[0051] In one implementation, the device is a projector, television, vehicle lights, or vehicle.

[0052] In another implementation, the device is a chip, chip system, or circuit used in a projector, television, car light, or vehicle.

[0053] Fourthly, this application provides a processor for performing the method provided in the first aspect above.

[0054] Unless otherwise specified, or if it does not contradict its actual function or internal logic in the relevant description, the transmission and acquisition / reception operations involved in the processor can be understood as processor output and reception, input and other operations, or as transmission and reception operations performed by radio frequency circuits and antennas. This application does not limit them in this regard.

[0055] Fifthly, a computer-readable storage medium is provided that stores program code for execution by a device, the program code including a method for performing the first aspect or any of the above-described implementations of the first aspect.

[0056] In a sixth aspect, a computer program product containing instructions is provided, which, when run on a computer, causes the computer to perform the method provided in the first aspect or any of the above implementations of the first aspect.

[0057] In a seventh aspect, a chip is provided, the chip including a processor and a communication interface, the processor reading instructions stored in a memory through the communication interface and executing the method provided in the first aspect or any of the above implementations of the first aspect.

[0058] Optionally, as one implementation, the chip further includes a memory storing computer programs or instructions, and a processor for executing the computer programs or instructions stored in the memory. When the computer programs or instructions are executed, the processor is used to execute the method provided by the first aspect or any of the above implementations of the first aspect.

[0059] Eighthly, an electronic device is provided, including the image generating apparatus as described in the second aspect.

[0060] Alternatively, as one implementation, the electronic device may include, but is not limited to, a projector, a television, a vehicle headlight, a vehicle, or any device with projection capabilities. Attached Figure Description

[0061] Figure 1 A schematic diagram of the system architecture of an image generation system 100 is shown.

[0062] Figure 2 A schematic diagram of the system architecture of an image generation system 200 is shown.

[0063] Figure 3 A schematic diagram of the system architecture 300 provided in an embodiment of this application is shown.

[0064] Figure 4 A schematic diagram of a convolutional neural network 400 provided in an embodiment of this application is shown.

[0065] Figure 5 A schematic diagram of the chip hardware structure 500 provided in an embodiment of this application is shown.

[0066] Figure 6 A schematic diagram of the headlight projection is shown.

[0067] Figure 7 Another schematic diagram of the vehicle headlight projection is shown.

[0068] Figure 8 A schematic diagram of an image generation method 800 provided in an embodiment of this application is shown.

[0069] Figure 9 A schematic diagram of an electronic device 900 provided in an embodiment of this application is shown.

[0070] Figure 10 A schematic diagram of the hardware structure of an image generation apparatus 1000 according to an embodiment of this application is shown.

[0071] Figure 11 A schematic diagram of the hardware structure of a neural network training device 1100 according to an embodiment of this application is shown. Detailed Implementation

[0072] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0073] All or part of the steps in the image generation method provided in this application embodiment can be executed by an image generation system.

[0074] Figure 1 A schematic diagram of the system architecture of an image generation system 100 is shown. This system 100 can exist in projection devices, televisions, vehicle lights, and vehicles including vehicle lights, all possessing information acquisition and processing capabilities. Figure 1 As shown, now for Figure 1 A brief introduction to each unit shown in the image is provided.

[0075] 1. First acquisition unit: used to acquire the parameters of the training desired image, the original training image, the desired position of the desired image, and the original image. The training expectation image is the input image used to calculate the training mapping relationship. It can be divided into a first training expectation image displayed on the first reference plane and a second training expectation image displayed on the second reference plane. The parameters of the training expectation image can include the expected coordinates of the training expectation image, and optionally, it can also include parameters such as the pixel value of the training expectation image. The training original image is the image that is to be projected onto the reference plane for calculating the training mapping relationship. It can be divided into a first training original image displayed on the first reference plane and a second training original image displayed on the second reference plane. The training original image can include parameters such as the image resolution, image size, image color, and image content of the training original image. The expectation image is the image obtained after mapping the original image. It can be divided into a first expectation image displayed on the first plane and a second expectation image displayed on the second plane. The expected position of the expectation image can be represented by coordinates. Optionally, it can also obtain parameters such as the pixel value of the expectation image. The original image is the image that is to be projected onto the plane. It can be divided into a first original image displayed on the first plane and a second original image displayed on the second plane. The original image can include parameters such as the image resolution, image size, image color, and image content of the original image. The first plane and the second plane are connected, and the first reference plane and the second reference plane are connected.

[0076] Specifically, the first acquisition unit may acquire the desired location and the original image of the desired image through external storage devices (e.g., portable hard drives, USB flash drives) and network transmission (including in-vehicle networks, local area networks, or the Internet).

[0077] 2. Second acquisition unit: This unit is used to acquire distances, including the distance from the first projection unit to the first plane, the distance from the first projection unit to the second plane, the distance from the first projection unit to the first reference plane, and the distance from the first projection unit to the second reference plane.

[0078] The first plane and the second plane are planes that carry the final generated image, and the first plane and the second plane are connected; the first reference plane and the second reference plane are reference planes used to calculate the training mapping relationship before training the neural network model, and the first reference plane and the second reference plane are connected.

[0079] Specifically, the second acquisition unit may include, but is not limited to, acquiring information through structured light, time-of-flight (TOF), or lidar, and the information may include the position information of the first plane, the position information of the second plane, the distance from the first projection unit to the first plane, and the distance from the first projection unit to the second plane.

[0080] 3. Third Acquisition Unit: Used to acquire training projection images. These training projection images are images projected onto a reference plane from the original training images before training the neural network model. The training projection images can be divided into a first training projection image displayed on a first reference plane and a second training projection image displayed on a second reference plane. Furthermore, the training projection images are calibrated. The training projection images may include parameters such as image resolution, image size, image color, and image content.

[0081] Specifically, the third acquisition unit may include, but is not limited to, a camera or other devices with camera functionality.

[0082] It should be understood that the training projection image can be an image projected by a projector, a light projected by a projector, or another carrier projected by a projector that can obtain coordinate positions; this application does not limit it here.

[0083] 4. Processing unit: used to calculate projection mapping relationships, train mapping relationships, train neural network models, and fuse and reconstruct at least two images.

[0084] The projection mapping relationship is the mapping relationship between the expected image and the original image corresponding to the distance between the first projection unit and the plane. It can include the mapping relationship between the first expected image and the first original image corresponding to the distance between the first projection unit and the first plane, and the mapping relationship between the second expected image and the second original image corresponding to the distance between the first projection unit and the second plane. The training mapping relationship is the mapping relationship between the training expected image and the training original image corresponding to the distance between the first projection unit and the reference plane before training the neural network model. It can include the mapping relationship between the first training expected image and the first training original image corresponding to the distance between the first projection unit and the first reference plane, and the mapping relationship between the second training expected image and the second training original image corresponding to the distance between the first projection unit and the second reference plane. The neural network model is used to predict the mapping relationship between the expected image and the original image. Reconstructing at least two images can be understood as reconstructing the first expected image and the second expected image to obtain a first target image including the first expected image and the second expected image. The first target image is used to be continuously displayed on the first plane and the second plane. Fusion of at least two images can be understood as fusing the first target image and the second target image to finally generate a projection target image projected onto the first plane and the second plane.

[0085] Specifically, the processing unit may include, but is not limited to, embedded development board devices, computer devices, and other devices with processing and computing capabilities.

[0086] 5. First projection unit: used to project the first original image and the second original image onto the first plane and the second plane to obtain the first target image.

[0087] Specifically, the first projection unit may include, but is not limited to, devices with projection functions such as televisions, vehicle lights, and projection lamps.

[0088] Figure 2 A schematic diagram of the system architecture of an image generation system 200 is shown. This system 200 can be found in projection devices, televisions, vehicle lights, and vehicles that have information acquisition and processing capabilities. Figure 2 As shown, the system architecture can be Figure 1 Further expansions of the system architecture shown can be found by referring to [reference needed]. Figure 1 The functions and descriptions of the corresponding units shown are as follows: Figure 2 The system architecture shown adds a second projection unit, which functions and is described similarly to the first projection unit. This allows the first and second projection units to further fuse their respective target images, ultimately displaying a larger, clearer, or brighter target image on the first and second planes.

[0089] It should be understood that the images mentioned above can be static images or dynamically displayed images in the form of videos, animations, etc., and this application does not limit them here.

[0090] It should also be understood that Figure 1 and Figure 2 The first and third acquisition units can be two independent acquisition units or integrated into one acquisition unit; this application does not limit this.

[0091] It should also be understood that the above Figure 1 and Figure 2 This application is merely illustrative and is not intended to be limited thereto. For example, Figure 1 and Figure 2 The image generation system 100 shown is based on projection onto two planes. The technical solution shown in this application is applicable to projection scenarios that project onto at least two different planes, and the number of specific planes is not limited.

[0092] To better understand the technical solution of this application, the relevant terms and concepts that may be involved in the embodiments of this application will be introduced below.

[0093] (1) Neural Network

[0094] Neural networks can be composed of neural units, which can refer to units represented by x. s The arithmetic unit takes the intercept 1 as input, and its output can be shown in formula (1-1):

[0095]

[0096] Where s = 1, 2, ..., n, n is a natural number greater than 1, W s For x s The weights are denoted by b, where b is the bias of the neural unit. f is the activation function of the neural unit, used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into an output signal. The output signal of this activation function can be used as the input of the next convolutional layer; the activation function can be the sigmoid function. A neural network is a network formed by connecting multiple of the above-mentioned individual neural units together; that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the features of the local receptive field, which can be a region composed of several neural units.

[0097] (2) Deep Neural Networks

[0098] A deep neural network (DNN), also known as a multilayer neural network, can be understood as a neural network with multiple hidden layers. Based on the position of the layers, the internal neural network of a DNN can be divided into three categories: input layer, hidden layer, and output layer. Generally, the first layer is the input layer, the last layer is the output layer, and the layers in between are hidden layers. The layers are fully connected, meaning that any neuron in the i-th layer is connected to any neuron in the (i+1)-th layer.

[0099] Although DNNs appear complex, the operation of each layer is actually not complicated. The function of each layer in a deep neural network can be expressed mathematically. To describe it: From a physical perspective, the work of each layer in a deep neural network can be understood as transforming the input space (the set of input vectors) to the output space (i.e., from the row space to the column space of a matrix) through five operations on the input space. These five operations include: 1. Dimensionality increase / decrease; 2. Magnification / scaling; 3. Rotation; 4. Translation; 5. "Bending". Operations 1, 2, and 3 are... The operation 4 is completed using +b, and the operation 5 is implemented using a(). The term "space" is used here because the objects being classified are not individual things, but a class of things; space refers to the set of all individuals within this class of things. Here, W is the weight vector, where each value represents the weight of a neuron in that layer of the neural network. This vector W determines the spatial transformation from the input space to the output space, as described above; that is, the weights W of each layer control how the space is transformed. The purpose of training a deep neural network is to ultimately obtain the weight matrix of all layers of the trained neural network (a weight matrix formed by the vectors W from many layers). Therefore, the training process of a neural network is essentially learning how to control spatial transformation, more specifically, learning the weight matrix.

[0100] Therefore, DNN can be simply expressed as the following linear relationship: in, It is the input vector. It is the output vector. α is the offset vector, W is the weight matrix (also called coefficients), and α() is the activation function. Each layer is simply an adjustment of the input vector. The output vector is obtained through such a simple operation. Because DNNs have many layers, the coefficients W and the offset vector... The number of these parameters is also relatively large. The definitions of these parameters in DNNs are as follows: Taking the coefficient W as an example: Assuming a three-layer DNN, the linear coefficient from the 4th neuron in the second layer to the 2nd neuron in the third layer is defined as... The superscript 3 represents the layer number where coefficient W is located, while the subscript corresponds to the third layer index 2 of the output and the second layer index 4 of the input.

[0101] In summary, the coefficient from the k-th neuron in layer L-1 to the j-th neuron in layer L is defined as...

[0102] It's important to note that the input layer does not have a W parameter. In deep neural networks, more hidden layers allow the network to better represent complex real-world situations. Theoretically, the more parameters a model has, the higher its complexity and "capacity," meaning it can perform more complex learning tasks. Training a deep neural network is essentially the process of learning the weight matrix, with the ultimate goal of obtaining the weight matrix of all layers in the trained deep neural network (a weight matrix formed by the vectors W from many layers).

[0103] (3) Convolutional Neural Network

[0104] A convolutional neural network (CNN) is a deep neural network with convolutional structures. A CNN contains a feature extractor consisting of convolutional layers and subsampling layers, which can be viewed as a filter. A convolutional layer is a layer of neurons in a CNN that performs convolutional processing on the input signal. In a convolutional layer of a CNN, a neuron may only be connected to some of its neighboring neurons. A convolutional layer typically contains several feature planes, each composed of rectangularly arranged neural units. Neural units on the same feature plane share weights, which are the convolutional kernel. Shared weights can be understood as the way audio information is extracted being independent of location. The convolutional kernel can be initialized as a matrix of random size, and during the training process of the CNN, the kernel can learn appropriate weights. Furthermore, the direct benefit of shared weights is reducing the connections between layers in the CNN, while also reducing the risk of overfitting.

[0105] (4) Recurrent Neural Network

[0106] Recurrent Neural Networks (RNNs) are used to process sequential data. In traditional neural network models, the layers from the input layer to the hidden layer and then to the output layer are fully connected, but the nodes within each layer are unconnected. While this type of neural network has solved many difficult problems, it remains inadequate for many others. For example, predicting the next word in a sentence generally requires using the preceding words because words in a sentence are not independent. RNNs are called recurrent neural networks because the current output of a sequence is related to the outputs of previous sequences. Specifically, the network memorizes previous information and applies it to the calculation of the current output. That is, nodes within the same hidden layer are no longer unconnected but connected, and the input to a hidden layer includes not only the output of the input layer but also the output of the previous hidden layer. Theoretically, RNNs can process sequential data of any length. Training an RNN is similar to training a traditional CNN or DNN. This algorithm also uses the backpropagation algorithm, but with one key difference: when an RNN is expanded, its parameters, such as W, are shared; however, this is not the case with traditional neural networks as illustrated above. Furthermore, in gradient descent, the output at each step depends not only on the network at the current step but also on the states of the network in previous steps. This learning algorithm is called backpropagation through time (BPTT).

[0107] Since we already have convolutional neural networks (CNNs), why do we need recurrent neural networks (RNNs)? The reason is simple. CNNs rely on the fundamental assumption that elements are independent of each other, and that input and output are also independent—like a cat and a dog. However, in the real world, many elements are interconnected. For example, stock prices fluctuate over time. Or, imagine someone saying, "I love traveling, and my favorite place is Yunnan. I definitely want to go there someday." Humans know the answer to this question is "Yunnan." Humans can infer from context, but how can machines do the same? This is where RNNs come in. RNNs aim to give machines the ability to remember, just like humans. Therefore, the output of an RNN depends on both the current input information and historical memory information.

[0108] (5) Attention mechanism

[0109] Attention mechanisms mimic the internal processes of biological observation—aligning internal experience with external senses to increase the precision of observation in specific areas. They enable the rapid sifting of high-value information from a large volume of data using limited attentional resources. Attention mechanisms can quickly extract important features from sparse data and are therefore widely used in natural language processing tasks, particularly machine translation. Self-attention mechanisms, an improvement on attention mechanisms, reduce reliance on external information and are better at capturing the internal correlations of data or features. The core idea of ​​attention mechanisms can be rewritten as follows:

[0110]

[0111] Where Lx = ||Source|| represents the length of Source, and the formula means that the constituent elements of Source can be imagined as a series of...<Key,Value> The data pairs are used to form an Attention mechanism. Given an element Query in the Target, the similarity or relevance between the Query and each Key is calculated to obtain the weight coefficient of the Value corresponding to each Key. Then, the Values ​​are weighted and summed to obtain the final Attention value. Essentially, the Attention mechanism is a weighted sum of the Values ​​of elements in the Source, while the Query and Key are used to calculate the weight coefficients of their corresponding Values. Conceptually, Attention can be understood as selectively filtering a small amount of important information from a large amount of information and focusing on this important information, ignoring most of the unimportant information. The focusing process is reflected in the calculation of the weight coefficients; the larger the weight, the more focused it is on its corresponding Value. That is, the weight represents the importance of the information, and the Value is the corresponding information. Self-attention can be understood as internal Attention. The Attention mechanism occurs between the Query element in the Target and all elements in the Source. Self-attention refers to the Attention mechanism that occurs between elements within the Source or between elements within the Target. It can also be understood as the attention calculation mechanism in the special case where Target = Source. The specific calculation process is the same; only the calculation object changes.

[0112] (6) Transformation Neural Network

[0113] Transformer neural networks are a type of neural network architecture based on a self-attention mechanism. A transformer neural network can include an encoder, a decoder, a loss function, and more.

[0114] The encoder consists of multiple identical layers, each containing two sub-layers. The first sub-layer is a multi-head attention layer, and the second sub-layer is a fully connected layer. Each sub-layer can add residual connections and normalization.

[0115] The decoder consists of multiple identical layers, each containing three sub-layers. These three sub-layers are a self-attention layer, an encoder-decoder attention layer, and a fully connected layer. The first two sub-layers are based on multi-head attention layers.

[0116] Transformation neural networks can be processed in parallel, reducing training time.

[0117] (7) Image Fusion

[0118] Image fusion refers to the process of combining image data of the same target acquired from multiple sources through image processing and computer technology to extract the most useful information from each source and finally synthesize it into a high-quality image. This process aims to improve the utilization rate of image information, enhance the accuracy and reliability of computer interpretation, and improve the spatial and spectral resolution of the original image.

[0119] It can combine two or more images into a new image through specific algorithms. The fusion result can take advantage of the spatiotemporal correlation and information complementarity of the two (or more) images, and make the fused image have a more comprehensive and clear description of the scene, which is more conducive to human eye recognition and machine automatic detection.

[0120] Image fusion uses images containing brightness, color, temperature, distance, and other features. These images can be presented as a single image or a series of images. Image fusion combines information from two or more images into a single image, resulting in a fused image that contains more information and is easier for humans to observe or computers to process. Image fusion can expand the temporal and spatial information contained in the original images, reduce uncertainty, increase reliability, and improve the robustness of the system.

[0121] Efficient image fusion methods can integrate information from multiple sources as needed, thereby effectively improving the utilization rate of image information. The aim is to integrate multi-band information from a single sensor or information from different types of sensors, eliminating potential redundancy and contradictions between multiple sensor information. This enhances the transparency of information in the image, improves the accuracy, reliability, and utilization rate of interpretation, and ultimately forms a clear, complete, and accurate description of the target.

[0122] (8) Pixel value

[0123] Pixel value is a value assigned by a computer when the original image is digitized. It represents the average brightness information of a small square in the original image, or the average reflectance (transmission) density information of that small square. When converting a digital image into a halftone image, the halftone dot area ratio (halftone dot percentage) is directly related to the pixel value (grayscale value) of the digital image; that is, the size of the halftone dot represents the average brightness information of a small square in the original image.

[0124] A pixel value in an image can be a red-green-blue (RGB) color value, which can be a long integer representing the color. For example, a pixel value of 256*Red+100*Green+76Blue, where Blue represents the blue component, Green represents the green component, and Red represents the red component. Within each color component, the smaller the value, the lower the brightness; the larger the value, the higher the brightness. For grayscale images, the pixel value can be a grayscale value.

[0125] (9) Projection

[0126] Projection refers to projecting the shape of an object onto a plane using a set of rays. The image obtained on that plane is also called a "projection". Projections can be divided into orthographic projection and oblique projection. Orthographic projection is in which the center line of the projection rays is perpendicular to the projection plane, while oblique projection is in which the center line of the projection rays is not perpendicular to the projection plane.

[0127] The system architecture provided in the embodiments of this application is described below.

[0128] Figure 3 This is a schematic diagram of the system architecture 300 provided in an embodiment of this application. Figure 3 As shown, the data acquisition device 360 ​​is used to collect training data and store the training data in the database 330. The training device 320 trains the target model / rule 301 based on the training data maintained in the database 330. The target model / rule 301 can process the input data.

[0129] The target model / rule 301 can be used to implement the image generation method provided in the embodiments of this application. That is, the distance between the first original image and the first projector to the first plane, and the distance between the second original image and the first projector to the second plane can be preprocessed and input into the target model / rule 301 to obtain the first mapping relationship and the second mapping relationship. The description of the first original image, the first projector, the first plane, the second original image, the second plane, the first mapping relationship, and the second mapping relationship can be referred to the description in the following embodiment 800.

[0130] Specifically, the neural network model can be trained using the distance from the first projector to the first reference plane, the first training mapping relationship, the distance from the first projector to the second reference plane, and the second training mapping relationship, and the number of groups can be one or more. In this embodiment, 500 training distances (i.e., the distance from the projector to the reference plane) and the corresponding training mapping relationship between the expected training image and the original training image can be used for modeling, and each training mapping relationship parameter is a 3x3 matrix. The 500 training distance parameters are used as input for training. Using deep learning, two hidden layers are set with parameters W1 = [9,1], b1 = [9,1], W2 = [9,9], and b2 = [9,1], respectively. The activation function is set to ReLU, the loss function is cross-entropy, and the learning rate is set to 1*e. -4 And, W1 and W2 are the weight matrix parameters, b1 and b2 are the bias matrix parameters. Finally, the Adam gradient descent optimizer is used to update the parameters, and the number of training iterations is set to 100,000. Note that the model can be trained using online tuning or offline training methods, which are not limited here. In this embodiment, a training mapping relationship is used to train the neural network model, but a mapping relationship calculated by the user can also be used as the training parameters, which is not limited here.

[0131] The target model / rule 301 in this application embodiment can specifically be a neural network model. In the embodiments provided in this application, the neural network model is obtained by training a CNN, a recurrent neural network, or a transform neural network. It should be noted that in practical applications, the training data maintained in the database 330 may not all come from the data acquisition device 360; it may also be received from other devices. Furthermore, it should be noted that the training device 320 may not necessarily train the target model / rule 301 entirely based on the training data maintained in the database 330; it may also obtain training data from the cloud or other sources for model training. The above description should not be construed as limiting the embodiments of this application.

[0132] The neural network model can be trained using a second set of parameters, which includes the distance from the first projector to the first reference plane, a first training mapping relationship, the distance from the first projector to the second reference plane, and a second training mapping relationship. The first training mapping relationship is the mapping between the first expected training image and the first original training image corresponding to the distance from the first projector to the first reference plane. The second training mapping relationship is the mapping between the second expected training image and the second original training image corresponding to the distance from the first projector to the second reference plane. The first and second original training images are the original training images projected by the first projector, used to train the neural network model, and are the input images that are expected to be projected onto the reference plane. The first expected training image is displayed on the first reference plane, and the second expected training image is displayed on the second reference plane. These images are used to train the neural network model and are the images expected to be projected onto the reference plane. The first and second reference planes are connected, and the angle they form does not include 0°, 180°, or 360°.

[0133] The target model / rule 301 trained using training device 320 can be applied to different systems or devices, such as... Figure 3 The execution device 310 shown can be a terminal, such as a mobile phone terminal, tablet computer, laptop computer, AR / VR, vehicle terminal, etc., or it can be a server or cloud service. Figure 3 In the process, the execution device 310 is equipped with an I / O interface 312 for data interaction with external devices. Users can input data to the I / O interface 312 through the client device 340.

[0134] Preprocessing modules 313 and 314 are used to preprocess the input data (i.e., the data to be processed, such as the distance between the first original image and the first projector to the first plane, and the distance between the second original image and the first projector to the second plane) received by the I / O interface 312. In this embodiment, preprocessing modules 313 and 314 may be omitted (or only one of them may be used), and the calculation module 311 may be used directly to process the input data.

[0135] During the preprocessing of input data by the execution device 310, or during the calculation module 311 of the execution device 310 performing calculations and other related processes, the execution device 310 can call data, code, etc. in the data storage system 350 for corresponding processing, or store the data, instructions, etc. obtained from the corresponding processing into the data storage system 350.

[0136] Finally, I / O interface 312 returns the processing results, such as the first mapping relationship and the second mapping relationship obtained above, to client device 340.

[0137] It is worth noting that the training device 320 can generate corresponding target models / rules 301 based on different training data for different objectives or tasks. The corresponding target models / rules 301 can be used to achieve the above objectives or complete the above tasks, thereby providing the user with the required results.

[0138] exist Figure 3 In the illustrated scenario, the user can manually provide input data, which can be done through the interface provided by I / O interface 312. Alternatively, the client device 340 can automatically send input data to I / O interface 312. If user authorization is required for the client device 340 to automatically send input data, the user can set the corresponding permissions in the client device 340. The user can view the output results of the execution device 310 on the client device 340, which can be presented in various forms such as display, sound, or animation. The client device 340 can also act as a data acquisition terminal, collecting the input data and output results of the input I / O interface 312 as shown in the figure, and storing them as new sample data in database 330. Alternatively, data can be collected directly from the I / O interface 312 without going through the client device 340, using the input data and output results of the input I / O interface 312 as shown in the figure, and storing them as new sample data in database 330.

[0139] It is worth noting that, Figure 3 This is merely a schematic diagram of the system architecture 300 provided in this application embodiment. The positional relationships between the devices, components, modules, etc. shown in the diagram do not constitute any limitation. For example, in Figure 3 In this context, the data storage system 350 is an external memory relative to the execution device 310. In other cases, the data storage system 350 may also be placed within the execution device 310.

[0140] It should be noted that, in the embodiments of this application, the training device 320 and the execution device 310 may be the same device or different devices. The client device 340 and the execution device 310 may be the same device or different devices. The training device 320 and the client device 340 may be the same device or different devices. This application does not impose any limitations on these aspects.

[0141] like Figure 3 As shown, the target model / rule 301 is obtained by training the training device 320. In this embodiment of the application, the target model / rule 301 may be a neural network model.

[0142] As introduced in the basic concepts above, a Convolutional Neural Network (CNN) is a deep neural network with a convolutional structure. It is a deep learning architecture, which refers to learning at multiple levels of abstraction using machine learning algorithms. As a deep learning architecture, a CNN is a feed-forward artificial neural network, in which each neuron can respond to the input data.

[0143] Figure 4 This is a schematic diagram of a convolutional neural network 400 provided in an embodiment of this application. Figure 4 As shown, the convolutional neural network (CNN) 400 may include an input layer 410, a convolutional / pooling layer 420 (where the pooling layer is optional), and a neural network layer 430.

[0144] Convolutional / pooling layers 420:

[0145] Convolutional layers:

[0146] like Figure 4 The convolutional / pooling layer 420 shown may include layers as in Examples 421-426. For instance, in one implementation, layer 421 is a convolutional layer, layer 422 is a pooling layer, layer 423 is a convolutional layer, layer 424 is a pooling layer, layer 425 is a convolutional layer, and layer 426 is a pooling layer; in another implementation, layers 421 and 422 are convolutional layers, layer 423 is a pooling layer, layers 424 and 425 are convolutional layers, and layer 426 is a pooling layer. That is, the output of the convolutional layer can be used as the input to a subsequent pooling layer, or as the input to another convolutional layer to continue the convolution operation.

[0147] The following section will use convolutional layer 421 as an example to introduce the internal working principle of a convolutional layer.

[0148] Convolutional layer 421 can include many convolution operators, also known as kernels. In data processing, a convolution operator acts as a filter that extracts specific information from the input information matrix. A convolution operator can essentially be a weight matrix, which is usually predefined.

[0149] The weight values ​​in these weight matrices need to be obtained through extensive training in practical applications. The weight matrices formed by the weight values ​​obtained through training can be used to extract information from the input data, thereby enabling the convolutional neural network 400 to make correct predictions.

[0150] When a convolutional neural network 400 has multiple convolutional layers, the initial convolutional layers (e.g., 421) tend to extract more general features, which can also be called low-level features. As the depth of the convolutional neural network 400 increases, the features extracted by later convolutional layers (e.g., 426) become more and more complex, such as high-level semantic features. Features with higher semantic levels are more suitable for the problem to be solved.

[0151] Pooling layer:

[0152] Because it is often necessary to reduce the number of training parameters, pooling layers are often introduced periodically after convolutional layers, such as... Figure 4 In the example of 420, layers 421-426 can be a convolutional layer followed by a pooling layer, or multiple convolutional layers followed by one or more pooling layers. The sole purpose of pooling layers in data processing is to reduce the spatial size of the data.

[0153] Neural network layer 430:

[0154] After processing by the convolutional / pooling layers 420, the convolutional neural network 400 is still insufficient to output the required information. As mentioned earlier, the convolutional / pooling layers 420 only extract features and reduce the parameters introduced by the input data. However, to generate the final output information (the required class information or other relevant information), the convolutional neural network 400 needs to utilize neural network layers 430 to generate one or more outputs representing the required number of classes. Therefore, neural network layers 430 can include multiple hidden layers (such as...). Figure 4 As shown in 431, 432 to 43n) and output layer 440, the parameters contained in these multi-layer hidden layers can be pre-trained based on relevant training data for specific task types.

[0155] After the multiple hidden layers in neural network layer 430, the final layer of the entire convolutional neural network 400 is the output layer 440. This output layer 440 has a loss function similar to classification cross-entropy, specifically used to calculate the prediction error. Once the entire convolutional neural network 400 has undergone forward propagation (such as...), the loss function is applied. Figure 4 Propagation from 410 to 440 degrees is considered forward propagation, while reverse propagation (such as...) is completed. Figure 4 The propagation from 440 to 410 (backpropagation) will begin to update the weight values ​​and biases of the layers mentioned above, in order to reduce the loss of the convolutional neural network 400 and the error between the output of the convolutional neural network 400 through the output layer and the ideal result.

[0156] In this embodiment, the forward propagation formula is predict = W2 × relu(W1 × input). X+b1)+b2. Where, the value of `predict` is the training mapping parameter, and `input` is... X This is the distance parameter from the projector to the projection plane.

[0157] It should be noted that, as Figure 4 The convolutional neural network 400 shown is only an example of a convolutional neural network. In specific applications, convolutional neural networks can also exist in the form of other network models.

[0158] The following describes a chip hardware structure provided by an embodiment of this application.

[0159] Figure 5 This is a schematic diagram of a chip hardware structure 500 provided in an embodiment of this application. The chip includes a neural network processor 50. The chip can be configured as follows: Figure 3 The execution device 310 shown is used to perform the calculations of the calculation module 311. This chip can also be placed in, for example... Figure 3 The training device 320 shown is used to complete the training work of the training device 320 and output the target model / rule 301. For example... Figure 4 The algorithms for each layer in the convolutional neural network shown can all be implemented in, for example... Figure 5 This is achieved in the chip shown.

[0160] The Neural Processing Unit (NPU) 50 is mounted as a coprocessor on the host CPU, which allocates tasks to it. The core of the NPU is the arithmetic circuit 503, which is controlled by the controller 504 to retrieve data from the memory (weight memory or input memory) and perform calculations.

[0161] In some implementations, the arithmetic circuit 503 internally includes multiple process engines (PEs). In some implementations, the arithmetic circuit 503 is a two-dimensional pulsating array. The arithmetic circuit 503 can also be a one-dimensional pulsating array or other electronic circuits capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 503 is a general-purpose matrix processor.

[0162] For example, suppose we have an input matrix A, a weight matrix B, and an output matrix C. The arithmetic circuit retrieves the corresponding data of matrix B from the weight memory 502 and caches it in each PE of the arithmetic circuit. The arithmetic circuit retrieves the data of matrix A from the input memory 501 and performs matrix operations with matrix B. The partial result or the final result of the obtained matrix is ​​stored in the accumulator 508.

[0163] The vector computation unit 507 can further process the output of the arithmetic circuit, such as vector multiplication, vector addition, exponentiation, logarithmic operations, size comparisons, etc. For example, the vector computation unit 507 can be used for network computation in non-convolutional / non-FC layers of neural networks, such as pooling, batch normalization, local response normalization, etc.

[0164] In some implementations, the vector computation unit 507 can store the processed output vector into a unified memory 506. For example, the vector computation unit 507 can apply a nonlinear function to the output of the arithmetic circuit 503, such as a vector of accumulated values, to generate activation values. In some implementations, the vector computation unit 507 generates normalized values, merged values, or both. In some implementations, the processed output vector can be used as an activation input to the arithmetic circuit 503, for example, for use in subsequent layers of a neural network.

[0165] The unified memory 506 is used to store input data and output data.

[0166] The weight data is directly transferred from the external memory to the input memory 501 and / or the unified memory 506 through the direct memory access controller 505 (DMAC), the weight data in the external memory is stored in the weight memory 502, and the data in the unified memory 506 is stored in the external memory.

[0167] The bus interface unit (BIU) 510 is used to enable interaction between the main CPU, DMAC and instruction fetch memory 509 via a bus.

[0168] The instruction fetch buffer 509, which is connected to the controller 504, is used to store the instructions used by the controller 504.

[0169] The controller 504 is used to call the instructions cached in the instruction memory 509 to control the operation of the computing accelerator.

[0170] Generally, the unified memory 506, input memory 501, weight memory 502, and instruction fetch memory 509 are all on-chip memories, while the external memory is memory outside the NPU. This external memory can be double data rate synchronous dynamic random access memory (DDR SDRAM), high bandwidth memory (HBM), or other readable and writable memory.

[0171] in, Figure 4 The operations of each layer in the convolutional neural network shown can be performed by the operation circuit 503 or the vector calculation unit 507.

[0172] Currently, in scenarios where projection devices project onto one or more planes, there are many issues that make it difficult to guarantee the quality of the projected image. For example, consider vehicle projection. Figure 6 A schematic diagram of the headlight projection is shown, such as Figure 6 As shown, the headlight projection and the lighting controller are separate, resulting in images from different directions. This projection method cannot fuse the images of at least two headlight projections through algorithms or other means. Figure 7 Another schematic diagram of the headlight projection is shown, such as Figure 7 As shown, the two headlights of the vehicle can merge the projected image at a certain position or on a certain plane to form a merged image. However, when there are at least two projection planes, this method cannot guarantee accurate projection of the image for different projection planes, which may lead to distortion of the projected image. Furthermore, when the distance between the vehicle or projection device and the projection plane changes, this method requires calculating the mapping relationship between the projected image and the image to be projected for each projection, which lacks flexibility.

[0173] In view of this, this application provides an image generation method that can improve the above-mentioned problems.

[0174] Figure 8 This is a schematic diagram of an image generation method 800 provided in an embodiment of this application. Figure 8 As shown, the method 800 includes the following steps:

[0175] S810, obtain the first parameter set.

[0176] Specifically, the first parameter set includes a first original image for displaying on the first plane, a second original image for displaying on the second plane, the distance from the first projector to the first plane, the distance from the first projector to the second plane, the desired position of the first desired image displayed on the first plane, and the desired position of the second desired image displayed on the second plane.

[0177] Among them, the first original image and the second original image are the original images projected by the first projector. The original images are the input images that are intended to be projected onto the first plane and the second plane. The first original image and the second original image constitute part or all of the images intended to be projected onto the first plane and the second plane. Obtaining the first original image and the second original image means obtaining parameters such as image resolution, image size, and image color of the first original image and the second original image. The first expected image and the second expected image are the expected images projected by the first projector. The expected images are the images obtained after mapping the original images. The expected positions of the first expected image and the second expected image can be represented by the expected coordinates of the first expected image and the expected coordinates of the second expected image. Optionally, parameters such as the brightness of the first expected image and the second expected image can also be obtained. The first plane and the second plane are connected, which means that the first plane and the second plane are seamlessly connected, and the angle formed is not equal to 0°, 180°, or 360°.

[0178] S820, determine the first mapping relationship and the second mapping relationship.

[0179] Wherein, the first mapping relationship is the mapping relationship between the first desired image and the first original image, the second mapping relationship is the mapping relationship between the second desired image and the second original image, the first mapping relationship corresponds to the distance from the first projector to the first plane, and the second mapping relationship corresponds to the distance from the first projector to the second plane.

[0180] To determine the first mapping relationship and the second mapping relationship, the method 800 further includes: inputting the first original image and the distance from the first projector to the first plane into the first neural network model, and inputting the second original image and the distance from the first projector to the second plane into the first neural network model, thereby obtaining the first mapping relationship and the second mapping relationship output by the first neural network model respectively.

[0181] Optionally, the first neural network model can be trained using a second set of parameters, which includes:

[0182] At least one distance from a first projector to a first reference plane, at least one first training mapping relationship, at least one distance from a first projector to a second reference plane, and at least one second training mapping relationship. Wherein, the first training mapping relationship is the mapping relationship between the first training expected image and the first training original image, and the second training mapping relationship is the mapping relationship between the second training expected image and the second training original image. The first training mapping relationship corresponds to the distance from the first projector to the first reference plane, and the second training mapping relationship corresponds to the distance from the first projector to the second reference plane. The first training original image and the second training original image are training original images projected by the first projector. They are used to calculate the training mapping relationship and are the original images that are expected to be projected onto the reference plane. The training original image may include parameters such as image resolution, image size, image color, and image content. The first training expected image is displayed on the first reference plane, and the second training expected image is displayed on the second reference plane. The training expected image is the input expected image used to calculate the training mapping relationship. The expected position of the training expected image includes parameters such as the coordinate position and brightness of the training expected image. The first reference plane and the second reference plane are connected, and the angle formed does not include 0°, 180°, and 360°.

[0183] It should be understood that the first plane and the first reference plane can be the same plane or different planes; the second plane and the second reference plane can also be the same plane or different planes. When the first plane and the first reference plane are the same plane, and the second plane and the second reference plane are the same plane, the distance from the first projector to the first plane can include the distance from the first projector to the first reference plane, and the distance from the first projector to the second plane can include the distance from the first projector to the second reference plane. The number of distances from the first projector to the first reference plane is equal to or less than the number of distances from the first projector to the first plane, and the number of distances from the first projector to the second reference plane is equal to or less than the number of distances from the first projector to the second plane.

[0184] Specifically, for a description of the first neural network model and the description of training the first neural network model, please refer to the above. Figures 3 to 5 For the sake of brevity, the relevant descriptions of neural network models and training neural network models in the previous application will not be repeated here.

[0185] Optionally, in order to obtain the first training mapping relationship and the second training mapping relationship, the method 800 further includes: determining the first training mapping relationship based on the parameters of the first training projection image displayed on the first reference plane, the first training original image, and the parameters of the first training expected image; and determining the second training mapping relationship based on the parameters of the second training projection image displayed on the second reference plane, the second training original image, and the parameters of the second training expected image.

[0186] Wherein, the first training projection image is the image of the first training original image projected onto the first reference plane by the first projector, and the second training projection image is the image of the second training original image projected onto the second reference plane by the first projector. Further, the first training projection image and the second training projection image are images obtained after calibration. The parameters of the training expectation image can be the expected coordinates of the training expectation image, and the parameters of the training projection image include the position parameters of the training projection image.

[0187] Specifically, taking a 2x2 dot matrix image as an example, the coordinate position of the calibration point in the first training image is set to Pos. srcl1 The coordinates of the four endpoints of the image are (640, 360), (1280, 360), (640, 720), and (1280, 720), respectively. The first training original image is projected onto the first reference plane through the first projector to obtain the first training projected image, and the coordinates of the obtained calibration points are set as Pos. map1 The coordinates of the four endpoints of the image are (480, 180), (960, 180), (480, 450), and (960, 450), respectively; the coordinates of the first training desired image are set to Pos. mapdst1 The coordinates of the four endpoints of the image are (720, 90), (1200, 90), (720, 360), and (1200, 360). The first training mapping relationship is obtained using the following method:

[0188] 1. Establish the mapping relationship between the first training original image and the first training projected image.

[0189] Let M be the mapping coefficient between the first training projected image and the first training original image. dstI→srcI1 According to Pos map1 ·M map1→srcI1 =Pos srcl1 , Pos map1 coordinates and Pos srcl1 Substituting into the above equation and solving using the least squares method, we obtain M. map1→srcI1 =[[1.33,0,0],[0,1.33,120],[0,0,1]].

[0190] 2. Calculate the coordinates of the first training desired image mapped to the first training original image.

[0191] Let Pos be the coordinates of the first training desired image mapped to the first training original image. dst1 According to Pos dst1 =Pos mapdst1 ·M map→srcI1 , Pos mapdst1 The value of M map→srcI1 Substituting the values ​​into the above equation and solving using the least squares method, we obtain Pos. dst1 =[[960,240],[1600,240],[960,600],[1600,600]].

[0192] 3. Calculate the mapping relationship between the first training original image and the first training expected image.

[0193] Let M be the mapping coefficient between the first training original image and the first training desired image. scr→dst According to Pos dst1 =Pos srcl1 ·M scr1→dst1 , Pos dst1 With Pos srcl1 Substituting the values ​​of into the above formula and solving using the least squares method, we obtain M. scr1→dst1 = [[1,0,320],[0,1,-120],[0,0,1]]. Thus, the first training mapping relationship is determined.

[0194] It should be understood that the calculation method for the second training mapping relationship is the same as that for the first training mapping relationship. Only the first original training image, the first projected training image, and the first expected training image need to be replaced with the second original training image, the first projected training image, and the second expected training image. For the sake of brevity, this application will not elaborate further.

[0195] It should also be understood that the training projection image can be an image projected by a projector, a light projected by a projector, or another carrier projected by a projector that can obtain coordinate positions; this application does not limit this.

[0196] S830, determine the first desired image based on the first original image and the first mapping relationship, and determine the second desired image based on the second original image and the second mapping relationship.

[0197] Specifically, let the coordinate position of the first desired image be Pos. imagedst1 According to the formula Pos mapdst1 ·M′ src→dst1 =Pos imagedst1Obtain the coordinates of the first desired image mapped from the first original image, where Pos mapdst1 Let M′ be the coordinates of the first original image. src2→dst1 This represents the first mapping relationship output by the first neural network model. Similarly, let the coordinate position of the first desired image be Pos. imagedst2 According to the formula Pos mapdst2 ·M′ src2→dst2 =Pos imagedst2 Obtain the coordinates of the second desired image mapped from the second original image, where Pos mapdst2 M′ represents the coordinates of the second original image. src2→dst2 This is the second mapping relationship output by the first neural network model.

[0198] The first desired image and the second desired image may include parameters such as image resolution, image size, image color, and image content.

[0199] S840, the first target image is obtained based on the first expected image, the expected position of the first expected image, the second expected image, and the expected position of the second expected image.

[0200] The first target image includes a first desired image and a second desired image, and the first target image is displayed on a first plane and a second plane by a first projector.

[0201] Specifically, the first desired image and the second desired image are fused to obtain the first target image. For example, the desired positions of the first and second desired images are represented by coordinates. The region formed by the desired positions of the first and second desired images is the effective region. Let the horizontal coordinate range of the effective region of the first and second desired images be [-x, x] and the vertical coordinate range be [-y, y], then the mapping is valid. The first desired image and the second desired image are then fused into a single image, namely, the first target image. The first target image is the image projected onto the first and second planes by the first projector.

[0202] Optionally, when the first original image and the second original image constitute a portion of the image to be projected onto the first plane and the second plane, the method 800 further includes:

[0203] Obtain a third set of parameters, which includes: a third original image for display on the first plane, a fourth original image for display on the second plane, the distance from the second projector to the first plane, the distance from the second projector to the second plane, the desired position of the third desired image for display on the first plane, and the desired position of the fourth desired image for display on the second plane.

[0204] In this application, the third and fourth original images are original images projected by the second projector, and the third and fourth expected images are expected images projected by the second projector. The original images projected by the first and second projectors constitute one image. The descriptions of the expected positions of the third and fourth expected images can be found in the descriptions of the expected positions of the first and second expected images described above. Similarly, the descriptions of the third and fourth original images can be found in the descriptions of the first and second original images described above. For brevity, these descriptions will not be repeated here.

[0205] Next, the third mapping relationship and the fourth mapping relationship are determined. The third mapping relationship is the mapping relationship between the third expected image and the third original image. The fourth mapping relationship is the mapping relationship between the fourth expected image and the fourth original image. The third mapping relationship corresponds to the distance from the second projector to the first plane, and the fourth mapping relationship corresponds to the distance from the second projector to the second plane.

[0206] Next, the third desired image is determined based on the third original image and the third mapping relationship, and the fourth desired image is determined based on the fourth original image and the fourth mapping relationship.

[0207] Finally, a second target image is obtained based on the third expected image, the expected position of the third expected image, the fourth expected image, and the expected position of the fourth expected image. The second target image is used to display on the first plane and the second plane by the second projector.

[0208] The second projector and the first projector are devices with the same projection function. The two projectors can project part or all of the content of an image respectively, and the two projectors are located on the same plane. The first target image and the second target image constitute part of the projected target image. That is, the first target image and the second target image respectively include part of the information of the projected target image.

[0209] It should be understood that the projected target image at this time can be a seamlessly merged image or two discontinuous images; this application does not limit this.

[0210] Furthermore, regarding the determination of the third and fourth mapping relationships, the third and fourth mapping relationships output by the second neural network model can be obtained by inputting the distance between the third original image and the second projector to the first plane into the second neural network model, and by inputting the distance between the fourth original image and the second projector to the second plane into the second neural network model.

[0211] It should be understood that the second neural network model and the first neural network model can be different neural network models or the same neural network model, and this application does not limit this.

[0212] The second neural network model is trained based on the fourth parameter set, which includes:

[0213] The system includes at least one distance from a second projector to a third reference plane, at least one third training mapping relationship, at least one distance from a second projector to a fourth reference plane, and at least one fourth training mapping relationship. The third training mapping relationship is a mapping between a third training desired image and a third training original image. The fourth training mapping relationship is a mapping between a fourth training desired image and a fourth training original image. The third and fourth training original images are training original images projected by the second projector. The third training desired image is displayed on the third reference plane, and the fourth training desired image is displayed on the fourth reference plane. The third and fourth reference planes are connected, and the included angle does not include 0°, 180°, or 360°. The third training mapping relationship corresponds to the distance from the second projector to the third reference plane, and the fourth training mapping relationship corresponds to the distance from the second projector to the second reference plane.

[0214] It should be understood that the descriptions of the third training expected image, the fourth training expected image, the third training original image, and the fourth training original image can be referred to the above descriptions of the first training expected image, the second training expected image, the first training original image, and the second training original image, respectively. For the sake of brevity, this application will not repeat them here.

[0215] It should also be understood that the description of the second neural network model and its training can be found in [reference needed]. Figures 3 to 5 For the sake of brevity, the relevant descriptions of neural network models and training neural network models in the previous application will not be repeated here.

[0216] Optionally, in order to obtain the third training mapping relationship and the fourth training mapping relationship, the method 800 further includes: determining the third training mapping relationship based on the parameters of the third training projection image displayed on the third reference plane, the third training original image, and the parameters of the third training expected image; and determining the fourth training mapping relationship based on the parameters of the fourth training projection image displayed on the fourth reference plane, the fourth training original image, and the parameters of the fourth training expected image, wherein the third training projection image is the image of the third training original image projected onto the first reference plane by the second projector, and the fourth training projection image is the image of the fourth training original image projected onto the second reference plane by the second projector.

[0217] The description of obtaining the third and fourth training mapping relationships can be found in the description of obtaining the first training mapping relationship described above. For the sake of brevity, this application will not repeat it here.

[0218] It should be understood that the first training mapping relationship and the third training mapping relationship can be the same mapping relationship or different mapping relationships; the third training mapping relationship and the fourth training mapping relationship can be the same mapping relationship or different mapping relationships.

[0219] Furthermore, regarding the description of obtaining the second target image based on the third and fourth expected images, please refer to the relevant description of obtaining the first target image based on the first and second expected images mentioned above. For the sake of brevity, this application will not repeat it here.

[0220] Optionally, in order to obtain a projected image with a better user experience, the method 800 further includes: obtaining a projected target image based on the first target image and the second target image.

[0221] The projected target image includes a first target image and a second target image, which are aligned on a horizontal plane. Furthermore, the first and second projectors can employ three projection methods to obtain different projected target images:

[0222] 1. If the goal is to maximize the display area of ​​the projected target image, the area of ​​the projected target image can be made equal to the sum of the areas of the first target image and the second target image. In this case, the first target image and the second target image have no overlapping parts.

[0223] 2. If the optimal display quality of the projected target image is desired, the area of ​​the projected target image can be made equal to the area of ​​the first target image or the second target image. In this case, the first target image and the second target image completely overlap, wherein the area of ​​the first target image is equal to the area of ​​the second target image.

[0224] 3. If the goal is to achieve both high display quality and a large display area for the projected target image, the area of ​​the projected target image can be made smaller than the sum of the first target image and the second target image, but larger than the area of ​​either the first target image or the second target image. In this case, the first target image and the second target image will partially overlap.

[0225] It should be understood that the embodiments of this application take two different planes as examples for projection. If there are three or more intersecting planes, the image generation method provided in the embodiments of this application can still be used. That is, different mapping relationships between the desired image and the original image are obtained for different projection planes, and the image is generated through the mapping relationship of the corresponding projection planes, forming the target image on different projection planes. For specific image generation and projection methods, please refer to the methods provided in the embodiments of this application.

[0226] Based on the above scheme, when only a first projector exists, the mapping relationship between the first desired image and the first original image corresponding to the distance from the first projector to the first plane, and the mapping relationship between the second desired image and the second original image corresponding to the distance from the first projector to the second plane are determined respectively. Then, based on the above mapping relationship, the first original image, and the second original image, the first desired image and the second desired image are determined respectively. Finally, based on the first desired image, the desired position of the first desired image, the second desired image, and the desired position of the second desired image, a continuous first target image that can be presented on two different planes is obtained. Furthermore, since different mapping relationships are used for different planes, the first target image will not suffer from image distortion or other problems, thus improving the display quality of the projected image.

[0227] Furthermore, if the original image from the first projector is a portion of an image, or if a larger and higher-quality target image needs to be displayed on at least two planes, a second projector can be used. Employing the same method as the first projector, different mapping relationships are applied to different planes, ultimately displaying the second target image on both the first and second planes. Finally, the first and second target images are fused to obtain the final projected target image. This projected target image, compared to the first and second target images, has the advantages of a larger display area and / or better display quality.

[0228] Finally, by training a neural network model, the model can predict or obtain the mapping relationship for different distances and different planes, eliminating the need for manual calibration for each projection at different distances, simplifying the image projection process and improving the flexibility of image projection.

[0229] The above, combined with Figure 8 The methods provided in the embodiments of this application are described in detail below. Figures 9 to 11 This application provides a detailed description of the image generation apparatus provided in the embodiments. It should be understood that the descriptions of the apparatus embodiments correspond to the descriptions of the method embodiments; therefore, any content not described in detail can be found in the above method embodiments, and for the sake of brevity, will not be repeated here.

[0230] This application embodiment can divide functional units according to the above method example. For example, functional units can be divided according to each function, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional module. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods. The following description uses the division of functional units according to each function as an example.

[0231] Figure 9 This is a schematic diagram of an electronic device 900 provided in an embodiment of this application.

[0232] The electronic device 900 includes a storage module 910 and a processing module 920.

[0233] Storage module 910 is used to store program instructions.

[0234] When program instructions are executed in processing module 920, processing module 920 is used to execute... Figure 8 The method shown.

[0235] When processing module 920 executes Figure 8 When the method is shown, the electronic device 900 can also be called an image generation device.

[0236] Specifically, the processing module 920 is used to: obtain a first parameter set, the first parameter set including: a first original image for display on a first plane, a second original image for display on a second plane, the distance from the first projector to the first plane, the distance from the first projector to the second plane, the desired position of a first desired image displayed on the first plane, and the desired position of a second desired image displayed on the second plane, wherein the first original image and the second original image belong to the original images projected by the first projector, the first desired image and the second desired image belong to the desired images projected by the first projector, and the first plane and the second plane are connected; determine a first mapping relationship and a second mapping relationship, the first mapping relationship being the distance from the first projector to the first plane. The first expected image and the first original image are mapped to the distance between the planes. The second mapping relationship is the mapping relationship between the second expected image and the second original image, which is the distance between the first projector and the second plane. The first mapping relationship corresponds to the distance between the first projector and the first plane, and the second mapping relationship corresponds to the distance between the first projector and the second plane. The first expected image is determined based on the first original image and the first mapping relationship, and the second expected image is determined based on the second original image and the second mapping relationship. The first target image is obtained based on the first expected image, the expected position of the first expected image, the second expected image, and the expected position of the second expected image. The first target image is displayed on the first plane and the second plane by the first projector.

[0237] Optionally, the processing module 920 is specifically used to input the first original image and the distance from the first projector to the first plane into the first neural network model to obtain a first mapping relationship, and to input the second original image and the distance from the first projector to the second plane into the first neural network model to obtain a second mapping relationship.

[0238] Optionally, the first neural network model is trained based on a second set of parameters, which includes: the distance from the first projector to the first reference plane, a first training mapping relationship, the distance from the first projector to the second reference plane, and a second training mapping relationship. The first training mapping relationship is the mapping relationship between the first training expected image and the first training original image corresponding to the distance from the first projector to the first reference plane, and the second training mapping relationship is the mapping relationship between the second training expected image and the second training original image corresponding to the distance from the first projector to the second reference plane. The first training original image and the second training original image are training original images projected by the first projector. The first training expected image is displayed on the first reference plane, and the second training expected image is displayed on the second reference plane. The first reference plane and the second reference plane are connected.

[0239] Optionally, the processing module 920 is further configured to determine a first training mapping relationship based on the parameters of the first training projection image displayed on the first reference plane, the first training original image, and the parameters of the first training desired image; and to determine a second training mapping relationship based on the parameters of the second training projection image displayed on the second reference plane, the second training original image, and the parameters of the second training desired image. The first training projection image is the image of the first training original image projected onto the first reference plane by the first projector, and the second training projection image is the image of the second training original image projected onto the second reference plane by the first projector. The first training mapping relationship corresponds to the distance from the first projector to the first reference plane, and the second training mapping relationship corresponds to the distance from the first projector to the second reference plane.

[0240] Optionally, the processing module 920 is further configured to, when the first original image and the second original image constitute part of an image, obtain a third parameter set, the third parameter set including: a third original image for display on the first plane, a fourth original image for display on the second plane, the distance from the second projector to the first plane, the distance from the second projector to the second plane, the desired position of the third desired image displayed on the first plane, and the desired position of the fourth desired image displayed on the second plane, wherein the third original image and the fourth original image belong to the original images projected by the second projector, the third desired image and the fourth desired image belong to the desired images projected by the second projector, and the original images projected by the first projector and the original images projected by the second projector constitute an image. The image is used to determine a third mapping relationship and a fourth mapping relationship. The third mapping relationship is the mapping relationship between the third expected image and the third original image corresponding to the distance from the second projector to the first plane. The fourth mapping relationship is the mapping relationship between the fourth expected image and the fourth original image corresponding to the distance from the second projector to the second plane. The third expected image is determined based on the third original image and the third mapping relationship. The fourth expected image is determined based on the fourth original image and the fourth mapping relationship. The second target image is obtained based on the third expected image, the expected position of the third expected image, the fourth expected image, and the expected position of the fourth expected image. The second target image is displayed on the first plane and the second plane through the second projector. The first target image and the second target image respectively constitute a part of the projected target image.

[0241] Optionally, the processing module 920 is further specifically configured to input the third original image and the distance from the second projector to the first plane into the second neural network model to obtain a third mapping relationship, and input the fourth original image and the distance from the second projector to the second plane into the second neural network model to obtain a fourth mapping relationship.

[0242] Optionally, the second neural network model is trained based on a fourth parameter set, which includes: the distance from the second projector to the third reference plane, a third training mapping relationship, the distance from the second projector to the fourth reference plane, and a fourth training mapping relationship. The third training mapping relationship is the mapping relationship between the third training expected image and the third training original image corresponding to the distance from the second projector to the third reference plane. The fourth training mapping relationship is the mapping relationship between the fourth training expected image and the fourth training original image corresponding to the distance from the second projector to the fourth reference plane. The third training original image and the fourth training original image are training original images projected by the second projector. The third training expected image is displayed on the third reference plane, and the fourth training expected image is displayed on the fourth reference plane. The third reference plane and the fourth reference plane are connected.

[0243] Optionally, the processing module 920 is further configured to determine a third training mapping relationship based on the parameters of the third training projection image displayed on the third reference plane, the third training original image, and the parameters of the third training expected image; and to determine a fourth training mapping relationship based on the parameters of the fourth training projection image displayed on the fourth reference plane, the fourth training original image, and the parameters of the fourth training expected image. The third training projection image is the image of the third training original image projected onto the first reference plane via the second projector, and the fourth training projection image is the image of the fourth training original image projected onto the second reference plane via the second projector. The third training mapping relationship corresponds to the distance from the second projector to the third reference plane, and the fourth training mapping relationship corresponds to the distance from the second projector to the second reference plane.

[0244] Optionally, the processing module 920 is further configured to obtain a projected target image based on the first target image and the second target image, wherein the first target image and the second target image are aligned in the horizontal plane.

[0245] Optionally, if the area of ​​the projected target image is equal to the sum of the areas of the first target image and the second target image, the first target image and the second target image have no overlapping portion; or if the area of ​​the projected target image is equal to the area of ​​either the first target image or the second target image, the first target image and the second target image completely overlap, and the areas of the first target image and the second target image are equal; or if the area of ​​the projected target image is less than the sum of the areas of the first target image and the second target image, but greater than the area of ​​either the first target image or the second target image, the first target image and the second target image partially overlap.

[0246] Figure 10 This is a schematic diagram of the hardware structure of the image generation apparatus 1000 according to an embodiment of this application. Figure 10 The image generation apparatus 1000 shown includes a memory 1001, a processor 1002, a communication interface 1003, and a bus 1004. The memory 1001, processor 1002, and communication interface 1003 are interconnected via the bus 1004.

[0247] The memory 1001 may be a ROM, a static storage device, or RAM. The memory 1001 may store a program, and when the program stored in the memory 1001 is executed by the processor 1002, the processor 1002 and the communication interface 1003 are used to execute the various steps of the image generation method of the embodiments of this application.

[0248] The processor 1002 may be a general-purpose CPU, microprocessor, ASIC, GPU, or one or more integrated circuits, used to execute relevant programs to implement the functions required by the units in the image generation apparatus of this application embodiment, or to execute the image generation method of this application method embodiment.

[0249] The processor 1002 can also be an integrated circuit chip with signal processing capabilities; for example, it could be... Figure 5 The chip shown. In implementation, each step of the image generation method of this application embodiment can be completed by the integrated logic circuit of the hardware in the processor 1002 or by instructions in the form of software.

[0250] The processor 1002 described above can also be a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 1001. The processor 1002 reads the information in memory 1001 and, in conjunction with its hardware, completes the functions required by the units included in the image generation apparatus of the embodiments of this application, or executes the image generation method of the method embodiments of this application.

[0251] The communication interface 1003 uses a transceiver device, such as, but not limited to, a transceiver, to enable communication between the device 1000 and other devices or communication networks. For example, audio can be acquired through the communication interface 1003.

[0252] Bus 1004 may include a pathway for transmitting information between various components of device 1000 (e.g., memory 1001, processor 1002, communication interface 1003).

[0253] Figure 11 This is a schematic diagram of the hardware structure of the neural network training device 1100 according to an embodiment of this application. Similar to the devices 900 and 1000 described above, Figure 11 The neural network training device 1100 shown includes a memory 1101, a processor 1102, a communication interface 1103, and a bus 1104. The memory 1101, processor 1102, and communication interface 1103 are interconnected via the bus 1104.

[0254] Can be achieved through Figure 11 The neural network training device 1100 shown trains the neural network, and the trained neural network can be used to execute the image generation method of the embodiments of this application.

[0255] Specifically, Figure 11 The device shown can acquire training data and the neural network to be trained from the outside through the communication interface 1103, and then the processor trains the neural network to be trained according to the training data.

[0256] It should be noted that although only a memory, processor, and communication interface are shown in the above-described devices 1000 and 1100, those skilled in the art should understand that in specific implementations, devices 1000 and 1100 may also include other devices necessary for normal operation. Furthermore, depending on specific needs, those skilled in the art should understand that devices 1000 and 1100 may also include hardware devices for implementing other additional functions. In addition, those skilled in the art should understand that devices 1000 and 1100 may only include the devices necessary for implementing the embodiments of this application, and may not necessarily include... Figure 10 and Figure 11 All the devices shown.

[0257] It should be understood that the processor in the embodiments of this application can be a central processing unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0258] It should also be understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0259] The descriptions of the processes corresponding to the above-mentioned figures each have their own emphasis. For parts of a process that are not described in detail, please refer to the relevant descriptions of other processes.

[0260] This application also provides a computer-readable storage medium having program instructions that, when executed directly or indirectly, enable the method described above to be implemented.

[0261] In this embodiment of the application, a computer program product containing instructions is also provided, which, when run on a computing device, causes the computing device to perform the methods described above, or causes the computing device to perform the functions of the apparatus described above.

[0262] This application also provides a chip system including at least one processor, wherein when program instructions are executed in the at least one processor, the method described above is implemented.

[0263] This application also provides an electronic device, including a display device and the device described above.

[0264] Alternatively, as one implementation, the electronic device may include, but is not limited to, a projector, a television, a vehicle headlight, a vehicle, or any device with projection capabilities.

[0265] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0266] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0267] In this application, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0268] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0269] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0270] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0271] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0272] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0273] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0274] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0275] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for image generation, characterized in that, include: Obtain a first parameter set, which includes: a first original image for display on a first plane, a second original image for display on a second plane, the distance from the first projector to the first plane, the distance from the first projector to the second plane, the desired position of the first desired image displayed on the first plane, and the desired position of the second desired image displayed on the second plane, wherein... The first original image and the second original image belong to the original images projected by the first projector, the first desired image and the second desired image belong to the desired images projected by the first projector, and the first plane and the second plane are connected. The first original image and the distance from the first projector to the first plane are input into the first neural network model to obtain the first mapping relationship. The first mapping relationship corresponds to the distance from the first projector to the first plane. The first mapping relationship is the mapping relationship between the first expected image and the first original image. The second original image and the distance from the first projector to the second plane are input into the first neural network model to obtain a second mapping relationship. The second mapping relationship corresponds to the distance from the first projector to the second plane. The second mapping relationship is the mapping relationship between the second expected image and the second original image. The first desired image is determined based on the first original image and the first mapping relationship, and the second desired image is determined based on the second original image and the second mapping relationship; A first target image is obtained based on the first desired image, the desired position of the first desired image, the second desired image, and the desired position of the second desired image. The first target image is displayed on the first plane and the second plane by the first projector.

2. The method according to claim 1, characterized in that, The first neural network model is trained based on a second set of parameters, which includes: The distance from the first projector to the first reference plane, the first training mapping relationship, the distance from the first projector to the second reference plane, and the second training mapping relationship. Wherein, the first training mapping relationship is the mapping relationship between the first training expected image and the first training original image, the second training mapping relationship is the mapping relationship between the second training expected image and the second training original image, the first training original image and the second training original image belong to the training original images projected by the first projector, the first training expected image is displayed on the first reference plane, the second training expected image is displayed on the second reference plane, and the first reference plane and the second reference plane are connected.

3. The method according to claim 2, characterized in that, The method further includes: The first training mapping relationship is determined based on the parameters of the first training projection image displayed on the first reference plane, the first training original image, and the parameters of the first training expected image. The second training mapping relationship is determined based on the parameters of the second training projection image displayed on the second reference plane, the second training original image, and the parameters of the second training expected image. The first training projection image is the image of the first training original image projected onto the first reference plane by the first projector, and the second training projection image is the image of the second training original image projected onto the second reference plane by the first projector. The first training mapping relationship corresponds to the distance from the first projector to the first reference plane, and the second training mapping relationship corresponds to the distance from the first projector to the second reference plane.

4. The method according to claim 2 or 3, characterized in that, When the first original image and the second original image constitute a portion of a single image, the method further includes: Obtain a third parameter set, which includes: a third original image for displaying on the first plane, a fourth original image for displaying on the second plane, the distance from the second projector to the first plane, the distance from the second projector to the second plane, the desired position of the third desired image displayed on the first plane, and the desired position of the fourth desired image displayed on the second plane, wherein... The third original image and the fourth original image belong to the original images projected by the second projector, the third expected image and the fourth expected image belong to the expected images projected by the second projector, and the original image projected by the first projector and the original image projected by the second projector constitute one image. Determine a third mapping relationship and a fourth mapping relationship, wherein the third mapping relationship is the mapping relationship between the third desired image and the third original image, and the fourth mapping relationship is the mapping relationship between the fourth desired image and the fourth original image; The third desired image is determined based on the third original image and the third mapping relationship, and the fourth desired image is determined based on the fourth original image and the fourth mapping relationship; A second target image is obtained based on the third expected image, the expected position of the third expected image, the fourth expected image, and the expected position of the fourth expected image. The second target image is displayed on the first plane and the second plane by the second projector. The first target image and the second target image respectively constitute a part of the projected target image.

5. The method according to claim 4, characterized in that, Determining the third and fourth mapping relationships includes: The third original image and the distance from the second projector to the first plane are input into the second neural network model to obtain the third mapping relationship. The fourth original image and the distance from the second projector to the second plane are input into the second neural network model. The third mapping relationship corresponds to the distance from the second projector to the first plane, and the fourth mapping relationship corresponds to the distance from the second projector to the second plane.

6. The method according to claim 5, characterized in that, The second neural network model is trained based on a fourth parameter set, which includes: The distance from the second projector to the third reference plane, the third training mapping relationship, the distance from the second projector to the fourth reference plane, and the fourth training mapping relationship. Wherein, the third training mapping relationship is the mapping relationship between the third training expected image and the third training original image, the fourth training mapping relationship is the mapping relationship between the fourth training expected image and the fourth training original image, the third training original image and the fourth training original image belong to the training original images projected by the second projector, the third training expected image is displayed on the third reference plane, the fourth training expected image is displayed on the fourth reference plane, and the third reference plane is connected to the fourth reference plane.

7. The method according to claim 6, characterized in that, The method further includes: The third training mapping relationship is determined based on the parameters of the third training projection image displayed on the third reference plane, the parameters of the third training original image, and the parameters of the third training expected image. The fourth training mapping relationship is determined based on the parameters of the fourth training projection image displayed on the fourth reference plane, the parameters of the fourth training original image, and the parameters of the fourth training expected image. The third training projection image is the image of the third training original image projected onto the first reference plane by the second projector, and the fourth training projection image is the image of the fourth training original image projected onto the fourth reference plane by the second projector. The third training mapping relationship corresponds to the distance from the second projector to the third reference plane, and the fourth training mapping relationship corresponds to the distance from the second projector to the fourth reference plane.

8. The method according to any one of claims 4 to 7, characterized in that, The method further includes: The projected target image is obtained based on the first target image and the second target image, wherein the first target image and the second target image are aligned on the horizontal plane.

9. The method according to claim 8, characterized in that, When the area of ​​the projected target image is equal to the sum of the areas of the first target image and the second target image, the first target image and the second target image have no overlapping portion; or When the area of ​​the projected target image is equal to the area of ​​the first target image or the second target image, the first target image and the second target image completely overlap, and the area of ​​the first target image is equal to the area of ​​the second target image; or If the area of ​​the projected target image is less than the sum of the first target image and the second target image, but greater than the area of ​​either the first target image or the second target image, then the first target image and the second target image partially overlap.

10. An image generation apparatus, characterized in that, include: Acquisition unit: used to acquire a first original image for display on a first plane, a second original image for display on a second plane, a desired position of a first desired image displayed on the first plane, and a desired position of a second desired image displayed on the second plane, wherein the first original image and the second original image belong to the original image projected by the first projector, the first desired image and the second desired image belong to the desired image projected by the first projector, and the first plane and the second plane are connected. The acquisition unit is further configured to acquire the distance from the first projector to the first plane and the distance from the first projector to the second plane; Processing unit: Inputs the first original image and the distance from the first projector to the first plane into the first neural network model to obtain a first mapping relationship; inputs the second original image and the distance from the first projector to the second plane into the first neural network model to obtain a second mapping relationship; the first mapping relationship corresponds to the distance from the first projector to the first plane; the second mapping relationship corresponds to the distance from the first projector to the second plane; the first mapping relationship is the mapping relationship between the first desired image and the first original image; the second mapping relationship is the mapping relationship between the second desired image and the second original image. The processing unit is further configured to determine the first desired image based on the first original image and the first mapping relationship, and to determine the second desired image based on the second original image and the second mapping relationship; The processing unit is further configured to obtain a first target image based on the first desired image, the desired position of the first desired image, the second desired image, and the desired position of the second desired image, wherein the first target image is used to display on the first plane and the second plane by the first projector.

11. The apparatus according to claim 10, characterized in that, The first neural network model is trained based on a second set of parameters, which includes: The distance from the first projector to the first reference plane, the first training mapping relationship, the distance from the first projector to the second reference plane, and the second training mapping relationship. Wherein, the first training mapping relationship is the mapping relationship between the first training expected image and the first training original image, the second training mapping relationship is the mapping relationship between the second training expected image and the second training original image, the first training original image and the second training original image belong to the training original images projected by the first projector, the first training expected image is displayed on the first reference plane, the second training expected image is displayed on the second reference plane, and the first reference plane and the second reference plane are connected.

12. The apparatus according to claim 11, characterized in that, The processing unit is also used for: The first training mapping relationship is determined based on the parameters of the first training projection image displayed on the first reference plane, the first training original image, and the parameters of the first training expected image. The second training mapping relationship is determined based on the parameters of the second training projection image displayed on the second reference plane, the second training original image, and the parameters of the second training expected image. The first training projection image is the image of the first training original image projected onto the first reference plane by the first projector, and the second training projection image is the image of the second training original image projected onto the second reference plane by the first projector. The first training mapping relationship corresponds to the distance from the first projector to the first reference plane, and the second training mapping relationship corresponds to the distance from the first projector to the second reference plane.

13. The apparatus according to claim 11 or 12, characterized in that, In the case where the first original image and the second original image constitute a portion of a single image. The acquisition unit is further configured to acquire a third original image for display on the first plane, a fourth original image for display on the second plane, a desired position of a third desired image displayed on the first plane, and a desired position of a fourth desired image displayed on the second plane, wherein the third original image and the fourth original image belong to the original images projected by the second projector, the third desired image and the fourth desired image belong to the desired images projected by the second projector, and the original images projected by the first projector and the original images projected by the second projector constitute the image. The acquisition unit is further configured to acquire the distance from the second projector to the first plane, and the distance from the second projector to the second plane; The processing unit is further configured to determine a third mapping relationship and a fourth mapping relationship, wherein the third mapping relationship is the mapping relationship between the third desired image and the third original image, and the fourth mapping relationship is the mapping relationship between the fourth desired image and the fourth original image; The processing unit is further configured to determine the third desired image based on the third original image and the third mapping relationship, and to determine the fourth desired image based on the fourth original image and the fourth mapping relationship; The processing unit is further configured to obtain a second target image based on the third expected image, the expected position of the third expected image, the fourth expected image, and the expected position of the fourth expected image. The second target image is displayed on the first plane and the second plane by the second projector. The first target image and the second target image constitute part of the projected target image.

14. The apparatus according to claim 13, characterized in that, The processing unit is also specifically used for: The third original image and the distance from the second projector to the first plane are input into the second neural network model to obtain the third mapping relationship. The fourth original image and the distance from the second projector to the second plane are input into the second neural network model to obtain the fourth mapping relationship. The third mapping relationship corresponds to the distance from the second projector to the first plane, and the fourth mapping relationship corresponds to the distance from the second projector to the second plane. Obtain the third mapping relationship and the fourth mapping relationship output by the second neural network model respectively.

15. The apparatus according to claim 14, characterized in that, The second neural network model is trained based on a fourth parameter set, which includes: The distance from the second projector to the third reference plane, the third training mapping relationship, the distance from the second projector to the fourth reference plane, and the fourth training mapping relationship. Wherein, the third training mapping relationship is the mapping relationship between the third training expected image and the third training original image, the fourth training mapping relationship is the mapping relationship between the fourth training expected image and the fourth training original image, the third training original image and the fourth training original image belong to the training original images projected by the second projector, the third training expected image is displayed on the third reference plane, the fourth training expected image is displayed on the fourth reference plane, and the third reference plane is connected to the fourth reference plane.

16. The apparatus according to claim 15, characterized in that, The processing unit is also used for: The third training mapping relationship is determined based on the parameters of the third training projection image displayed on the third reference plane, the parameters of the third training original image, and the parameters of the third training expected image. The fourth training mapping relationship is determined based on the parameters of the fourth training projection image displayed on the fourth reference plane, the parameters of the fourth training original image, and the parameters of the fourth training expected image. The third training projection image is the image of the third training original image projected onto the first reference plane by the second projector, and the fourth training projection image is the image of the fourth training original image projected onto the fourth reference plane by the second projector. The third training mapping relationship corresponds to the distance from the second projector to the third reference plane, and the fourth training mapping relationship corresponds to the distance from the second projector to the fourth reference plane.

17. The apparatus according to any one of claims 13 to 16, characterized in that, The processing unit is also used for: The projected target image is obtained based on the first target image and the second target image, wherein the first target image and the second target image are aligned on the horizontal plane.

18. The apparatus according to claim 17, characterized in that, When the area of ​​the projected target image is equal to the sum of the areas of the first target image and the second target image, the first target image and the second target image have no overlapping portion; or When the area of ​​the projected target image is equal to the area of ​​the first target image or the second target image, the first target image and the second target image completely overlap, and the area of ​​the first target image is equal to the area of ​​the second target image; or If the area of ​​the projected target image is less than the sum of the first target image and the second target image, but greater than the area of ​​either the first target image or the second target image, then the first target image and the second target image partially overlap, and the area of ​​the first target image is equal to the area of ​​the second target image.

19. An image generation apparatus, characterized in that, include: A processor for executing a computer program stored in memory to cause the apparatus to perform the method as described in any one of claims 1 to 9.

20. The apparatus according to claim 19, characterized in that, The device also includes the memory.

21. An electronic device, characterized in that, Includes the image generation apparatus as described in any one of claims 10-18.

Images