Neural network-based 3D image generation method and system

Abstract

The invention relates to the technical field of naked eye 3D, and provides a 3D image generation method and system based on a neural network, and the method comprises the steps: dividing each object in a to-be-processed 2D image, calculating the depth-of-field distance of each object through the neural network, finding out a main object, and calculating the depth-of-field distance of each object based on the difference between binocular imaging and monocular imaging; according to the method, the position difference between the to-be-processed 2D image and each object in the corresponding 3D image is calculated, and the left eye image and the right eye image are adjusted and then spliced, so that the 2D image can be efficiently generated into the 3D image without being limited by an original image or a shooting condition, and the watching experience of a user is improved.

Description

Technical Field

[0001] The present invention relates to the field of naked-eye 3D technology, and in particular to a 3D image generation method and system based on a neural network. Background Art

[0002] As technology develops, users are becoming more and more fond of the immersive experience that 3D technology brings to them, and they hope that more images or videos can be produced with 3D effects; but due to the limitations of shooting equipment or shooting technology, not everyone can directly shoot 3D pictures, and many users' private image data can no longer be reproduced with 3D shooting equipment. Summary of the invention

[0003] The present invention provides a 3D image generation method based on a neural network, which is used to solve the problem of harsh conditions for obtaining 3D images in the prior art.

[0004] A first aspect of the present invention provides a 3D image generation method based on a neural network, comprising:

[0005] Acquire the 2D image to be processed and divide each object in the image, input the 2D image to be processed into the first neural network model to identify the depth of field distance corresponding to each object; input the 2D image to be processed into the second neural network model, generate the object relationship in the image scene and identify the main object in the image; establish a plane rectangular coordinate system with the center of the image of the 2D image to be processed as the origin, and obtain the horizontal coordinates of each object;

[0006] Obtain the user's 3D device parameters to obtain the user's screen distance; substitute the horizontal coordinates and depth of field distance of each non-screen main object into the object translation calculation model to obtain the object translation parameters. The object translation calculation model is specifically:

[0007]

[0008] Among them, L0 is the user screen distance, θ is the angle between the current monocular line of sight looking at the main object and the connecting line between the main object and the screen, d0 is the user pupil distance, x0 is the horizontal coordinate of the main object, L1 is the depth of field distance of the main object, x n is the horizontal coordinate of the nth non-image main object, L n is the depth of field distance of the nth non-image main object, Δd is the difference between the horizontal coordinate of the nth non-image main object and the horizontal coordinate of the main object, and D is the object translation parameter. When calculating the translation of the object from the left eye perspective, a=1, and when calculating the translation of the object from the right eye perspective, a=0;

[0009] Each object in the 2D image to be processed is adjusted by the left eye object translation parameter to obtain a left eye 3D image, and each object in the 2D image to be processed is adjusted by the right eye object translation parameter to obtain a right eye 3D image; based on the user's 3D device playback mode, the left eye 3D image and the right eye 3D image are spliced ​​to display the 3D image to the user.

[0010] Optionally, also include:

[0011] A second 2D image is acquired, and based on image features, the second 2D image is compared with the 2D image to be processed for similarity; if the similarity exceeds a preset first threshold, the second 2D image is divided to obtain a second image object; a main object in the 2D image to be processed is compared with the second image object for similarity, and the second image object whose similarity exceeds the preset second threshold is set as the main object.

[0012] Optionally, after the left-eye 3D image and the right-eye 3D image are spliced, the method further includes:

[0013] The left-eye 3D image and the right-eye 3D image are respectively input into the third neural network model to fill in the missing areas of the image after the object is translated.

[0014] The second aspect of the present application provides a 3D image generation system based on a neural network, comprising:

[0015] The object data acquisition module is used to acquire the 2D image to be processed and divide the objects in the image, input the 2D image to be processed into the first neural network model to identify the depth of field distance corresponding to each object; input the 2D image to be processed into the second neural network model to generate the object relationship in the image scene and identify the main object in the image; establish a plane rectangular coordinate system with the center of the image of the 2D image to be processed as the origin to obtain the horizontal coordinates of each object;

[0016] The translation parameter calculation module is used to obtain the user's 3D device parameters and obtain the user's screen distance; the horizontal coordinates and depth of field distance of each non-screen main object are respectively substituted into the object translation calculation model to obtain the object translation parameters. The object translation calculation model is specifically:

[0017]

[0018] Among them, L0 is the user screen distance, θ is the angle between the current monocular line of sight looking at the main object and the connecting line between the main object and the screen, d0 is the user pupil distance, x0 is the horizontal coordinate of the main object, L1 is the depth of field distance of the main object, x n is the horizontal coordinate of the nth non-image main object, L nis the depth of field distance of the nth non-image main object, Δd is the difference between the horizontal coordinate of the nth non-image main object and the horizontal coordinate of the main object, and D is the object translation parameter. When calculating the translation of the object from the left eye perspective, a=1, and when calculating the translation of the object from the right eye perspective, a=0;

[0019] The 3D image generation module is used to adjust each object in the 2D image to be processed with the left eye object translation parameter to obtain the left eye 3D image, and adjust each object in the 2D image to be processed with the right eye object translation parameter to obtain the right eye 3D image; based on the user's 3D device playback mode, the left eye 3D image and the right eye 3D image are spliced ​​to display the 3D image to the user.

[0020] Optionally, also include:

[0021] The continuous image processing module is used to obtain a second 2D image, compare the second 2D image with the 2D image to be processed based on image features, and if the similarity exceeds a preset first threshold, divide the second 2D image to obtain a second image object; compare the similarity between the main object in the 2D image to be processed and the second image object, and set the second image object whose similarity exceeds the preset second threshold as the main object.

[0022] Optionally, in the 3D image generation module, after splicing the left-eye 3D image and the right-eye 3D image, the module further includes:

[0023] The left-eye 3D image and the right-eye 3D image are respectively input into the third neural network model to fill in the missing areas of the image after the object is translated.

[0024] The third aspect of the present application provides a 3D image generation method device based on a neural network, the device comprising a processor and a memory:

[0025] The memory is used to store program code and transmit the program code to the processor;

[0026] The processor is used to execute the neural network-based 3D image generation method described in any one of the first aspects of the present invention according to the instructions in the program code.

[0027] A fourth aspect of the present application provides a computer-readable storage medium, characterized in that the computer-readable storage medium is used to store program code, and the program code is used to execute a neural network-based 3D image generation method as described in any one of the first aspects of the present invention.

[0028] It can be seen from the above technical solutions that the present invention has the following advantages: by dividing the objects in the 2D image to be processed, calculating the depth of field distance of each object with a neural network, and finding the main object, based on the difference between binocular imaging and monocular imaging, the position difference of each object in the 2D image to be processed and the corresponding 3D image is calculated, and the left and right eye images are adjusted and spliced ​​respectively, so that the 2D image can be efficiently generated into a 3D image without being restricted by the original image or shooting conditions, thereby improving the user's viewing experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0030] Figure 1 The first flowchart of a neural network-based 3D image generation method;

[0031] Figure 2 This is a schematic diagram of left eye 2D image translation parameter calculation;

[0032] Figure 3 This is a structural diagram of a 3D image generation system based on a neural network. DETAILED DESCRIPTION

[0033] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] The present invention provides a 3D image generation method based on a neural network, which is used to solve the problem of harsh conditions for obtaining 3D images in the prior art.

[0035] See also Figure 1 , Figure 1 A first flow chart of a 3D image generation method based on a neural network provided in an embodiment of the present invention.

[0036] S100, obtaining a 2D image to be processed and dividing each object in the image, inputting the 2D image to be processed into a first neural network model to identify the depth of field distance corresponding to each object; inputting the 2D image to be processed into a second neural network model to generate the object relationship in the image scene and identify the main object in the image; establishing a plane rectangular coordinate system with the center of the image of the 2D image to be processed as the origin, and obtaining the horizontal coordinates of each object;

[0037] It should be noted that the image to be processed may be a 2D picture or a 2D video obtained by the user. Each frame of the 2D video is intercepted for processing. The 2D image may be segmented by a thresholding method or an edge-based method to obtain various objects.

[0038] The first neural network model may be a bp neural network model, which is pre-trained with data analysis fitting using two-dimensional pixel coordinates and depth of field distance, so that the neural network model can estimate three-dimensional time coordinates based on two-dimensional pixel coordinates to achieve monocular ranging; after the 2D image to be processed is input into the first neural network model, the depth of field distance of each object relative to the lens position can be obtained, and the 2D image processed in this embodiment needs to be similar to the training set when the first neural network model is trained;

[0039] The second neural network model may specifically be a graph neural network model, which, after pre-training, can identify the relationship between scene objects in a 2D image and determine the relationship between various objects in the image scene. When composing a video or photo, there will be a subject in the picture, which may be a person, an animal or a plant landscape. The subject of the picture is the object that the user's attention will focus on when viewing the image. The subject is regarded as the main object, and the main object as the subject in the image is identified based on the scene relationship between various objects in the picture.

[0040] Only a two-dimensional plane rectangular coordinate system can be established on the 2D image to be processed. The image center point is used as the origin coordinate system. The horizontal coordinates on both sides of the vertical central axis have positive and negative values. The horizontal coordinates are identified based on the position of each object in the image. In this embodiment, only the object translation caused by the distance between the left and right eyes is considered, and there is no need to consider the influence of the vertical direction of the object in the image. Therefore, it is only necessary to identify the horizontal coordinates of the object; each object is calculated using the coordinates of its horizontal midpoint position. If the second neural network model identifies multiple main objects, the multiple main objects can be regarded as one.

[0041] S200, obtaining the user's 3D device parameters to obtain the user's screen distance; respectively substituting the horizontal coordinates and depth of field distance of each non-screen main object into the object translation calculation model to obtain the object translation parameters, wherein the object translation calculation model is specifically:

[0042]

[0043] Among them, L0 is the user screen distance, θ is the angle between the current monocular line of sight looking at the main object and the connecting line between the main object and the screen, d0 is the user pupil distance, x0 is the horizontal coordinate of the main object, L1 is the depth of field distance of the main object, x n is the horizontal coordinate of the nth non-image main object, L n is the depth of field distance of the nth non-image main object, Δd is the difference between the horizontal coordinate of the nth non-image main object and the horizontal coordinate of the main object, and D is the object translation parameter. When calculating the translation of the object from the left eye perspective, a=1, and when calculating the translation of the object from the right eye perspective, a=0;

[0044] It should be noted that the user's 3D device may be a 3D display or a head-mounted 3D device. The recommended user viewing screen distance can be obtained based on the device installation parameters of the 3D display. For head-mounted 3D devices, the distance between the internal display area of ​​the device and the user's eyes is the distance between the user and the screen.

[0045] See also Figure 2 , 1 is the user's left eye, 2 is the main object, and 3 is the nth non-screen main object. The human eye recognizes 3D images mainly by relying on the position differences between different objects seen by binoculars to recognize the distance and depth of objects. Since 2D images are taken by a monocular camera, the relative positions of objects in the 2D image can be regarded as the effect of viewing from a monocular perspective in the center of the screen. The main object is where the user's sight focuses when watching an image or video, and the main object will not move. To display it as a 3D image, the relative positions of objects in the image need to be adjusted according to the binocular positions. Figure 2 Taking the position of the user's left eye as an example, after the sight direction is changed, the horizontal relative position between the nth non-main object and the main object relative to the user's left eye perspective changes. After comparing with the original horizontal coordinate difference, the translation parameter of the non-main object can be obtained. The translation parameter has positive and negative values, which are reflected as two horizontal translation directions in the coordinate system.

[0046] The user's interpupillary distance is generally 6 cm, which can be set according to specific user needs. In this embodiment, the horizontal positions of the left and right eyes are respectively half the interpupillary distance on both sides of the origin, and there are positive and negative differences in the coordinates. There are differences in the object translation parameter calculation models corresponding to the left and right eyes. When calculating the translation parameters of each object in the 3D picture of the user's left eye, a needs to be taken as 1, and when calculating the translation parameters of each object in the 3D picture of the user's right eye, a needs to be taken as 0.

[0047] S300, adjusting each object in the 2D image to be processed with the left eye object translation parameter to obtain a left eye 3D image, and adjusting each object in the 2D image to be processed with the right eye object translation parameter to obtain a right eye 3D image; splicing the left eye 3D image and the right eye 3D image based on the user's 3D device playback mode to display the 3D image to the user.

[0048] It should be noted that the segmented objects are translated with the corresponding translation parameters. When the translation parameter is a positive value, the objects are translated in the positive direction of the horizontal coordinate axis, and when the translation parameter is a negative value, the objects are translated in the negative direction of the horizontal coordinate axis. The 3D image finally needs to be spliced ​​before it can be played for the user. According to the playback mode of the user's 3D device, if it is a binocular projection device, the edges of the left eye 3D image and the right eye 3D image can be directly fitted and spliced. If it is a light barrier or cylindrical lens display mode, the left eye 3D image and the right eye 3D image can be cropped and then spliced ​​alternately. After splicing, it can be played on the user's 3D device to show 3D to the user.

[0049] In this embodiment, by dividing the objects in the 2D image to be processed, calculating the depth of field distance of each object with a neural network, and finding the main object, the position difference of each object in the 2D image to be processed and the corresponding 3D image is calculated based on the difference between binocular imaging and monocular imaging, and the left and right eye images are adjusted and spliced ​​respectively, the 2D image can be efficiently generated into a 3D image without being restricted by the original image or shooting conditions, thereby improving the user's viewing experience.

[0050] The above is a detailed description of the first embodiment of a 3D image generation method based on a neural network provided by the present application. The following is a detailed description of the second embodiment of a 3D image generation method based on a neural network provided by the present application.

[0051] In this embodiment, a 3D image generation method based on a neural network is further provided, which specifically includes step S400, specifically:

[0052] S400, acquiring a second 2D image, performing a similarity comparison between the second 2D image and the 2D image to be processed based on image features, and if the similarity exceeds a preset first threshold, dividing the second 2D image to obtain a second image object; performing a similarity comparison between a main object in the 2D image to be processed and an object in the second image, and setting the second image object whose similarity exceeds the preset second threshold as the main object.

[0053] It should be noted that when the subsequently processed 2D images are similar images, such as continuous flat illustrations or continuous frames of 2D videos, the movement of objects in the images and the movement of the lens are also continuous, so the objects in the pictures are actually roughly similar; based on the image similarity calculation algorithm, the similarity calculation can be performed on the 2D image to be processed in the aforementioned step S100 and the second 2D image to be processed subsequently; when the similarity exceeds a preset first threshold, it means that the second 2D image is similar to the object in the 2D image to be processed, and the images must be continuous; the first threshold can be based on an empirical value or image content. After determining that the second 2D image is similar, the objects in the image are identified, and each object in the second 2D image is first divided out, and then the main object in the 2D image to be processed is compared with the second image object for similarity. If there is a second image object whose similarity exceeds a preset second threshold, it can be set as the main object. Because the audience's line of sight in a continuous picture, such as in a video, will follow the main object, such as following the movement of the protagonist in a movie, the main object in the continuous picture can remain unchanged, and there is no need to input the second 2D image into the second neural network model, thereby improving the processing efficiency of subsequent 3D images.

[0054] Furthermore, in the aforementioned step S300, after the left eye 3D image and the right eye 3D image are spliced, the method further includes: inputting the left eye 3D image and the right eye 3D image into the third neural network model respectively, and repairing the vacant area of ​​the image after the object is translated. It should be noted that there is a mutual occlusion relationship between the objects in the 2D image to be processed. After the objects are adjusted with the translation parameters in step S300, the original occlusion area may be vacant due to the translation, and the coverage relationship of the two overlapping objects is adjusted after the translation according to the depth of field distance relationship of the objects; the third neural network model can use a BP neural network or a self-organizing feature mapping neural network, which is pre-trained according to the overall shape of the image object, and can fill the vacant area according to the overall picture relationship in the translated left eye 3D image or the right eye 3D image. For example, after the person is translated in the grass, the grass becomes vacant. The third neural network model can repair the grass around the vacant area, and collect the grass image to repair the vacant area.

[0055] The above is a detailed description of a neural network-based 3D image generation method according to the first aspect of the present application. The following is a detailed description of an embodiment of a neural network-based 3D image generation system according to the second aspect of the present application.

[0056] See also Figure 3 , Figure 3 This embodiment provides a 3D image generation system based on a neural network, including:

[0057] The object data acquisition module 10 is used to acquire the 2D image to be processed and divide each object in the image, input the 2D image to be processed into the first neural network model to identify the depth of field distance corresponding to each object; input the 2D image to be processed into the second neural network model to generate the object relationship in the image scene and identify the main object in the image; establish a plane rectangular coordinate system with the center of the image of the 2D image to be processed as the origin to obtain the horizontal coordinates of each object;

[0058] The translation parameter calculation module 20 is used to obtain the user's 3D device parameters and obtain the user's screen distance; the horizontal coordinates and depth of field distance of each non-screen main object are respectively substituted into the object translation calculation model to obtain the object translation parameters. The object translation calculation model is specifically:

[0059]

[0060] Among them, L0 is the user screen distance, θ is the angle between the current monocular line of sight looking at the main object and the connecting line between the main object and the screen, d0 is the user pupil distance, x0 is the horizontal coordinate of the main object, L1 is the depth of field distance of the main object, x n is the horizontal coordinate of the nth non-image main object, L n is the depth of field distance of the nth non-image main object, Δd is the difference between the horizontal coordinate of the nth non-image main object and the horizontal coordinate of the main object, and D is the object translation parameter. When calculating the translation of the object from the left eye perspective, a=1, and when calculating the translation of the object from the right eye perspective, a=0;

[0061] The 3D image generation module 30 is used to adjust each object in the 2D image to be processed with the left eye object translation parameter to obtain a left eye 3D image, and adjust each object in the 2D image to be processed with the right eye object translation parameter to obtain a right eye 3D image; based on the user's 3D device playback mode, the left eye 3D image and the right eye 3D image are spliced ​​to display the 3D image to the user.

[0062] Furthermore, it also includes:

[0063] The continuous image processing module 40 is used to obtain a second 2D image, compare the second 2D image with the 2D image to be processed based on image features, and if the similarity exceeds a preset first threshold, divide the second 2D image to obtain a second image object; compare the similarity between the main object in the 2D image to be processed and the second image object, and set the second image object whose similarity exceeds the preset second threshold as the main object.

[0064] Furthermore, in the 3D image generation module 30, after splicing the left-eye 3D image and the right-eye 3D image, the following steps are specifically included:

[0065] The left-eye 3D image and the right-eye 3D image are respectively input into the third neural network model to fill in the missing areas of the image after the object is translated.

[0066] The third aspect of the present application also provides a 3D image generation method device based on a neural network, including a processor and a memory: wherein the memory is used to store program code and transmit the program code to the processor; the processor is used to execute the above-mentioned 3D image generation method based on a neural network according to the instructions in the program code.

[0067] A fourth aspect of the present application provides a computer-readable storage medium, characterized in that the computer-readable storage medium is used to store program code, and the program code is used to execute the above-mentioned neural network-based 3D image generation method.

[0068] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and equipment can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0069] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0070] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0071] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0072] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0073] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A 3D image generation method based on a neural network, characterized in that include: Acquire the 2D image to be processed and divide each object in the image, input the 2D image to be processed into the first neural network model to identify the depth of field distance corresponding to each object; input the 2D image to be processed into the second neural network model, generate the object relationship in the image scene and identify the main object in the image; establish a plane rectangular coordinate system with the center of the image of the 2D image to be processed as the origin, and obtain the horizontal coordinates of each object; Obtain the user's 3D device parameters to obtain the user's screen distance; substitute the horizontal coordinates and depth of field distance of each non-screen main object into the object translation calculation model to obtain the object translation parameters. The object translation calculation model is specifically: Among them, L0 is the user screen distance, θ is the angle between the current monocular line of sight looking at the main object and the connecting line between the main object and the screen, d0 is the user pupil distance, x0 is the horizontal coordinate of the main object, L1 is the depth of field distance of the main object, x n is the horizontal coordinate of the nth non-image main object, L n is the depth of field distance of the nth non-image main object, Δd is the difference between the horizontal coordinate of the nth non-image main object and the horizontal coordinate of the main object, and D is the object translation parameter. When calculating the translation of the object from the left eye perspective, a=1, and when calculating the translation of the object from the right eye perspective, a=0; Each object in the 2D image to be processed is adjusted by the left eye object translation parameter to obtain a left eye 3D image, and each object in the 2D image to be processed is adjusted by the right eye object translation parameter to obtain a right eye 3D image; based on the user's 3D device playback mode, the left eye 3D image and the right eye 3D image are spliced ​​to display the 3D image to the user.

2. The 3D image generation method based on a neural network according to claim 1, characterized in that: Also includes: A second 2D image is acquired, and based on image features, the second 2D image is compared with the 2D image to be processed for similarity; if the similarity exceeds a preset first threshold, the second 2D image is divided to obtain a second image object; a main object in the 2D image to be processed is compared with the second image object for similarity, and the second image object whose similarity exceeds the preset second threshold is set as the main object.

3. The 3D image generation method based on a neural network according to claim 1, characterized in that: After the left-eye 3D image and the right-eye 3D image are stitched together, the method further includes: The left-eye 3D image and the right-eye 3D image are respectively input into the third neural network model to fill in the missing areas of the image after the object is translated.

4. A 3D image generation system based on a neural network, characterized in that: include: The object data acquisition module is used to acquire the 2D image to be processed and divide the objects in the image, input the 2D image to be processed into the first neural network model to identify the depth of field distance corresponding to each object; input the 2D image to be processed into the second neural network model to generate the object relationship in the image scene and identify the main object in the image; establish a plane rectangular coordinate system with the center of the image of the 2D image to be processed as the origin to obtain the horizontal coordinates of each object; The translation parameter calculation module is used to obtain the user's 3D device parameters and obtain the user's screen distance; the horizontal coordinates and depth of field distance of each non-screen main object are respectively substituted into the object translation calculation model to obtain the object translation parameters. The object translation calculation model is specifically: Among them, L0 is the user screen distance, θ is the angle between the current monocular line of sight looking at the main object and the connecting line between the main object and the screen, d0 is the user pupil distance, x0 is the horizontal coordinate of the main object, L1 is the depth of field distance of the main object, x n is the horizontal coordinate of the nth non-image main object, L n is the depth of field distance of the nth non-image main object, Δd is the difference between the horizontal coordinate of the nth non-image main object and the horizontal coordinate of the main object, and D is the object translation parameter. When calculating the translation of the object from the left eye perspective, a=1, and when calculating the translation of the object from the right eye perspective, a=0; The 3D image generation module is used to adjust each object in the 2D image to be processed with the left eye object translation parameter to obtain the left eye 3D image, and adjust each object in the 2D image to be processed with the right eye object translation parameter to obtain the right eye 3D image; based on the user's 3D device playback mode, the left eye 3D image and the right eye 3D image are spliced ​​to display the 3D image to the user.

5. The 3D image generation system based on a neural network according to claim 4, characterized in that: Also includes: The continuous image processing module is used to obtain a second 2D image, compare the second 2D image with the 2D image to be processed based on image features, and if the similarity exceeds a preset first threshold, divide the second 2D image to obtain a second image object; compare the similarity between the main object in the 2D image to be processed and the second image object, and set the second image object whose similarity exceeds the preset second threshold as the main object.

6. The 3D image generation system based on a neural network according to claim 4, characterized in that: In the 3D image generation module, after splicing the left-eye 3D image and the right-eye 3D image, the following is specifically included: The left-eye 3D image and the right-eye 3D image are respectively input into the third neural network model to fill in the missing areas of the image after the object is translated.

7. A 3D image generation method and device based on a neural network, characterized in that: The device comprises a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the neural network-based 3D image generation method described in any one of claims 1 to 3 according to the instructions in the program code.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program codes, and the program codes are used to execute the neural network-based 3D image generation method according to any one of claims 1 to 3.

Images