A 3D image generation method and system based on neural network

Abstract

The present invention relates to the field of naked-eye 3D technology and provides a neural network-based 3D image generation method and system. The method comprises: dividing each object in a 2D image to be processed, calculating the depth of field distance of each object using a neural network, and after finding the main object, calculating the position difference between each object in the 2D image to be processed and the corresponding 3D image based on the difference between binocular imaging and monocular imaging, and adjusting the left and right eye images respectively before splicing them. In this way, the 2D image can be efficiently generated into a 3D image without being restricted by the original image or shooting conditions, thereby improving the user's viewing experience.

Description

Technical Field

[0001] The present invention relates to the field of naked-eye 3D technology, and in particular to a 3D image generation method and system based on a neural network. Background Art

[0002] With the development of technology, users are becoming more and more fond of the immersive experience brought by 3D technology, and they hope that more images or videos can be produced with 3D effects. However, due to the limitations of shooting equipment or shooting technology, not everyone can directly shoot 3D images, and many users' private image data can no longer be reproduced with 3D shooting equipment. Summary of the Invention

[0003] The present invention provides a 3D image generation method based on a neural network, which is used to solve the problem of harsh conditions for acquiring 3D images in the prior art.

[0004] A first aspect of the present invention provides a 3D image generation method based on a neural network, comprising:

[0005] Obtaining a 2D image to be processed and dividing each object in the image, inputting the 2D image to be processed into a first neural network model to identify the depth of field distance corresponding to each object; inputting the 2D image to be processed into a second neural network model to generate the object relationship in the image scene and identify the main object in the image; establishing a plane rectangular coordinate system with the center of the image to be processed as the origin to obtain the horizontal coordinates of each object;

[0006] Obtain the user's 3D device parameters to obtain the user's screen distance; substitute the horizontal coordinates and depth of field distance of each non-screen main object into the object translation calculation model to obtain the object translation parameters. The object translation calculation model is specifically as follows:

[0007]

[0008] Among them, L0 is the distance between the user and the screen, θ is the angle between the current monocular sight line looking at the main object and the line connecting the main object and the screen, d0 is the user's pupil distance, x0 is the horizontal coordinate of the main object, L1 is the depth of field distance of the main object, x n is the horizontal coordinate of the nth non-image main object, L n is the depth of field distance of the nth non-image main object, Δd is the difference between the horizontal coordinate of the nth non-image main object and the horizontal coordinate of the main object, and D is the object translation parameter. When calculating the object translation from the left eye perspective, a=1, and when calculating the object translation from the right eye perspective, a=0;

[0009] Each object in the 2D image to be processed is adjusted using the left-eye object translation parameters to obtain a left-eye 3D image, and each object in the 2D image to be processed is adjusted using the right-eye object translation parameters to obtain a right-eye 3D image; based on the playback mode of the user's 3D device, the left-eye 3D image and the right-eye 3D image are spliced ​​to display a 3D image to the user.

[0010] Optionally, also include:

[0011] A second 2D image is acquired, and based on image features, the second 2D image is compared with the 2D image to be processed for similarity. If the similarity exceeds a preset first threshold, the second 2D image is divided to obtain second image objects. A primary object in the 2D image to be processed is compared with the second image object for similarity, and the second image object whose similarity exceeds the preset second threshold is set as the primary object.

[0012] Optionally, after stitching the left-eye 3D image and the right-eye 3D image, the method further includes:

[0013] The left-eye 3D image and the right-eye 3D image are respectively input into the third neural network model to fill in the vacant areas of the image after the object is translated.

[0014] A second aspect of the present application provides a 3D image generation system based on a neural network, comprising:

[0015] The object data acquisition module is used to acquire the 2D image to be processed and divide the objects in the image, input the 2D image to be processed into the first neural network model to identify the depth of field distance corresponding to each object; input the 2D image to be processed into the second neural network model to generate the object relationship in the image scene and identify the main object in the image; establish a plane rectangular coordinate system with the center of the image to be processed as the origin to obtain the horizontal coordinates of each object;

[0016] The translation parameter calculation module is used to obtain the user's 3D device parameters and obtain the user screen distance; the horizontal coordinates and depth of field distance of each non-screen main object are respectively substituted into the object translation calculation model to obtain the object translation parameters. The object translation calculation model is specifically:

[0017]

[0018] Among them, L0 is the distance between the user and the screen, θ is the angle between the current monocular sight line looking at the main object and the line connecting the main object and the screen, d0 is the user's pupil distance, x0 is the horizontal coordinate of the main object, L1 is the depth of field distance of the main object, x n is the horizontal coordinate of the nth non-image main object, L nis the depth of field distance of the nth non-image main object, Δd is the difference between the horizontal coordinate of the nth non-image main object and the horizontal coordinate of the main object, and D is the object translation parameter. When calculating the object translation from the left eye perspective, a=1, and when calculating the object translation from the right eye perspective, a=0;

[0019] The 3D image generation module is used to adjust each object in the 2D image to be processed using the left-eye object translation parameters to obtain a left-eye 3D image, and to adjust each object in the 2D image to be processed using the right-eye object translation parameters to obtain a right-eye 3D image; based on the playback mode of the user's 3D device, the left-eye 3D image and the right-eye 3D image are spliced ​​together to display the 3D image to the user.

[0020] Optionally, also include:

[0021] The continuous image processing module is configured to obtain a second 2D image, perform a similarity comparison between the second 2D image and the 2D image to be processed based on image features, and if the similarity exceeds a preset first threshold, divide the second 2D image to obtain second image objects; perform a similarity comparison between a main object in the 2D image to be processed and an object in the second image, and set the second image object whose similarity exceeds the preset second threshold as the main object.

[0022] Optionally, after the left-eye 3D image and the right-eye 3D image are spliced ​​together, the 3D image generation module further includes:

[0023] The left-eye 3D image and the right-eye 3D image are respectively input into the third neural network model to fill in the vacant areas of the image after the object is translated.

[0024] A third aspect of the present application provides a 3D image generation method and device based on a neural network, the device comprising a processor and a memory:

[0025] The memory is used to store program code and transmit the program code to the processor;

[0026] The processor is configured to execute the neural network-based 3D image generation method according to any one of the first aspects of the present invention according to the instructions in the program code.

[0027] A fourth aspect of the present application provides a computer-readable storage medium, characterized in that the computer-readable storage medium is used to store program code, and the program code is used to execute the neural network-based 3D image generation method described in any one of the first aspects of the present invention.

[0028] As can be seen from the above technical solution, the present invention has the following advantages: by dividing the objects in the 2D image to be processed, calculating the depth of field distance of each object with a neural network, and finding the main object, based on the difference between binocular imaging and monocular imaging, calculating the position difference of each object in the 2D image to be processed and the corresponding 3D image, and adjusting the left and right eye images respectively and then splicing them, the 2D image can be efficiently generated into a 3D image without being restricted by the original image or shooting conditions, thereby improving the user's viewing experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0030] Figure 1 This is the first flowchart of a neural network-based 3D image generation method;

[0031] Figure 2 Schematic diagram for calculating the translation parameters of the left eye 2D image;

[0032] Figure 3 This is a structural diagram of a 3D image generation system based on a neural network. DETAILED DESCRIPTION

[0033] In order to make the purpose, features, and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0034] The present invention provides a 3D image generation method based on a neural network, which is used to solve the problem of harsh conditions for acquiring 3D images in the prior art.

[0035] See also Figure 1 , Figure 1 This is a first flowchart of a neural network-based 3D image generation method provided by an embodiment of the present invention.

[0036] S100, obtaining a 2D image to be processed and dividing each object in the image, inputting the 2D image to be processed into a first neural network model to identify the depth of field distance corresponding to each object; inputting the 2D image to be processed into a second neural network model to generate object relationships in the image scene and identify the main object in the image; establishing a plane rectangular coordinate system with the center of the image of the 2D image to be processed as the origin to obtain the horizontal coordinates of each object;

[0037] It should be noted that the image to be processed can be a 2D picture or a 2D video obtained by the user. Each frame of the 2D video is intercepted for processing. The 2D image can be segmented using a thresholding method or an edge-based method to obtain individual objects.

[0038] The first neural network model may be a BP neural network model. The BP neural network model is pre-trained using data analysis and fitting using two-dimensional pixel coordinates and depth of field distances, so that the neural network model can estimate three-dimensional time coordinates based on the two-dimensional pixel coordinates to achieve monocular ranging. After inputting the 2D image to be processed into the first neural network model, the depth of field distance of each object relative to the lens position can be obtained. The 2D image processed in this embodiment must be similar to the training set used when training the first neural network model.

[0039] The second neural network model can specifically be a graph neural network model. After pre-training, it can identify the relationship between scene objects in 2D images and determine the relationship between each object in the image scene. When composing a video or photo, there will be a main subject, which may be a person, an animal, or a plant scene. The main subject is the object that the user's attention is focused on when viewing the image. It is regarded as the main object. Based on the scene relationship between each object in the image, the main object as the main subject in the image is identified;

[0040] Only a two-dimensional plane rectangular coordinate system can be established on the 2D image to be processed. The image center point is used as the origin coordinate for establishment. The horizontal coordinates on both sides of the vertical central axis have positive and negative values. The horizontal coordinates are identified based on the position of each object in the image. In this embodiment, only the object translation caused by the distance between the left and right eyes is considered, and there is no need to consider the influence of the vertical direction of the object in the image. Therefore, it is only necessary to identify the horizontal coordinates of the object; each object is calculated using the coordinates of its horizontal midpoint position. If the second neural network model identifies multiple main objects, the multiple main objects can be regarded as one.

[0041] S200: Obtain user 3D device parameters to obtain the user screen distance; substitute the horizontal coordinates and depth of field distance of each non-screen main object into the object translation calculation model to obtain object translation parameters. The object translation calculation model is specifically:

[0042]

[0043] Among them, L0 is the distance between the user and the screen, θ is the angle between the current monocular sight line looking at the main object and the line connecting the main object and the screen, d0 is the user's pupil distance, x0 is the horizontal coordinate of the main object, L1 is the depth of field distance of the main object, x n is the horizontal coordinate of the nth non-image main object, L n is the depth of field distance of the nth non-image main object, Δd is the difference between the horizontal coordinate of the nth non-image main object and the horizontal coordinate of the main object, and D is the object translation parameter. When calculating the object translation from the left eye perspective, a=1, and when calculating the object translation from the right eye perspective, a=0;

[0044] It should be noted that the user's 3D device may be a 3D display or a head-mounted 3D device. The recommended user viewing screen distance can be obtained based on the device installation parameters of the 3D display. For head-mounted 3D devices, the user-screen distance is obtained by the distance between the device's internal display area and the user's eyes.

[0045] See Figure 2 , 1 is the user's left eye, 2 is the main object, and 3 is the nth non-image main object. The human eye recognizes 3D images mainly by relying on the position differences between different objects seen by the binoculars to recognize the distance and depth of the objects. Since the 2D image is taken by a monocular camera, the relative position of the objects in the 2D image can be regarded as the effect of viewing from a monocular perspective in the center of the picture. The main object is where the user's sight is focused when watching the image or video, and the main object will not move. To display it as a 3D image, the relative positions of the objects in the image need to be adjusted according to the binocular positions. Figure 2 Taking the position of the user's left eye as an example, after the line of sight direction is changed, the horizontal relative position between the nth non-main object and the main object relative to the user's left eye perspective changes. After comparing the difference with the original horizontal coordinate, the translation parameter of the non-main object can be obtained. This translation parameter has positive and negative values ​​and is reflected as two horizontal translation directions in the coordinate system.

[0046] The user's interpupillary distance is generally 6 cm and can be set according to specific user needs. In this embodiment, the horizontal positions of the left and right eyes are half the interpupillary distance on either side of the origin, and there are positive and negative coordinates. The object translation parameter calculation models corresponding to the left and right eyes are different. When calculating the translation parameters of each object in the user's left eye 3D picture, a needs to be set to 1. When calculating the translation parameters of each object in the user's right eye 3D picture, a needs to be set to 0.

[0047] S300: Adjust each object in the 2D image to be processed using the left-eye object translation parameter to obtain a left-eye 3D image, and adjust each object in the 2D image to be processed using the right-eye object translation parameter to obtain a right-eye 3D image; based on the playback mode of the user's 3D device, splice the left-eye 3D image and the right-eye 3D image to display the 3D image to the user.

[0048] It should be noted that the segmented objects are translated with the corresponding translation parameters. When the translation parameter is positive, the objects are translated in the positive direction of the horizontal coordinate axis, and when the translation parameter is negative, the objects are translated in the negative direction of the horizontal coordinate axis. The 3D image finally needs to be spliced ​​before it can be played for the user. According to the playback mode of the user's 3D device, if it is a binocular projection device, the edges of the left eye 3D image and the right eye 3D image can be directly aligned and spliced. If it is a light barrier or cylindrical lens display mode, the left eye 3D image and the right eye 3D image can be cropped and then spliced ​​alternately. After splicing, it can be played on the user's 3D device to show the user 3D.

[0049] In this embodiment, by dividing the objects in the 2D image to be processed, calculating the depth of field distance of each object using a neural network, and finding the main object, the position difference of each object in the 2D image to be processed and the corresponding 3D image is calculated based on the difference between binocular imaging and monocular imaging, and the left and right eye images are adjusted and spliced ​​separately. In this way, the 2D image can be efficiently generated into a 3D image without being restricted by the original image or shooting conditions, thereby improving the user's viewing experience.

[0050] The above is a detailed description of the first embodiment of a 3D image generation method based on a neural network provided by the present application. The following is a detailed description of the second embodiment of a 3D image generation method based on a neural network provided by the present application.

[0051] In this embodiment, a 3D image generation method based on a neural network is further provided, which specifically includes step S400, specifically:

[0052] S400: Acquire a second 2D image, perform a similarity comparison between the second 2D image and the 2D image to be processed based on image features, and if the similarity exceeds a preset first threshold, divide the second 2D image to obtain second image objects; perform a similarity comparison between the main object in the 2D image to be processed and the second image object, and set the second image object whose similarity exceeds the preset second threshold as the main object.

[0053] It should be noted that when the 2D images to be processed subsequently are similar images, such as continuous flat illustrations or continuous frames of 2D videos, the movement of objects in the images and the movement of the camera are also continuous, so the objects in the pictures are actually roughly similar; based on the image similarity calculation algorithm, the similarity calculation can be performed on the 2D image to be processed in the aforementioned step S100 and the second 2D image to be processed subsequently. When the similarity exceeds a preset first threshold, it means that the second 2D image is similar to the object in the 2D image to be processed, and the images must be continuous; the first threshold can be based on an empirical value or image content. After determining that the second 2D image is similar, the objects in the image are identified. After first dividing the various objects in the second 2D image, the main object in the 2D image to be processed is compared with the second image object for similarity. If there is a second image object whose similarity exceeds a preset second threshold, it can be set as the main object. Because the audience's eyesight in continuous pictures, such as in a video, will follow the main object, such as following the movement of the protagonist in a movie, the main object in the continuous picture can remain unchanged, and there is no need to input the second 2D image into the second neural network model, thereby improving the processing efficiency of subsequent 3D images.

[0054] Furthermore, in the aforementioned step S300, after the left-eye 3D image and the right-eye 3D image are spliced ​​together, the method further includes: inputting the left-eye 3D image and the right-eye 3D image into a third neural network model respectively, and patching the image vacant areas after the objects are translated. It should be noted that there is a mutual occlusion relationship between the objects in the 2D image to be processed. When the objects are adjusted with the translation parameters in step S300, the original occluded areas may become vacant due to the translation, and the coverage relationship of the two overlapping objects is adjusted after the translation based on the depth of field distance relationship of the objects; the third neural network model can adopt a BP neural network or a self-organizing feature mapping neural network, which is pre-trained based on the overall shape of the image objects. It can fill the vacant areas based on the overall picture relationship in the translated left-eye 3D image or the right-eye 3D image. For example, after a person is translated in the grass, the grass becomes vacant. The third neural network model can patch the grass around the vacant area and collect the grass image to patch the vacant area.

[0055] The above is a detailed description of a neural network-based 3D image generation method according to the first aspect of the present application. The following is a detailed description of an embodiment of a neural network-based 3D image generation system according to the second aspect of the present application.

[0056] See also Figure 3 , Figure 3 This embodiment provides a 3D image generation system based on a neural network, including:

[0057] The object data acquisition module 10 is used to acquire the 2D image to be processed and divide the objects in the image, input the 2D image to be processed into the first neural network model to identify the depth of field distance corresponding to each object; input the 2D image to be processed into the second neural network model to generate the object relationship in the image scene and identify the main object in the image; establish a plane rectangular coordinate system with the center of the image to be processed as the origin to obtain the horizontal coordinates of each object;

[0058] The translation parameter calculation module 20 is used to obtain the user's 3D device parameters and obtain the user screen distance; the horizontal coordinates and depth of field distance of each non-screen main object are respectively substituted into the object translation calculation model to obtain the object translation parameters. The object translation calculation model is specifically:

[0059]

[0060] Among them, L0 is the distance between the user and the screen, θ is the angle between the current monocular sight line looking at the main object and the line connecting the main object and the screen, d0 is the user's pupil distance, x0 is the horizontal coordinate of the main object, L1 is the depth of field distance of the main object, x n is the horizontal coordinate of the nth non-image main object, L n is the depth of field distance of the nth non-image main object, Δd is the difference between the horizontal coordinate of the nth non-image main object and the horizontal coordinate of the main object, and D is the object translation parameter. When calculating the object translation from the left eye perspective, a=1, and when calculating the object translation from the right eye perspective, a=0;

[0061] The 3D image generation module 30 is used to adjust each object in the 2D image to be processed using the left-eye object translation parameters to obtain a left-eye 3D image, and to adjust each object in the 2D image to be processed using the right-eye object translation parameters to obtain a right-eye 3D image; based on the playback mode of the user's 3D device, the left-eye 3D image and the right-eye 3D image are spliced ​​together to display the 3D image to the user.

[0062] Furthermore, it also includes:

[0063] The continuous image processing module 40 is configured to obtain a second 2D image, perform a similarity comparison between the second 2D image and the 2D image to be processed based on image features, and if the similarity exceeds a preset first threshold, divide the second 2D image to obtain second image objects; perform a similarity comparison between the main object in the 2D image to be processed and the second image object, and set the second image object whose similarity exceeds the preset second threshold as the main object.

[0064] Furthermore, after the left-eye 3D image and the right-eye 3D image are spliced ​​together, the 3D image generation module 30 may further include:

[0065] The left-eye 3D image and the right-eye 3D image are respectively input into the third neural network model to fill in the vacant areas of the image after the object is translated.

[0066] The third aspect of the present application also provides a 3D image generation method device based on a neural network, including a processor and a memory: the memory is used to store program code and transmit the program code to the processor; the processor is used to execute the above-mentioned 3D image generation method based on a neural network according to the instructions in the program code.

[0067] A fourth aspect of the present application provides a computer-readable storage medium, characterized in that the computer-readable storage medium is used to store program code, and the program code is used to execute the above-mentioned neural network-based 3D image generation method.

[0068] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and equipment can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0069] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0070] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0071] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0072] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0073] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A 3D image generation method based on a neural network, characterized in that include: Obtaining a 2D image to be processed and dividing each object in the image, inputting the 2D image to be processed into a first neural network model to identify the depth of field distance corresponding to each object; inputting the 2D image to be processed into a second neural network model to generate the object relationship in the image scene and identify the main object that is the focus of attention in the image composition; establishing a plane rectangular coordinate system with the center of the 2D image to be processed as the origin to obtain the horizontal coordinates of each object; Get the user's 3D device parameters and the user's screen distance; Substitute the horizontal coordinates and depth of field distance of each non-screen main object into the object translation calculation model respectively, and obtain the left-eye object translation parameters and right-eye object translation parameters of each non-screen main object based on the main object position. The object translation calculation model is specifically: Among them, L0 is the distance between the user and the screen, θ is the angle between the current monocular sight line looking at the main object and the line connecting the main object and the screen, d0 is the user's pupil distance, x0 is the horizontal coordinate of the main object, L1 is the depth of field distance of the main object, x n is the horizontal coordinate of the nth non-image main object, L n is the depth of field distance of the nth non-image main object, Δd is the difference between the horizontal coordinate of the nth non-image main object and the horizontal coordinate of the main object, and D is the object translation parameter. When calculating the object translation from the left eye perspective, a=1, and when calculating the object translation from the right eye perspective, a=0; Each non-image main object in the 2D image to be processed is adjusted using the left-eye object translation parameter to obtain a left-eye 3D image. Each non-image main object in the 2D image to be processed is adjusted using the right-eye object translation parameter to obtain a right-eye 3D image, while the main object is not adjusted. Based on the playback mode of the user's 3D device, the left-eye 3D image and the right-eye 3D image are spliced ​​to display a 3D image to the user.

2. The 3D image generation method based on a neural network according to claim 1, characterized in that: Also includes: A second 2D image is acquired, and based on image features, the second 2D image is compared with the 2D image to be processed for similarity. If the similarity exceeds a preset first threshold, the second 2D image is divided to obtain second image objects. A primary object in the 2D image to be processed is compared with the second image object for similarity, and the second image object whose similarity exceeds the preset second threshold is set as the primary object.

3. The 3D image generation method based on a neural network according to claim 1, characterized in that: After the left-eye 3D image and the right-eye 3D image are stitched together, the method further includes: The left-eye 3D image and the right-eye 3D image are respectively input into the third neural network model to fill in the vacant areas of the image after the object is translated.

4. A 3D image generation system based on a neural network, characterized in that: include: The object data acquisition module is used to acquire the 2D image to be processed and divide the objects in the image, input the 2D image to be processed into the first neural network model to identify the depth of field distance corresponding to each object; input the 2D image to be processed into the second neural network model to generate the object relationship in the image scene and identify the main object that focuses attention in the image composition; establish a plane rectangular coordinate system with the center of the 2D image to be processed as the origin to obtain the horizontal coordinates of each object; The translation parameter calculation module is used to obtain the user's 3D device parameters and obtain the user's screen distance; Substitute the horizontal coordinates and depth of field distance of each non-screen main object into the object translation calculation model respectively, and obtain the left-eye object translation parameters and right-eye object translation parameters of each non-screen main object based on the main object position. The object translation calculation model is specifically: Among them, L0 is the distance between the user and the screen, θ is the angle between the current monocular sight line looking at the main object and the line connecting the main object and the screen, d0 is the user's pupil distance, x0 is the horizontal coordinate of the main object, L1 is the depth of field distance of the main object, x n is the horizontal coordinate of the nth non-image main object, L n is the depth of field distance of the nth non-image main object, Δd is the difference between the horizontal coordinate of the nth non-image main object and the horizontal coordinate of the main object, and D is the object translation parameter. When calculating the object translation from the left eye perspective, a=1, and when calculating the object translation from the right eye perspective, a=0; The 3D image generation module is configured to adjust each non-image main object in the 2D image to be processed using the left-eye object translation parameter to obtain a left-eye 3D image, and adjust each non-image main object in the 2D image to be processed using the right-eye object translation parameter to obtain a right-eye 3D image, while the main objects remain unchanged; and based on the playback mode of the user's 3D device, splice the left-eye 3D image and the right-eye 3D image to display the 3D image to the user.

5. The neural network-based 3D image generation system according to claim 4, characterized in that: Also includes: The continuous image processing module is configured to obtain a second 2D image, perform a similarity comparison between the second 2D image and the 2D image to be processed based on image features, and if the similarity exceeds a preset first threshold, divide the second 2D image to obtain second image objects; perform a similarity comparison between a main object in the 2D image to be processed and an object in the second image, and set the second image object whose similarity exceeds the preset second threshold as the main object.

6. The neural network-based 3D image generation system according to claim 4, characterized in that: In the 3D image generation module, after splicing the left-eye 3D image and the right-eye 3D image, the module further includes: The left-eye 3D image and the right-eye 3D image are respectively input into the third neural network model to fill in the vacant areas of the image after the object is translated.

7. A 3D image generation method and device based on a neural network, characterized in that: The device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the neural network-based 3D image generation method according to any one of claims 1 to 3 according to the instructions in the program code.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store program code, and the program code is used to execute the neural network-based 3D image generation method according to any one of claims 1 to 3.

Images