User image populating system, method and computer program product

Abstract

The application discloses a user image filling system, method and computer program product, relates to the technical field of image processing, and the user image filling system comprises a central device, a wearable device and an isolation layer. The central device comprises a first infrared sensor, a first image sensor and a first terahertz sensor, the wearable device comprises a second infrared sensor, a second image sensor and a second terahertz sensor, and the isolation layer is used for forming a closed space. The central device acquires heat distribution information of a user through the infrared sensor, acquires user image information through the image sensor, and is also used for acquiring detailed information of the user through the terahertz sensor. The central device determines position information of the user according to the heat distribution information and the user image information. The user image is filled according to the position information of the user, the user image information and the detailed information of the user, so that the integrity of the user image is ensured, and the visual effect is improved and the application is expanded.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to user image filling systems, methods and computer program products. Background Technology

[0002] Virtual Reality (VR) technology is a technology that uses computer technology and sensing devices to simulate a virtual environment similar to the real world, allowing users to immerse themselves and interact with this virtual environment. VR technology brings users into a virtual environment and provides a sense of presence. Users can see, hear, and feel various scenes and objects in the virtual environment, thus gaining an immersive experience. VR technology offers multiple interaction methods, including controllers, gesture recognition, and eye tracking, enabling users to interact with objects and scenes in the virtual environment. Users can manipulate virtual objects and interact with the virtual environment through gestures, touch, or other means. VR technology has wide applications in various fields, including gaming, education and training, and business meetings. It provides users with a completely new way of experiencing things and has significant application prospects in many areas.

[0003] Current VR technology focuses on single-person use and experience, and has not been well implemented for multi-person connection use, especially when the user is obstructed or the lighting is dim, the character image cannot be fully displayed.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main purpose of this application is to provide a user image filling system, method and computer program product, which aims to solve the technical problem of incomplete display of human image.

[0006] To achieve the above objectives, this application proposes a user image filling system, the system comprising: a central device, a wearable device, and an isolation layer, wherein the central device comprises a first infrared sensor, a first image sensor, and a first terahertz sensor, and the wearable device comprises a second infrared sensor, a second image sensor, and a second terahertz sensor;

[0007] The isolation layer is used to form a closed space to isolate terahertz signals outside the closed space, and the central device and the wearable device are disposed within the closed space;

[0008] Both the first infrared sensor and the second infrared sensor are used to acquire the user's heat distribution information and transmit the heat distribution information to the central device in the form of terahertz signals.

[0009] Both the first image sensor and the second image sensor are used to acquire user image information and transmit the user image information to the central device in the form of terahertz signals;

[0010] Both the first terahertz sensor and the second terahertz sensor are used to acquire detailed information about the user and transmit the detailed information to the central device in the form of terahertz signals.

[0011] The central device is used to acquire the user's heat distribution information through the first infrared sensor and the second infrared sensor, and to acquire the user's image information through the first image sensor and the second image sensor. It is also used to acquire the user's detailed information through the first terahertz sensor and the second terahertz sensor.

[0012] The central device is also used to determine the user's location information based on the heat distribution information and the user image information;

[0013] The central device is also used to fill in the user image based on the user's location information, the user's image information, and the user's detailed information.

[0014] In one embodiment, the central device further includes a first transmitting component and a first receiving component; the wearable device further includes a second transmitting component and a second receiving component.

[0015] The first transmitting component is used to transmit the terahertz signal to the wearable device;

[0016] The first receiving component is used to receive terahertz signals emitted by the wearable device;

[0017] The second transmitting component is used to transmit the terahertz signal to the central device;

[0018] The second receiving component is used to receive terahertz signals emitted by the central device.

[0019] In one embodiment, the system further includes: a wireless signal transmitting device; the wearable device further includes: a first power generating device and a second power generating device, and an energy receiving module;

[0020] The wireless signal transmitting device is used to send a wireless charging signal to the first power generation device;

[0021] The first power generation device is used to convert the generated wireless charging signal into electrical energy and send the electrical energy to the energy receiving module;

[0022] The second power generation device is used to convert the generated mechanical energy into electrical energy and send the electrical energy to the energy receiving module;

[0023] The energy receiving module is used to receive the electrical energy and charge the wearable device based on the electrical energy.

[0024] To achieve the above objectives, this application proposes a user image filling method, which is applied to the user image filling system. The system includes: a central device, a wearable device, and an isolation layer. The central device includes a first infrared sensor, a first image sensor, and a first terahertz sensor. The wearable device includes a second infrared sensor, a second image sensor, and a second terahertz sensor.

[0025] The isolation layer forms a closed space, isolating terahertz signals outside the closed space, and the central device and the wearable device are located inside the closed space;

[0026] The first infrared sensor and the second infrared sensor acquire the user's heat distribution information and transmit the heat distribution information to the central device in the form of a terahertz signal.

[0027] The first image sensor and the second image sensor acquire user image information and transmit the user image information to the central device in the form of a terahertz signal;

[0028] The first terahertz sensor and the second terahertz sensor acquire detailed information about the user and transmit the detailed information to the central device in the form of terahertz signals;

[0029] The central device acquires the user's heat distribution information through the first infrared sensor and the second infrared sensor, and acquires the user's image information through the first image sensor and the second image sensor. It is also used to acquire the user's detailed information through the first terahertz sensor and the second terahertz sensor.

[0030] The central device also determines the user's location information based on the heat distribution information and the user image information;

[0031] The central device also fills in the user image based on the user's location information, the user's image information, and the user's detailed information.

[0032] In one embodiment, the central device further includes a first transmitting component and a first receiving component; the wearable device further includes a second transmitting component and a second receiving component.

[0033] The first transmitting component sends the terahertz signal to the wearable device;

[0034] The first receiving component receives a terahertz signal emitted by the wearable device;

[0035] The second transmitting component sends the terahertz signal to the central device;

[0036] The second receiving component receives terahertz signals emitted by the central device.

[0037] In one embodiment, the system further includes: a wireless signal transmitting device; the wearable device further includes: a first power generating device and a second power generating device, and an energy receiving module;

[0038] The wireless signal transmitting device sends a wireless charging signal to the first power generation device.

[0039] The first power generation device converts the generated wireless charging signal into electrical energy and sends the electrical energy to the energy receiving module;

[0040] The second power generation device converts the generated mechanical energy into electrical energy and sends the electrical energy to the energy receiving module.

[0041] The energy receiving module receives the electrical energy and charges the wearable device based on the electrical energy.

[0042] In one embodiment, the user's location information includes user category and user location coordinates, and the step of determining the user's location information based on the heat distribution information and the user image information includes:

[0043] Input the current image into the user recognition model to obtain user feature information;

[0044] The user category is determined based on the user feature information, and the corresponding user location coordinates are obtained based on the user category.

[0045] In one embodiment, prior to the step of inputting the current image to the user recognition model, the following steps are included:

[0046] The user's center location coordinates and heat profile are determined based on the heat distribution information;

[0047] Determine the user's color and brightness information based on the user's image information;

[0048] A user identification model is established based on the user's center position coordinates, thermal profile, color information, and brightness information.

[0049] In one embodiment, the step of filling the user image based on the user's location information, the user's image information, and the user's detailed information includes:

[0050] An image filling model is established based on the user's location information, the user's image information, and the user's detailed information.

[0051] Input the damaged image into the image filling model to obtain the user-repaired image.

[0052] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the user image filling method as described above.

[0053] One or more technical solutions proposed in this application have at least the following technical effects:

[0054] The user image filling system includes a central device, a wearable device, and an isolation layer. The central device includes a first infrared sensor, a first image sensor, and a first terahertz sensor. The wearable device includes a second infrared sensor, a second image sensor, and a second terahertz sensor. By setting up the central device and the wearable device, the computing and data processing capabilities of the VR device can be effectively increased, and it can fully cope with scenarios where multiple people interact with VR at the same time. An isolation layer is used to form a closed space, isolating terahertz signals outside the closed space. The central device and wearable device are located inside the closed space. The first and second infrared sensors are both used to acquire the user's heat distribution information and transmit it to the central device in the form of terahertz signals. The first and second image sensors are both used to acquire the user's image information and transmit it to the central device in the form of terahertz signals. The first and second terahertz sensors are both used to acquire detailed user information and transmit it to the central device in the form of terahertz signals. Terahertz communication can achieve very high data transmission rates, utilize a wider spectrum bandwidth, and support larger capacity data transmission. For image and video transmission, using terahertz communication for VR devices can enable timely communication between the central device and wearable devices, achieving real-time interaction of multi-user information and effectively ensuring signal security for both the central device and wearable devices. The central device is used to acquire the user's heat distribution information through a first infrared sensor and a second infrared sensor, and to acquire the user's image information through a first image sensor and a second image sensor. It is also used to acquire the user's detailed information through a first terahertz sensor and a second terahertz sensor. Furthermore, the central device is used to determine the user's location information based on the heat distribution information and the user's image information. Finally, the central device is used to fill in the user's image based on the user's location information, user image information, and user detailed information. By equipping the wearable device or the central device with infrared sensors for acquiring heat distribution, terahertz sensors for acquiring object details, and image sensors for acquiring object images, information about the person can be effectively acquired. Based on this information, the user image is filled in, ensuring the integrity of the user image and providing convenience for improving visual effects and expanding applications. Attached Figure Description

[0055] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0056] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This is a schematic diagram of the user image filling system in an embodiment of this application;

[0058] Figure 2 This is a schematic diagram of the central device according to an embodiment of this application;

[0059] Figure 3 This is a schematic diagram of a wearable device according to an embodiment of this application;

[0060] Figure 4 This is a schematic diagram of object occlusion in a user image filling system according to an embodiment of this application;

[0061] Figure 5 This is a schematic diagram of a wearable charging device including a first power generation device, as described in an embodiment of this application.

[0062] Figure 6 This is a schematic diagram of a wearable charging device including a second power generation device, as described in an embodiment of this application.

[0063] Figure 7 This is a flowchart of the user image filling method in the embodiments of this application.

[0064] Explanation of icon numbers:

[0065] 1. Central equipment; 11. First infrared sensor; 12. First image sensor; 13. First terahertz sensor; 2. Wearable device; 21. Second infrared sensor; 22. Second image sensor; 23. Second terahertz sensor; 31. First power generation equipment; 32. Wireless signal transmission equipment; 33. Second power generation equipment.

[0066] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0067] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0068] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0069] Current VR technology focuses on single-user use and experience, with limited success in multi-user connectivity. This is because traditional communication methods may not meet the demands of large-scale multi-user connections, as VR technology requires the real-time transmission of massive amounts of image, sound, and interactive data. Besides communication needs, security is also a crucial consideration for multi-user VR use. As the demand for multi-user connectivity increases, ensuring user data and privacy becomes paramount. Intrusion into games via external connections could lead to the leakage of personal information, disruption of gameplay, or even sabotage. Security and privacy are especially critical during business meetings, particularly when dealing with confidential and sensitive information. Furthermore, multi-user VR technology requires isolating the user from the environment to create a virtual environment for interaction; current research in this area is lacking. Additionally, when users are obstructed or in dim lighting, the character's image is not fully displayed, severely impacting the user experience.

[0070] Based on this, embodiments of this application provide a user image filling system, referring to... Figure 1 , Figure 1 This is a schematic diagram of the user image filling system in the first embodiment of this application.

[0071] The user image filling system includes: a central device 1, a wearable device 2, and an isolation layer 3, such as... Figure 2 As shown, the central device 1 includes a first infrared sensor 11, a first image sensor 12, and a first terahertz sensor 13, as follows: Figure 3As shown, the wearable device includes a second infrared sensor 21, a second image sensor 22, and a second terahertz sensor 23; an isolation layer 3 is used to form a closed space to isolate terahertz signals outside the closed space, and the central device and the wearable device are disposed within the closed space; the first infrared sensor 11 and the second infrared sensor 21 are both used to acquire the user's heat distribution information and transmit the heat distribution information to the central device 1 in the form of terahertz signals; the first image sensor 12 and the second image sensor 22 are both used to acquire user image information and transmit the user image information to the central device 1 in the form of terahertz signals; the first terahertz sensor 13 and the second terahertz sensor 23... 3. All are used to acquire detailed user information and transmit the detailed information to the central device 1 in the form of terahertz signals; the central device 1 is used to acquire the user's heat distribution information through the first infrared sensor 11 and the second infrared sensor 21, and to acquire the user's image information through the first image sensor 12 and the second image sensor 22, and is also used to acquire the user's detailed information through the first terahertz sensor 13 and the second terahertz sensor 23; the central device 1 is also used to determine the user's location information based on the heat distribution information and the user's image information; the central device 1 is also used to fill the user image based on the user's location information, the user's image information and the user's detailed information.

[0072] For example, central device 1 includes a processor and storage device, and central device 1 and wearable device 2 are wirelessly connected via terahertz communication. Central device 1 has strong computing power and generally includes a high-performance graphics card, making it a miniature supercomputer. Wearable device 2 is worn on the head to place a display in front of the user's eyes. Wearable device 2 is typically equipped with a high-resolution display that projects virtual scenes in front of the user's eyes, allowing the user to immerse themselves in the virtual reality environment. The display is usually single or binocular, providing stereoscopic visual effects. To make the user feel truly immersed in the virtual environment, wearable device 2 is typically equipped with a tracking system that tracks the user's head movements and adjusts the perspective of the virtual scene accordingly, allowing the user to naturally observe and explore the virtual environment. Wearable device 2 is also typically equipped with an audio system, including headphones or built-in speakers, to provide surround sound and stereo effects, enhancing the user's immersion and experience. Furthermore, wearable device 2 is equipped with an interactive controller, allowing users to perform operations in the virtual environment through hand gestures, such as selecting, moving, and manipulating objects, enhancing the user's interactivity with the virtual environment. Wearable device 2 typically has a connection interface, allowing it to be wired to devices such as computers, game consoles, or smartphones to access virtual reality applications or game content. The isolation layer 3 is one or more of a polar material or a metallic material. For example, the isolation layer 3 can be a metal wall, a metal shielding box, or a specially designed polar material wall surface, such as a coating on walls, ceilings, and floors, or even some sheet materials. The material of the isolation layer 3 is a ferroelectric heterojunction or multiferroic heterojunction material. Ferroelectric heterojunction or multiferroic heterojunction materials are special functional materials that exhibit polarity reversal under the influence of an external electric or magnetic field. By fabricating the isolation layer 3 from ferroelectric heterojunction or multiferroic heterojunction materials, terahertz signals can be blocked. These materials undergo polarity reversal under the influence of an external electric or magnetic field, thereby changing their electromagnetic properties and preventing the propagation of terahertz signals. Ferroelectric heterojunction or multiferroic heterojunction materials not only have the function of shielding terahertz signals but may also have other functions, such as tunable electromagnetic properties, polymorphism, and charge control. Therefore, using these materials as isolation layer 3 can not only achieve signal blocking, but also provide other functions for the system. Furthermore, the heterojunction is very thin, allowing for coverage without causing other impacts.

[0073] It should be noted that the first infrared sensor 11 and the second infrared sensor 21 can detect the heat distribution on the surface of an object, thereby determining the object's outline and shape. The first terahertz sensor 13 and the second terahertz sensor 23 are both used to acquire detailed information about the user. It can be understood that, in addition to detecting detailed information about a person, the first terahertz sensor 13 and the second terahertz sensor 23 can also detect the fine structure and material properties of objects, such as surface morphology, thickness, and tissue structure. Terahertz waves have a wide frequency range, and terahertz communication and terahertz sensors can operate within different frequency ranges. By allocating communication devices and sensors to different frequency bandwidths, signal differentiation can be achieved. Furthermore, different encoding and modulation techniques can be used during data transmission to make the communication signal and sensor signal clearly distinguishable in the frequency domain or time domain. The first image sensor 12 and the second image sensor 22 can capture the appearance and color of objects, providing more intuitive visual information by capturing images of the object.

[0074] In this embodiment, the combined use of multiple infrared sensors, multiple terahertz sensors, and multiple image sensors allows for the acquisition of various types of information, including heat distribution, object details, and appearance features, achieving the fusion of multi-source information. The complementary information provided by different sensors enables comprehensive analysis of the object, allowing for in-depth research from aspects such as heat, structure, and appearance, providing more comprehensive data support for application scenarios. Furthermore, because appropriate sensor combinations can be selected as needed, and the system can be configured and adjusted according to the specific application requirements, it possesses a high degree of flexibility and customization.

[0075] like Figure 4 As shown, the character's image cannot be fully displayed when the user is obstructed or the lighting is dim.

[0076] Therefore, a wireless connection is required via terahertz communication. Based on this, a second embodiment of the user image filling system of this application is proposed.

[0077] In this embodiment, the central device 1 further includes a first transmitting component and a first receiving component, and the wearable device 2 further includes a second transmitting component and a second receiving component. The first transmitting component is used to transmit the terahertz signal to the wearable device 2, the first receiving component is used to receive the terahertz signal transmitted by the wearable device 2, the second transmitting component is used to transmit the terahertz signal to the central device 1, and the second receiving component is used to receive the terahertz signal transmitted by the central device 1.

[0078] For example, central device 1 and wearable device 2 are wirelessly connected via terahertz communication. Wearable device 2 includes a component for transmitting terahertz signals and a component for receiving terahertz signals. The transmitting component is responsible for converting data into terahertz signals and transmitting them, while the receiving component is responsible for receiving terahertz signals from central device 1 and converting them into usable data. Central device 1 also has components for transmitting and receiving terahertz signals, similar to wearable device 2. The transmission of terahertz signals is mainly achieved using terahertz antennas, such as terahertz sources based on Schottky diodes, terahertz sources based on Gunn diodes and IMPAT, terahertz sources based on QCL, and terahertz direct oscillation sources based on RTDs, etc. Central device 1 transmits terahertz signals and receives terahertz signals from wearable device 2, realizing bidirectional communication with wearable device 2. A terahertz modem is used to modulate and demodulate terahertz signals, converting digital data into terahertz signals for transmission, or converting received terahertz signals back into digital data for processing. These devices typically include modules such as modulators, demodulators, and digital signal processors.

[0079] In this embodiment, terahertz communication, a technology utilizing the terahertz band for data transmission, possesses high transmission rates and moderate penetration capabilities, making it highly promising for short-range communication and high-speed data transmission. Terahertz communication offers high transmission rates, enabling rapid data transmission and making it suitable for real-time data transmission and high-bandwidth scenarios. Its relatively low power consumption helps extend device battery life and improve endurance. The high frequency of the terahertz band reduces the risk of signal eavesdropping and interference, enhancing communication security. Furthermore, the terahertz band's penetration capability allows it to penetrate some common non-metallic materials, facilitating reliable connections between devices and maintaining stable communication even in the presence of obstacles. Terahertz communication enables communication between the central device and wearable devices, effectively ensuring timely communication for multiple users and avoiding the restriction of user freedom of movement caused by wired connections.

[0080] Based on the above embodiments, a third embodiment of the user image filling system of this application is proposed.

[0081] In this embodiment, the user image filling system further includes a wireless signal transmitting device 32; the wearable device 2 further includes a first power generating device 31, a second power generating device 33, and an energy receiving module. The wireless signal transmitting device 32 is used to send a wireless charging signal to the first power generating device 31. The first power generating device 31 is used to convert the generated wireless charging signal into electrical energy and send the electrical energy to the energy receiving module. The second power generating device 33 is used to convert the generated mechanical energy into electrical energy and send the electrical energy to the energy receiving module. The energy receiving module is used to receive the electrical energy and charge the wearable device 2 based on the electrical energy.

[0082] For example, such as Figure 5 As shown, wearable device 2 is connected to a first power generation device 31 that can be placed under the user's foot. A pressure sensor is located on the ground, and wireless charging is triggered by the pressure sensor. The first power generation device 31 is connected to wearable device 2 and can be placed under the foot after being worn. It contains a wireless charging circuit and an antenna, which is used to receive wireless charging signals from the ground and convert them into electrical energy to charge wearable device 2. When a person steps on the ground, the pressure sensor is triggered and sends a signal. The system automatically activates the wireless signal transmitting device 32 on the ground to send a wireless charging signal to wearable device 2. The energy receiving module in wearable device 2 receives electrical energy to charge wearable device 2.

[0083] For example, such as Figure 6 As shown, wearable device 2 is connected to a second power generation device 33. The second power generation device 33 is designed to be integrated into wearable device 2, such as in the sole, belt, or arm portion of wearable device 2. These devices utilize the mechanical energy generated by walking or exercise to convert it into electrical energy, and store the electrical energy in the device via a connecting wire or a built-in battery. When the user wears device 2 while walking or exercising, the walking power generation device generates mechanical energy and converts it into electrical energy to charge wearable device 2. In this way, the user can charge wearable device 2 during daily walking or exercise. The energy receiving module in wearable device 2 receives electrical energy to charge wearable device 2.

[0084] In this embodiment, since the charging device is installed on the ground, users do not need to carry an additional charger or search for a power outlet, making it very convenient and stress-free. When the user walks, the charging system automatically starts as soon as their foot touches the ground, requiring no manual operation and making the charging process more convenient. Charging the wearable device via a ground-based wireless signal transmitting device, a first power generation device, and a second power generation device provides a self-sufficient energy supply, reducing reliance on batteries and extending the device's lifespan.

[0085] Furthermore, this application proposes a fourth embodiment, which provides a user image filling method. The user image filling method includes an application in a user image filling system. The system includes: a central device, a wearable device, and an isolation layer. The central device includes a first infrared sensor, a first image sensor, and a first terahertz sensor. The wearable device includes a second infrared sensor, a second image sensor, and a second terahertz sensor. The isolation layer forms a closed space, isolating terahertz signals outside the closed space. The central device and the wearable device are disposed within the closed space. The first and second infrared sensors acquire the user's heat distribution information and transmit the heat distribution information to the central device in the form of terahertz signals. The first and second image sensors... The system acquires user image information and transmits it to a central device in the form of a terahertz signal; a first terahertz sensor and a second terahertz sensor acquire detailed user information and transmit it to the central device in the form of a terahertz signal; the central device acquires user heat distribution information through a first infrared sensor and a second infrared sensor, and acquires user image information through a first image sensor and a second image sensor, and is also used to acquire detailed user information through the first terahertz sensor and the second terahertz sensor; the central device also determines user location information based on the heat distribution information and user image information; the central device further fills in the user image based on the user location information, user image information, and user detailed information. For example, such as... Figure 7 As shown, Figure 7 A flowchart for filling user images.

[0086] It should be noted that the user's location information includes user category and user location coordinates. The step of determining the user's location information based on the heat distribution information and the user image information includes:

[0087] The current image is input into the user recognition model to obtain user feature information; the user category is determined based on the user feature information, and the corresponding user location coordinates are obtained based on the user category. Before the step of inputting the current image into the user recognition model, the following steps are taken: determining the user's center position coordinates and heat profile based on heat distribution information; determining the user's color information and brightness information based on user image information; and establishing a user recognition model based on the user's center position coordinates, heat profile, and color and brightness information.

[0088] For example, the specific implementation process and steps of the CPU of central device 1 using machine learning algorithms to accurately identify and locate objects are as follows:

[0089] Data Collection and Preparation: Collect a dataset containing infrared and visible light images of the object. This dataset is centered on the human object from an infrared sensor. The edge information obtained comes from image information such as color (P), brightness (W), and shape from the image sensor. Q is first obtained by the infrared sensor Get the center of the character object and thermal profile information Then the CPU determines the central... The image sensor is adjusted to capture images. The dataset is then labeled to indicate the location and attribute information of objects in each image.

[0090] The labeled dataset mentioned here refers to the collected infrared and image data that has been labeled to indicate different human subjects in each image, such as the thermal profile information of a human subject labeled k. Image information such as color (P), brightness (W), and shape from the image sensor. .

[0091] Feature extraction and preprocessing: The collected images are preprocessed, such as resizing and noise removal. Features are extracted from the images using various feature extraction methods, such as Histogram of Oriented Gradients (HOG), SIFT, and SURF.

[0092] Model Training: The preprocessed dataset is divided into training and test sets. The convolutional neural network model is trained using the training set to learn image features and the positional relationships of objects. During training, the model parameters are optimized using the backpropagation algorithm to improve the model's performance on the training set.

[0093] Model evaluation and tuning: The model's performance is evaluated using a test set. Common evaluation metrics include accuracy, recall, and F1 score. Based on the evaluation results, the model is tuned, which may involve adjusting model parameters, increasing training data, and modifying the model structure.

[0094] Object recognition and localization: When new image data is input, the trained model is used to identify and localize objects in the image. The model determines the category of the object based on image features and determines its location information in the image.

[0095] Output results: The output includes the results of object identification and localization, typically including information such as object category and location coordinates. These results will be used in subsequent steps, such as adjusting terahertz sensor parameters or filling in occluded parts of the object.

[0096] For example, the step of acquiring detailed user information through the first terahertz sensor and the second terahertz sensor includes recording the data in spherical coordinates, with the coordinates serving as labels to record the shape and texture of the occluded portion and the portion requiring detail, the data being a series of data points. The terahertz sensor transmits the acquired data to the CPU for further processing. After receiving the data from the terahertz sensor, the CPU of central device 1 uses image processing algorithms to enhance and optimize the object image. These image processing algorithms include, but are not limited to, edge enhancement, noise reduction, and color correction techniques to improve image quality and clarity. Because terahertz waves can penetrate non-metallic materials, they can easily penetrate obstacles such as tables and chairs to obtain the object's specific contour information. Terahertz waves can penetrate matter, so not only can the object's contour be obtained, but also internal information such as facial expressions can be acquired. This rich image information will help improve the accuracy and effectiveness of image processing. It should be noted that the image processed by the terahertz sensor and CPU contains enhanced and optimized object detail information. These processed images will be used in subsequent intelligent algorithm processing, such as filling in missing information in occluded parts of the object.

[0097] In one embodiment, the step of filling the user image based on the user's location information, user image information, and user details includes: establishing an image filling model based on the user's location information, user image information, and user details; inputting the damaged image into the image filling model to obtain the user-repaired image.

[0098] For example, training an object image inpainting model can include the following process:

[0099] Data preparation: Represent the image as a matrix. For color images, this can be represented as a three-dimensional matrix, where each channel represents the red, green, and blue (RGB) components. The image can be represented as: ,in, It is the image height. It is the image width. It refers to the number of channels.

[0100] Choosing an appropriate network architecture: The goal of Generative Adversarial Networks (GANs) is to enable the generator to... Generated and real images Images that are as close as possible, while discriminator The aim is to distinguish the generated image from the real image.

[0101] The goal of an autoencoder is to learn an encoder E to process an input image. The encoded Z is mapped to a low-dimensional space, and then the decoder D maps the encoded Z back to the reconstructed image. .

[0102] Conditional Generative Adversarial Networks (cGANs) add a condition (C) to Generative Adversarial Networks, allowing both the generator and discriminator to receive conditional information. This enables the generator to produce images under specific conditions.

[0103] Network training: During training, the difference between the generator G and the real image is minimized, while the discriminator D's ability to judge the generated image is also minimized. For GANs, the loss function typically includes the generator loss. and discriminator loss For autoencoders, the loss function typically includes reconstruction loss. For cGAN, conditional information is added to the loss function. The loss function can be expressed as:

[0104] Optimizer and Hyperparameter Selection: Commonly used optimizers include stochastic gradient descent (SGD) and the Adam optimizer. Hyperparameters include learning rate and batch size.

[0105] Training process monitoring and adjustment: Monitor how the model's loss function value changes with the number of training iterations. This can be visualized using tools such as TensorBoard.

[0106] Model evaluation and testing: The performance of the model is evaluated using a test dataset. The model's performance is quantified by calculating evaluation metrics (such as PSNR, SSIM, etc.) and a qualitative evaluation is performed.

[0107] Model deployment and application: The trained model is deployed to real-world applications to fill in the outlines and colors of occluded parts of objects. By inputting an image of a damaged or occluded part into the trained model, a repaired complete image can be generated.

[0108] In one embodiment, the central device further includes a first transmitting component and a first receiving component; the wearable device further includes a second transmitting component and a second receiving component; the first transmitting component transmits the terahertz signal to the wearable device; the first receiving component receives the terahertz signal transmitted by the wearable device; the second transmitting component transmits the terahertz signal to the central device; and the second receiving component receives the terahertz signal transmitted by the central device.

[0109] In one embodiment, the system further includes: a wireless signal transmitting device; the wearable device further includes: a first power generating device and a second power generating device, and an energy receiving module; the wireless signal transmitting device sends a wireless charging signal to the first power generating device; the first power generating device converts the generated wireless charging signal into electrical energy and sends the electrical energy to the energy receiving module; the second power generating device converts the generated mechanical energy into electrical energy and sends the electrical energy to the energy receiving module; the energy receiving module receives the electrical energy and charges the wearable device based on the electrical energy.

[0110] In one embodiment, the step of filling the user image based on the user's location information, user image information, and user details further includes: constructing a three-dimensional structure of the object based on three-dimensional structural information obtained from infrared sensors and terahertz sensors, capturing image data through multi-directional image sensors for texturing, obtaining a three-dimensional model of the object, and obtaining a new orientation view of the object through the three-dimensional model of the object.

[0111] For example, a specific process may include three steps: acquiring 3D structural information, capturing image data and textures, and acquiring a new orientational object view. Acquiring 3D structural information can be achieved using infrared and terahertz sensors. The infrared sensor captures the object's surface contours and temperature distribution, while the terahertz sensor captures detailed information. The data from the infrared and terahertz sensors are fused together, and a 3D model of the object is constructed using 3D reconstruction algorithms or point cloud processing techniques. Combining infrared, terahertz, and multi-directional image sensors allows for the acquisition of comprehensive 3D structural information and realistic textures, enhancing the object's realism. Capturing image data and textures involves using image sensors from multiple orientations to capture image data of the object, ensuring complete texture and detail information is obtained from different angles. The captured image data is applied to the 3D model for texture mapping, mapping the image textures onto the object's surface to enhance realism. Texture mapping technology is mature in mechanical drawing and can be well-suited for character modeling, enabling the rapid and efficient creation of highly realistic 3D character models in VR. The constructed 3D character models can be applied in metaverse and digital twin technologies to achieve richer functionality. New orientation object views can be easily generated from different orientations using the constructed 3D model of the object, without the need to recapture or process image data. These new orientation object views can be freely rotated and observed in 3D space as needed, providing a more flexible and comprehensive perspective. Through 3D reconstruction and texturing techniques, the 3D model of the object can be efficiently constructed and image textures applied, improving processing efficiency and accuracy. The acquisition of new orientation object views is flexible and diverse, allowing users to freely choose and observe the object from different angles as needed, enhancing visualization and interactivity.

[0112] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the user image filling method of this application. Any simple transformations based on this technical concept are within the protection scope of this application.

[0113] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the user image filling method described above.

[0114] The computer program product provided in this application can solve the technical problem of incomplete character image display. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the user image filling method provided in the above embodiments, and will not be repeated here.

[0115] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A user image filling system, characterized in that, The system includes: a central device, a wearable device, and an isolation layer. The central device includes a first infrared sensor, a first image sensor, and a first terahertz sensor. The wearable device includes a second infrared sensor, a second image sensor, and a second terahertz sensor. The isolation layer is used to form a closed space to isolate terahertz signals outside the closed space, and the central device and the wearable device are disposed within the closed space; Both the first infrared sensor and the second infrared sensor are used to acquire the user's heat distribution information and transmit the heat distribution information to the central device in the form of terahertz signals; Both the first image sensor and the second image sensor are used to acquire user image information and transmit the user image information to the central device in the form of terahertz signals; Both the first terahertz sensor and the second terahertz sensor are used to acquire detailed information about the user and transmit the detailed information to the central device in the form of terahertz signals. The central device is used to acquire the user's heat distribution information through the first infrared sensor and the second infrared sensor, and to acquire the user's image information through the first image sensor and the second image sensor. It is also used to acquire the user's detailed information through the first terahertz sensor and the second terahertz sensor. The central device is also used to determine the user's location information based on the heat distribution information and the user image information; The central device is also used to fill in the user image based on the user's location information, the user's image information, and the user's detailed information.

2. The user image filling system as described in claim 1, characterized in that, The central device further includes: a first transmitting component and a first receiving component; the wearable device further includes a second transmitting component and a second receiving component; The first transmitting component is used to transmit the terahertz signal to the wearable device; The first receiving component is used to receive terahertz signals emitted by the wearable device; The second transmitting component is used to transmit the terahertz signal to the central device; The second receiving component is used to receive terahertz signals emitted by the central device.

3. The user image filling system as described in claim 1, characterized in that, The system further includes: a wireless signal transmitting device; the wearable device further includes: a first power generating device, a second power generating device, and an energy receiving module; The wireless signal transmitting device is used to send a wireless charging signal to the first power generation device; The first power generation device is used to convert the generated wireless charging signal into electrical energy and send the electrical energy to the energy receiving module; The second power generation device is used to convert the generated mechanical energy into electrical energy and send the electrical energy to the energy receiving module; The energy receiving module is used to receive the electrical energy and charge the wearable device based on the electrical energy.

4. A user image filling method, characterized in that, The method is applied to a user image filling system, the system comprising: a central device, a wearable device, and an isolation layer; the central device comprising a first infrared sensor, a first image sensor, and a first terahertz sensor; and the wearable device comprising a second infrared sensor, a second image sensor, and a second terahertz sensor. The isolation layer forms a closed space, isolating terahertz signals outside the closed space, and the central device and the wearable device are located inside the closed space; The first infrared sensor and the second infrared sensor acquire the user's heat distribution information and transmit the heat distribution information to the central device in the form of a terahertz signal. The first image sensor and the second image sensor acquire user image information and transmit the user image information to the central device in the form of a terahertz signal; The first terahertz sensor and the second terahertz sensor acquire detailed information about the user and transmit the detailed information to the central device in the form of terahertz signals; The central device acquires the user's heat distribution information through the first infrared sensor and the second infrared sensor, and acquires the user's image information through the first image sensor and the second image sensor. It is also used to acquire the user's detailed information through the first terahertz sensor and the second terahertz sensor. The central device also determines the user's location information based on the heat distribution information and the user image information; The central device also fills in the user image based on the user's location information, the user's image information, and the user's detailed information.

5. The method as described in claim 4, characterized in that, The central device further includes: a first transmitting component and a first receiving component; the wearable device further includes a second transmitting component and a second receiving component; The first transmitting component sends the terahertz signal to the wearable device; The first receiving component receives a terahertz signal emitted by the wearable device; The second transmitting component sends the terahertz signal to the central device; The second receiving component receives terahertz signals emitted by the central device.

6. The method as described in claim 4, characterized in that, The system further includes: a wireless signal transmitting device; the wearable device further includes: a first power generating device, a second power generating device, and an energy receiving module; The wireless signal transmitting device sends a wireless charging signal to the first power generation device. The first power generation device converts the generated wireless charging signal into electrical energy and sends the electrical energy to the energy receiving module; The second power generation device converts the generated mechanical energy into electrical energy and sends the electrical energy to the energy receiving module. The energy receiving module receives the electrical energy and charges the wearable device based on the electrical energy.

7. The method as described in claim 4, characterized in that, The user's location information includes user category and user location coordinates. The step of determining the user's location information based on the heat distribution information and the user image information includes: Input the current image into the user recognition model to obtain user feature information; The user category is determined based on the user feature information, and the corresponding user location coordinates are obtained based on the user category.

8. The method as described in claim 7, characterized in that, Before the step of inputting the current image into the user recognition model, including The user's center location coordinates and heat profile are determined based on the heat distribution information; Determine the user's color and brightness information based on the user's image information; A user identification model is established based on the user's center position coordinates, thermal profile, color information, and brightness information.

9. The method as described in claim 4, characterized in that, The step of filling the user image based on the user's location information, the user's image information, and the user's detailed information includes: An image filling model is established based on the user's location information, the user's image information, and the user's detailed information. Input the damaged image into the image filling model to obtain the user-repaired image.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the user image filling method as described in any one of claims 4 to 9.

Images