Position information determination method and device, storage medium and electronic equipment

Abstract

The invention discloses a position information determination method and device, a storage medium and electronic equipment, and relates to the technical field of smart home, the position information determination method comprises the following steps: collecting first image information of a target area, and determining a plane of the target area according to the first image information; the plane and target equipment are displayed on the terminal equipment, and the target equipment exists on the plane; a first operation instruction of a target object is received, position information of the target device is determined according to the operation instruction, and the first operation instruction is used for adjusting the position of the target device. The problem that shopping experience is poor due to the fact that the placing effect of the equipment in the actual environment cannot be visually felt is solved.

Description

Technical Field

[0001] The present application relates to the field of smart home technology, and more specifically, to a method and device for determining location information, a storage medium, and an electronic device. Background Art

[0002] With the rapid development of e-commerce, more and more consumers choose to buy home appliances online. This trend has greatly facilitated shopping, but it has also brought new challenges. In the absence of a physical shopping environment, consumers cannot intuitively feel the actual placement of home appliances in their homes. Traditional online shopping platforms mainly display products through static or dynamic media such as product pictures, videos, and text descriptions. Although this method can provide basic information such as product appearance and size, it is difficult to accurately convey the integration effect of home appliances with actual living spaces, especially for the matching degree of size, color, style and interior decoration. Consumers can often only rely on imagination, which makes it impossible to accurately preview the effect of home appliances in the actual environment before purchasing.

[0003] With respect to the problem in the related art that it is impossible to accurately preview the effect of a device in an actual environment before purchasing, no effective solution has been proposed. Summary of the invention

[0004] The embodiments of the present application provide a method and device for determining location information, a storage medium, and an electronic device, so as to at least solve the problem in the related art that it is impossible to accurately preview the effect of the device in the actual environment before purchasing.

[0005] According to an embodiment of the present application, a method for determining position information is provided, including: acquiring first image information of a target area, and determining a plane of the target area based on the first image information; displaying the plane and the target device on the terminal device, wherein the target device exists on the plane; receiving a first operation instruction of the target object, and determining the position information of the target device based on the operation instruction, wherein the first operation instruction is used to adjust the position of the target device.

[0006] In an exemplary embodiment, the plane and the target device are displayed on the display interface of the terminal device through virtual reality technology, including: determining first size information of a free area of ​​the plane, wherein the first size information includes: length, width, and height; determining whether the plane meets the placement conditions of the target device according to the first size information of the free area of ​​the plane; and when it is determined that the plane meets the placement conditions of the target device, displaying the plane and the target device on the display interface of the terminal device through virtual reality technology.

[0007] In an exemplary embodiment, the plane and the target device are displayed on the display interface of the terminal device through virtual reality technology, including: determining a preset placement position of the target device; loading a three-dimensional model of the target device, and placing the three-dimensional model of the target device on the plane through virtual reality technology and the preset placement position.

[0008] In an exemplary embodiment, determining the plane of the target area based on the first image information includes: determining the current lighting conditions of the target area based on the first image information; adjusting the first image information through a lighting compensation algorithm and the current lighting conditions; and determining the plane of the target area based on the adjusted first image information.

[0009] In an exemplary embodiment, before determining the plane of the target area based on the first image information, the method also includes: determining feature points in the first image information; determining structural information formed by the feature points, and determining whether there is a plane in the target area based on the structural information; in a case where no plane exists in the target area, displaying notification information on the display interface, wherein the notification information is used to instruct the target object to move the terminal device; in a case where the terminal device moves, collecting second image information of the target area, and determining the plane of the target area based on the second image information.

[0010] In an exemplary embodiment, before displaying the plane and the target device on the display interface of the terminal device through virtual reality technology, the method further includes: establishing a device information library, wherein the device information library includes: device type, model information, and second size information of each device; determining first size information of the idle area of ​​the plane according to the first image information, and determining the device type of the target device; determining a first device set consistent with the device type of the target device in the device information library; and determining the target device in the first device set, wherein the second size information of the target device matches the first size information.

[0011] In an exemplary embodiment, after determining the location information of the target device according to the operation instruction, the method further includes: displaying confirmation information through the display interface of the terminal device, wherein the confirmation information is used to confirm whether the target object places the target device at the position corresponding to the location information; and upon receiving response information sent by the target object, sending a purchase link for the target device to the target object, wherein the response information is used to indicate that the target object confirms placing the target device at the position corresponding to the location information.

[0012] According to another embodiment of the embodiments of the present application, a device for determining position information is also provided, which is applied to a terminal device and includes: an acquisition module, used to acquire first image information of a target area, and determine the plane of the target area based on the first image information; a display module, used to display the plane and the target device on the terminal device, wherein the target device exists on the plane; a determination module, used to receive a first operation instruction of a target object, and determine the position information of the target device based on the operation instruction, wherein the first operation instruction is used to adjust the position of the target device.

[0013] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned method for determining the location information when running.

[0014] According to another aspect of an embodiment of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the method for determining location information through the computer program.

[0015] In an embodiment of the present application, first image information of a target area is collected, and the plane of the target area is determined based on the first image information; the plane and the target device are displayed on the terminal device, wherein the target device exists on the plane; a first operation instruction of a target object is received, and the position information of the target device is determined based on the operation instruction, wherein the first operation instruction is used to adjust the position of the target device; that is, in an embodiment of the present application, by collecting image information of the target area, the plane of the target area and the target device are presented in the display interface of the terminal device, so that the user can intuitively preview the placement effect of the device in the target area, and then determine the placement position information of the device through the operation instruction. Through the embodiment of the present application, the user can more accurately estimate the placement effect of the device, and the above technical solution is adopted to solve the problem of not being able to accurately preview the effect of the device in the actual environment before purchase. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0018] Figure 1 It is a schematic diagram of the hardware environment of a method for determining location information in an embodiment of the present application;

[0019] Figure 2 is a flowchart (I) of a method for determining location information according to an embodiment of the present application;

[0020] Figure 3 is a flowchart (II) of a method for determining location information according to an embodiment of the present application;

[0021] Figure 4 is a structural block diagram of a device for determining location information according to an embodiment of the present application (I);

[0022] Figure 5 This is a structural block diagram (II) of a device for determining location information according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] According to one aspect of an embodiment of the present application, a method for determining location information is provided. The method for determining location information is widely used in smart home, smart home, smart home device ecology, smart residential (Intelligence House) ecology and other whole-house intelligent digital control application scenarios. Optionally, in this embodiment, the above-mentioned method for determining location information can be applied to Figure 1In the hardware environment composed of the terminal device 102 and the server 104 shown in FIG. Figure 1 As shown, the server 104 is connected to the terminal device 102 via a network, and can be used to provide services (such as application services, etc.) for the terminal or a client installed on the terminal. A database can be set on the server or independently of the server to provide data storage services for the server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data computing services for the server 104.

[0026] The network may include but is not limited to at least one of the following: wired network, wireless network. The wired network may include but is not limited to at least one of the following: wide area network, metropolitan area network, local area network, and the wireless network may include but is not limited to at least one of the following: WIFI (Wireless Fidelity), Bluetooth. The terminal device 102 may be but is not limited to a PC, a mobile phone, a tablet computer, a smart air conditioner, a smart range hood, a smart refrigerator, a smart oven, a smart stove, a smart washing machine, a smart water heater, a smart washing device, a smart dishwasher, a smart projection device, a smart TV, a smart clothes drying rack, a smart curtain, a smart audio and video, a smart socket, a smart speaker, a smart fresh air device, a smart kitchen and bathroom device, a smart bathroom device, a smart sweeping robot, a smart window cleaning robot, a smart mopping robot, a smart air purification device, a smart steamer, a smart microwave oven, a smart kitchen treasure, a smart purifier, a smart water dispenser, a smart door lock, etc.

[0027] In this embodiment, a method for determining position information is provided, which is applied to a terminal device having a display function and a camera function. Figure 2 1 is a flow chart of a method for determining location information according to an embodiment of the present application (I), the flow comprising the following steps:

[0028] Step S202, collecting first image information of a target area, and determining a plane of the target area according to the first image information;

[0029] Step S202 captures an image of a target area (such as a user's living room, bedroom, etc.) through a camera or other image acquisition device to provide basic data for subsequent plane recognition.

[0030] Based on the first image information, the system uses image processing and computer vision technology to identify and determine the plane in the target area. Accurate plane identification is the basis for placing virtual objects in AR / VR applications, ensuring that virtual objects can be stably placed on the identified plane, such as the ground, desktop, wall, etc.

[0031] Step S204, displaying the plane and the target device on the terminal device, wherein the target device exists on the plane;

[0032] Based on step S202, the system uses virtual reality or augmented reality technology to present the identified plane on the display interface of the user's device (such as a smart phone, etc.). At the same time, the terminal device will superimpose the target device (such as virtual home appliances, furniture, etc.) on the displayed plane to achieve the integration of virtual and reality, allowing the user to intuitively see the placement effect of the target device in the actual environment.

[0033] The display position of the target device should match the plane position determined in step S202 to ensure that the user can accurately preview the position and spatial relationship of the device in the real environment.

[0034] Step S206: receiving a first operation instruction of the target object, and determining the position information of the target device according to the operation instruction, wherein the first operation instruction is used to adjust the position of the target device.

[0035] The device can receive user operation instructions for the target device. These instructions can be issued through gestures, touch screens, or voice, and are used to adjust the position and posture of the virtual device on the plane.

[0036] According to the user's first operation instruction, the system adjusts and updates the position information of the target device in the virtual environment in real time. The system should provide instant feedback so that users can see the real-time effect of the operation instruction on the position adjustment of the target device. At the same time, in order to improve the user experience, the system also has certain intelligent optimization capabilities, for example, automatically adjusting the distance between the device and the wall or other objects to ensure the rationality of the placement.

[0037] Through the above steps, the first image information of the target area is collected, and the plane of the target area is determined according to the first image information; the plane and the target device are displayed on the terminal device, wherein the target device exists on the plane; the first operation instruction of the target object is received, and the position information of the target device is determined according to the operation instruction, wherein the first operation instruction is used to adjust the position of the target device; that is, in the embodiment of the present application, by collecting the image information of the target area, the plane and the target device of the target area are presented in the display interface of the terminal device using virtual reality technology, so that the user can intuitively preview the placement effect of the device in the target area, and then determine the placement position information of the device through the operation instruction. Through the embodiment of the present application, the user can more accurately estimate the placement effect of the device, and the above technical solution is adopted to solve the problem of not being able to accurately preview the effect of the device in the actual environment before purchase.

[0038] Optionally, in order to better understand the above step S204, an implementation method is provided in an embodiment of the present application, as follows: determine the first size information of the idle area of ​​the plane, wherein the first size information includes: length, width and height; determine whether the plane meets the placement conditions of the target device according to the first size information of the idle area of ​​the plane; when it is determined that the plane meets the placement conditions of the target device, display the plane and the target device on the display interface of the terminal device through virtual reality technology.

[0039] In an embodiment of the present application, before using virtual reality technology to display a device, the size of the free area of ​​the target area plane is measured and evaluated to ensure that the virtual device can be reasonably and adaptively placed on the plane.

[0040] First, the system collects images of the target area through the camera of the terminal device (such as a smartphone, AR glasses, etc.), and then uses computer vision technology to identify planes in the area, such as floors, desktops, etc. Then, the identified planes are further analyzed to determine the size of their free areas, including length, width, and height. This step is completed through image processing and analysis algorithms, including but not limited to:

[0041] 1. Feature point detection: Identify the boundaries and internal features of the plane, such as corners, edges of obstacles, etc.

[0042] 2. Size measurement: Calculate the size information of the plane idle area based on feature points and known camera parameters.

[0043] 3. Supplementary height information: For some scenes, such as the placement of virtual decorations on the wall, the height of the wall must also be considered.

[0044] The system compares the measured size information with the size requirements of the target device to determine whether the target device can be placed in the free area. This determination process includes but is not limited to:

[0045] 1. Size comparison: Verify that the size of the target device (including installation space requirements) is less than or equal to the size of the free area.

[0046] 2. Installation condition assessment: Consider the specific installation requirements of the target equipment, such as the bearing capacity requirements for the wall and the flatness requirements for the ground.

[0047] 3. Visual effect evaluation: Evaluate whether the target device is beautiful and coordinated after placement, such as whether it conflicts with other furniture in the room.

[0048] After determining that the target device can be placed in an empty area of ​​a plane, it will use virtual reality technology to present an enhanced view of the plane in real time on the display interface of the terminal device, and display a virtual model of the target device in an appropriate position. This process includes but is not limited to:

[0049] 1. Location determination: Find an optimal placement in the free area, taking into account visual effects and actual usage needs.

[0050] 2. Visual fusion: Seamlessly integrate the virtual model of the target device with the image of the actual environment to ensure the realism and stability of the model.

[0051] 3. User interaction: Provide users with interactive methods to adjust the position and direction of virtual devices, such as gesture control, touch screen operation, etc.

[0052] For example, suppose a user wants to buy a new refrigerator at home. This method is used for virtual preview: the user opens the home appliance shopping app on the phone and aims the camera at the kitchen corner where the refrigerator is to be placed. The system identifies the plane of the kitchen floor through image analysis and measures the length, width and height of the free area in the corner (taking into account the distance between the top of the refrigerator and the kitchen ceiling). The system obtains the size information of the refrigerator, including height, width and depth, and then compares this information with the measured size of the free area in the kitchen corner. If the size of the refrigerator is smaller than the size of the free area, and the vertical height of the free area allows enough space between the top of the refrigerator and the ceiling, the system considers that the plane meets the conditions for placing the refrigerator. After confirming the placement conditions, the user sees a real-time image of the kitchen corner on the display interface of the APP, and the virtual model of the refrigerator is placed on the free area in the corner. The user can adjust the placement of the refrigerator by sliding the screen with gestures, and see the integration effect of the refrigerator with the surrounding environment in real time. The system also displays the actual size of the refrigerator, such as "height: 180cm, width: 70cm, depth: 65cm", to help users more intuitively evaluate whether the size of the refrigerator is appropriate.

[0053] Through the embodiments of the present application, users can see the virtual display effect of the refrigerator at home in advance, saying goodbye to the era of shopping based solely on product pictures and size parameters, and greatly improving the intuitiveness and accuracy of the shopping experience.

[0054] Optionally, the plane and the target device are displayed on the display interface of the terminal device through virtual reality technology, including: determining a preset placement position of the target device; loading a three-dimensional model of the target device, and placing the three-dimensional model of the target device on the plane through virtual reality technology and the preset placement position.

[0055] It should be noted that before displaying the device, the system needs to determine the preset placement of the target device in the target area. The system automatically analyzes the most suitable placement based on the image information of the target area. For example, for a refrigerator, the system may recommend placing it in the corner of the kitchen or on the wall near a power outlet.

[0056] In order to truly display the device in the virtual environment, the system needs to load the 3D model of the target device. The 3D model not only includes the geometric shape of the device, but may also include details such as material, texture, color, etc. to provide a more realistic visual effect. The loading of the 3D model includes: Model selection: Select a 3D model that matches the target device from the pre-stored model library. Model optimization: According to the performance of the terminal device, the model is appropriately optimized, such as reducing the complexity of the model to ensure a smooth display effect. Model import: Import the optimized 3D model into the virtual reality environment and prepare it for fusion with the real-time image.

[0057] In a virtual reality environment, the 3D model of the target device is placed in a preset position and fused with the real-time image of the target area to simulate the effect of the device in the actual environment. This process includes:

[0058] 1. Positioning: The system calculates the precise coordinates of the 3D model in the virtual environment based on the preset placement.

[0059] 2. Posture adjustment: Determine the tilt angle and direction of the 3D model when it is placed to ensure that it matches the plane of the actual environment.

[0060] 3. Visual fusion: Integrate the 3D model with the real-time image to ensure seamless connection between the model and the surrounding environment, thus improving the realism of the display.

[0061] 4. Interactive display: Users can interact with the placed 3D models through terminal devices, such as changing the position of the model, adjusting the viewing angle, etc., to gain a more comprehensive understanding of the placement effect of the device.

[0062] Optionally, determining the plane of the target area according to the first image information includes: determining the current lighting conditions of the target area according to the first image information; adjusting the first image information by an illumination compensation algorithm and the current lighting conditions; and determining the plane of the target area according to the adjusted first image information.

[0063] Among them, the illumination compensation algorithm is used to adjust the image information to reduce the negative impact of lighting conditions on plane recognition. The purpose of this process is to enhance the visibility and contrast of plane features in the image, ensuring that the system can accurately identify the plane even in an environment with insufficient or excessive lighting. The illumination compensation algorithm includes: Brightness adjustment: According to the light intensity, adjust the image brightness so that the overall brightness of the image is moderate and the plane details are clearly visible. Color correction: Based on color temperature analysis, correct the image color and restore the true color of the object. Shadow elimination or enhancement: Process the shadow area in the image, eliminate excessive shadow interference, or highlight the plane features by simulating enhanced shadows when the light is insufficient.

[0064] In the embodiment of the present application, image information is used to accurately identify the plane in the target area, while the influence of ambient lighting conditions on the recognition accuracy is taken into account, and the image information is optimized through a lighting compensation algorithm, thereby improving the accuracy of plane recognition. Specifically:

[0065] When the system acquires the first image information of the target area, it not only captures the geometric layout of the space, but also records the lighting conditions of the area. Lighting conditions include the position, intensity, color, and distribution of light in the space, which have a direct impact on image quality and subsequent plane recognition. The steps to determine lighting conditions include but are not limited to: Light source detection: Analyze the brightness and color distribution in the image to locate possible light source positions. Illuminance estimation: Calculate the average brightness of the image and estimate the light intensity of the target area. Color temperature analysis: Analyze the color distribution of the image and infer the color temperature of the light source.

[0066] The first image information is adjusted by the illumination compensation algorithm and the current illumination conditions. After the image information is adjusted by the illumination compensation algorithm, the system uses computer vision technology to identify the plane in the target area. This process includes: Feature point detection: Find feature points in the image, such as edges, corners, etc. Plane fitting: Based on the detected feature points, use mathematical algorithms (such as the RANSAC algorithm) to fit the plane and determine the tilt angle and position of the plane. Plane verification: Through further image analysis, verify the accuracy and stability of the fitted plane to ensure the subsequent virtual device display effect.

[0067] For example, suppose a user wants to use AR technology to preview the placement of a new TV in his living room in the evening. Due to insufficient natural light in the evening, there are shadow areas and dark colors in the image of the living room, which directly affects the accuracy of plane recognition. The system follows the following steps:

[0068] The system first analyzes the first image information collected from the living room and detects that the indoor light is dim and there is a color cast light source from the side windows. The system applies a lighting compensation algorithm to adjust the brightness of the image to make it moderate overall brightness; correct color shift to restore the true color of the living room walls and floors; and enhance the features of the shadow area so that it can still be clearly identified in low-light environments. Based on the adjusted image information, the system accurately identifies the floor plane and wall plane of the living room, including the obstructed parts of the furniture edges, so that the virtual model of the TV can be stably and accurately placed on the designated wall, and maintain a reasonable spatial relationship with the furniture on the ground.

[0069] Through the embodiments of the present application, even under poor lighting conditions, the system can provide high-quality plane recognition results, ensure the accuracy and realism of the virtual device display, and enhance the user experience of AR / VR shopping and decoration applications.

[0070] Optionally, before determining the plane of the target area according to the first image information, the method also includes: determining feature points in the first image information; determining structural information formed by the feature points, and determining whether there is a plane in the target area based on the structural information; in a case where no plane exists in the target area, displaying notification information on the display interface, wherein the notification information is used to instruct the target object to move the terminal device; in a case where the terminal device moves, collecting second image information of the target area, and determining the plane of the target area based on the second image information.

[0071] That is to say, before performing plane detection, the system first needs to extract feature points from the first image information. Feature points are key points in the image, such as edges, corners, textures, etc., which are crucial for subsequent plane recognition and positioning. Feature point extraction can be achieved in the following ways: Edge detection algorithm: such as Canny edge detection. Corner detection algorithm: such as Harris corner detection or FAST algorithm. Feature matching: such as SIFT or SURF algorithm.

[0072] The system determines whether there is a plane suitable for placing the device in the target area by analyzing the distribution and structure of the feature points. If the system fails to identify the plane in the first image information, it will display a notification message to the user on the display interface, instructing the user to move the terminal device to obtain new image information. When the user moves the terminal device according to the notification information, the system will collect new image information (second image information) and then repeat the above process of feature point extraction, structural information analysis and plane recognition. Through different perspectives or more image information, the system can more accurately identify the planes in the target area, providing a better environmental foundation for subsequent virtual device display.

[0073] Through the above process, even if the plane cannot be successfully identified in the first attempt, the system can obtain richer or more appropriate image information through the movement of the user's terminal device, improve the success rate of plane detection, and thus provide users with a high-quality virtual device display experience.

[0074] Optionally, before displaying the plane and the target device on the display interface of the terminal device through virtual reality technology, it also includes: establishing a device information library, wherein the device information library includes: the device type, model information, and second size information of each device; determining the first size information of the idle area of ​​the plane according to the first image information, and determining the device type of the target device; determining a first device set consistent with the device type of the target device in the device information library; determining the target device in the first device set, wherein the second size information of the target device matches the first size information.

[0075] In the embodiment of the present application, before using virtual reality technology to display the target device, the device information is prepared and screened, and the most appropriate device display is determined according to the actual size of the target area and the device requirements. Specifically:

[0076] Before displaying the equipment, the system needs to establish an equipment information library, which contains detailed information of various equipment. The data items in the information library may include: Equipment type: such as refrigerator, TV, sofa, etc., used to quickly classify and retrieve equipment information. Model information: Specific model name, such as "Haier XXXX" refrigerator, used to provide detailed specifications and functional descriptions of the equipment. Second dimension information: The three-dimensional dimensions of the equipment, including length, width, height, and possible installation space requirements, are the key basis for plane dimension matching.

[0077] After the user uses a terminal device (such as a smart phone) to aim at the target area and capture the first image information, the system needs to analyze the image and determine the size of the free area on the plane in the target area that can be used to place the device. The system needs to identify the type of device the user wants to preview. Based on the device type selected by the user, the system filters out all devices of this type from the device information library to form a first device set. This set contains device information of various models and sizes. The system will further filter from the first device set to determine the device that matches the size of the free area in the target area. This matching process includes: Size comparison: Compare the second size information of the device with the first size information of the free area to ensure that the device size is appropriate and can be placed in the target area. Placement condition verification: In addition to size matching, it is also necessary to verify the placement conditions of the device, such as whether the installation space is sufficient and whether the device is coordinated with the surrounding furniture.

[0078] For example, if you want to preview how a new refrigerator will look in your kitchen:

[0079] 1. Equipment information database establishment: The system maintains a database containing detailed information of various household appliances, including the type, model and size of refrigerators, washing machines, air conditioners and other equipment.

[0080] 2. Plane dimension measurement: The user uses a mobile phone to take a picture of a corner of the kitchen. The system identifies the plane of the kitchen floor through image analysis and measures the dimension information of the area, such as 150cm long, 80cm wide and 70cm high.

[0081] 3. Device type confirmation: The user selects "refrigerator" as the target device type for preview in the app.

[0082] 4. Equipment collection screening: The system screens out all "refrigerator" type equipment from the equipment information library to form a refrigerator equipment collection.

[0083] 5. Size-matched device display: The system compares the size of each refrigerator in the refrigerator device collection with the size of its preset free space in the kitchen. Finally, the system determines a refrigerator with a length of 140 cm, a width of 70 cm, and a height of 160 cm, which matches the size of the free space in the kitchen. In the virtual reality view seen by the user on the mobile phone, this refrigerator is placed in the preset kitchen corner, and the user can intuitively evaluate whether the refrigerator is of the right size, whether the appearance is coordinated, and whether it matches other furniture and decorations in the kitchen.

[0084] Through the embodiments of the present application, users can obtain a highly personalized virtual device display experience and accurately preview the placement effect of the device in the actual environment, so as to make more informed purchasing decisions, avoid problems such as size mismatch or style conflict, reduce the return rate after purchase, and improve shopping efficiency and satisfaction.

[0085] Optionally, after determining the location information of the target device according to the operation instruction, it also includes: displaying confirmation information through the display interface of the terminal device, wherein the confirmation information is used to confirm whether the target object places the target device at the position corresponding to the location information; upon receiving response information sent by the target object, sending a purchase link of the target device to the target object, wherein the response information is used to indicate that the target object confirms placing the target device at the position corresponding to the location information.

[0086] After the user previews and adjusts the position of the target device in the target area through operation instructions, the system needs to confirm the user's satisfaction with this position. The system displays confirmation information to the user through the display interface of the terminal device (such as a smartphone, tablet or AR glasses). The confirmation information may include the following aspects: Device position preview: Display a 3D preview of the target device in the target area, including details such as the size, color, and material of the device, to ensure that the user has an intuitive understanding of the appearance and placement of the device. Location information summary: Provide a concise location information summary, such as "The refrigerator is placed on the left side of the kitchen, 1 meter above the ground", to help users quickly confirm the placement of the device. Confirmation prompt: Ask the user whether they are satisfied with the current placement of the device. The user can respond with a simple operation (such as clicking the "Confirm" button).

[0087] If the user confirms that he is satisfied with the placement of the target device through the response information, the system will send the user a purchase link for the target device. The purchase link can directly guide the user to the product details page or provide a quick purchase option to simplify the user's purchase process.

[0088] For example, a user uses an AR shopping app to preview how a new washing machine will look in a laundry room:

[0089] The user places the virtual model of the washing machine on one side of the laundry room, at an appropriate height from the ground, matching the actual size of the washing machine, through gestures or touch operations. The system displays a confirmation message on the phone screen, such as "You have placed the washing machine on the left side of the laundry room, 90 cm above the ground. Are you satisfied with this position?" The user can choose "Confirm" or "Readjust". If the user clicks "Confirm", the system will consider that the user is satisfied with the preview position of the washing machine. The system generates a purchase link for the washing machine and sends it to the user through in-app messages, emails, or text messages. Users can directly access the online purchase page of the washing machine through the link to quickly complete the purchase process.

[0090] Through the embodiments of the present application, users can conveniently confirm the preview position of the device and make purchases directly from the application while enjoying a highly immersive virtual reality shopping experience, which greatly improves shopping efficiency and satisfaction.

[0091] In order to better understand the process of the method for determining the above-mentioned location information, the implementation method flow of the determination of the above-mentioned location information is described below in combination with an optional embodiment, but it is not used to limit the technical solution of the embodiment of the present application.

[0092] In this embodiment, a method for determining location information is provided. Figure 3 is a flow chart (II) of a method for determining location information according to an embodiment of the present application, such as Figure 3As shown, the specific steps are as follows:

[0093] Step S301: Load the three-dimensional model of the home appliance in detail in the APP.

[0094] Step S302: Activate the real-time image acquisition function to capture the dynamic image stream of the user environment through the APP.

[0095] Step S303: The system intelligently analyzes the current image to determine whether there is a horizontal surface suitable for placing home appliances. If an available surface is detected (True), the process automatically advances to S305. If no available surface is detected (False), step S304 is executed.

[0096] Step S304: The APP provides instant feedback and guides the user to adjust the angle or position of the phone until a suitable plane is captured;

[0097] Step S305: Using AR technology, the system accurately identifies all potential horizontal and vertical surfaces in the environment.

[0098] Step S306: Display the three-dimensional model of the home appliance on the selected plane through AR technology.

[0099] Step S307: With the help of gesture control, the user slides and rotates the virtual home appliance.

[0100] Step S308: The system calculates and displays the actual size of the device in real time according to the position of the home appliance adjusted by the user.

[0101] Step S309: On the APP interface, the three-dimensional model of the home appliance and its corresponding size information are synchronously presented.

[0102] Through the embodiments of the present application, not only the clarity of the technical implementation is enhanced, but also the smoothness and interactivity of the user experience are highlighted, ensuring that the entire process from model loading to size display is efficient and user-friendly, thereby enhancing the practical value and attractiveness of AR-based home appliance shopping and decoration application solutions.

[0103] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0104] Figure 4 is a structural block diagram of a device for determining location information according to an embodiment of the present application (I); Figure 4 As shown, including:

[0105] An acquisition module 42, configured to acquire first image information of a target area and determine a plane of the target area according to the first image information;

[0106] A display module 44, configured to display the plane and the target device on the terminal device, wherein the target device exists on the plane;

[0107] The determination module 46 is configured to receive a first operation instruction of a target object, and determine the position information of the target device according to the operation instruction, wherein the first operation instruction is used to adjust the position of the target device.

[0108] Through the above-mentioned device, the first image information of the target area is collected, and the plane of the target area is determined according to the first image information; the plane and the target device are displayed on the terminal device, wherein the target device exists on the plane; the first operation instruction of the target object is received, and the position information of the target device is determined according to the operation instruction, wherein the first operation instruction is used to adjust the position of the target device; that is, in the embodiment of the present application, by collecting the image information of the target area, the plane and the target device of the target area are presented in the display interface of the terminal device using virtual reality technology, so that the user can intuitively preview the placement effect of the device in the target area, and then determine the placement position information of the device through the operation instruction. Through the embodiment of the present application, the user can more accurately estimate the placement effect of the device, and the above-mentioned technical solution is adopted to solve the problem of not being able to accurately preview the effect of the device in the actual environment before purchase.

[0109] In an exemplary embodiment, the display module 44 is used to determine first size information of the free area of ​​the plane, wherein the first size information includes: length, width and height; determine whether the plane meets the placement conditions of the target device according to the first size information of the free area of ​​the plane; if it is determined that the plane meets the placement conditions of the target device, display the plane and the target device on the display interface of the terminal device through virtual reality technology.

[0110] In an exemplary embodiment, the display module 44 is used to determine a preset placement position of the target device; load a three-dimensional model of the target device, and place the three-dimensional model of the target device on the plane through virtual reality technology and the preset placement position.

[0111] In an exemplary embodiment, the acquisition module 42 is used to determine the current lighting conditions of the target area based on the first image information; adjust the first image information through a lighting compensation algorithm and the current lighting conditions; and determine the plane of the target area based on the adjusted first image information.

[0112] In an exemplary embodiment, a determination module 46 is used to determine feature points in the first image information; determine structural information formed by the feature points, and determine whether there is a plane in the target area based on the structural information; a display module 44 is used to display notification information on the display interface when no plane exists in the target area, wherein the notification information is used to instruct the target object to move the terminal device; and an acquisition module 42 is used to acquire second image information of the target area when the terminal device moves, and determine the plane of the target area based on the second image information.

[0113] In an exemplary embodiment, Figure 5 As shown, the above-mentioned device also includes, an establishing module 52, used for establishing a device information library, wherein the device information library includes: device type, model information, and second size information of each device; a determining module 46, used for determining the first size information of the idle area of ​​the plane according to the first image information, and determining the device type of the target device; determining a first device set consistent with the device type of the target device in the device information library; determining the target device in the first device set, wherein the second size information of the target device matches the first size information.

[0114] In an exemplary embodiment, the display module 44 is used to display confirmation information through the display interface of the terminal device, wherein the confirmation information is used to confirm whether the target object places the target device at the position corresponding to the location information; upon receiving response information sent by the target object, a purchase link for the target device is sent to the target object, wherein the response information is used to indicate that the target object confirms placing the target device at the position corresponding to the location information.

[0115] An embodiment of the present application further provides a storage medium, which includes a stored program, wherein the program executes any of the above methods when it is run.

[0116] Optionally, in this embodiment, the storage medium may be configured to store program codes for executing the following steps:

[0117] S1, collecting first image information of a target area, and determining a plane of the target area according to the first image information;

[0118] S2, displaying the plane and the target device on the terminal device, wherein the target device exists on the plane;

[0119] S3, receiving a first operation instruction of the target object, and determining the position information of the target device according to the operation instruction, wherein the first operation instruction is used to adjust the position of the target device.

[0120] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0121] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0122] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0123] S1, collecting first image information of a target area, and determining a plane of the target area according to the first image information;

[0124] S2, displaying the plane and the target device on the terminal device, wherein the target device exists on the plane;

[0125] S3, receiving a first operation instruction of the target object, and determining the position information of the target device according to the operation instruction, wherein the first operation instruction is used to adjust the position of the target device.

[0126] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store program codes.

[0127] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above method embodiments are implemented.

[0128] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0129] The embodiments of the present application also provide a computer program, which includes computer instructions, which are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in any one of the above method embodiments.

[0130] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0131] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0132] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for determining location information, characterized in that: Applied to terminal equipment, including: Collecting first image information of a target area, and determining a plane of the target area according to the first image information; Displaying the plane and the target device on the terminal device, wherein the target device exists on the plane; A first operation instruction of a target object is received, and location information of the target device is determined according to the operation instruction, wherein the first operation instruction is used to adjust the location of the target device.

2. The method for determining location information according to claim 1, characterized in that: Displaying the plane and the target device on the display interface of the terminal device by using virtual reality technology includes: Determine first size information of the idle area of ​​the plane, wherein the first size information includes: length, width and height; Determining whether the plane meets the placement condition of the target device according to the first size information of the free area of ​​the plane; When it is determined that the plane meets the placement condition of the target device, the plane and the target device are displayed on the display interface of the terminal device through virtual reality technology.

3. The method for determining location information according to claim 1, characterized in that: Displaying the plane and the target device on the display interface of the terminal device by using virtual reality technology includes: Determining a preset placement position of the target device; The three-dimensional model of the target device is loaded, and the three-dimensional model of the target device is placed on the plane through virtual reality technology and the preset placement position.

4. The method for determining location information according to claim 1, characterized in that: Determining the plane of the target area according to the first image information includes: determining a current lighting condition of the target area according to the first image information; adjusting the first image information by using an illumination compensation algorithm and the current illumination condition; The plane of the target area is determined according to the adjusted first image information.

5. The method for determining location information according to claim 1, characterized in that: Before determining the plane of the target area according to the first image information, the method further includes: Determining feature points in the first image information; Determining structural information formed by the feature points, and determining whether there is a plane in the target area according to the structural information; In the case where there is no plane in the target area, displaying notification information on the display interface, wherein the notification information is used to instruct the target object to move the terminal device; When the terminal device moves, second image information of the target area is collected, and the plane of the target area is determined according to the second image information.

6. The method for determining location information according to claim 1, characterized in that: Before displaying the plane and the target device on the display interface of the terminal device through virtual reality technology, the method further includes: establishing a device information library, wherein the device information library includes: device type, model information, and second size information of each device; Determine first size information of a free area of ​​the plane according to the first image information, and determine a device type of the target device; Determining in the device information database a first device set that is consistent with the device type of the target device; The target device is determined in the first device set, wherein the second size information of the target device matches the first size information.

7. The method for determining location information according to claim 1, characterized in that: After determining the location information of the target device according to the operation instruction, the method further includes: Displaying confirmation information through a display interface of the terminal device, wherein the confirmation information is used to confirm whether the target object places the target device at a position corresponding to the position information; In case of receiving the response information sent by the target object, sending the purchase link of the target device to the target object, wherein the response information is used to indicate that the target object confirms to place the target device at the location corresponding to the location information.

8. A device for determining position information, characterized in that: Applied to terminal equipment, including: An acquisition module, used for acquiring first image information of a target area, and determining a plane of the target area according to the first image information; A display module, configured to display the plane and the target device on the terminal device, wherein the target device exists on the plane; The determination module is used to receive a first operation instruction of a target object, and determine the position information of the target device according to the operation instruction, wherein the first operation instruction is used to adjust the position of the target device.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 7 when executed.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 7 through the computer program.

Images