A method and device for remotely viewing a vehicle based on augmented reality

Abstract

The embodiment of the present invention provides a method and device for remotely viewing a vehicle based on augmented reality, which relates to the field of the Internet. The method includes: an on-site device collects a world environment data stream, generates a first augmented reality data stream based on the world environment data stream, and sends the first augmented reality data stream to the remote device, so that the remote device generates a user collaboration data stream based on the first augmented reality data stream; receives the user collaboration data stream, then performs a corresponding action based on the user collaboration data stream, and collects a second augmented reality data stream when performing the corresponding action on the vehicle, and sends the second augmented reality data stream to the remote device. In this way, users can not only remotely view real vehicles instead of virtual vehicle models, but also perform real operations on the vehicles, which improves user experience. Moreover, the entire process does not require human participation, which greatly saves operating costs.

Description

Technical Field

[0001] The present invention relates to the field of Internet technology, and in particular to a method for remotely viewing a vehicle based on augmented reality and a device for remotely viewing a vehicle based on augmented reality. Background Art

[0002] Vehicle products cannot be separated from display. On average, consumers have started their car-buying journey three months before signing the car purchase contract. Combined with the consumption habits of contemporary vehicle consumers, online digital consumption scenarios run through the entire vehicle consumption customer journey.

[0003] One of the existing ways to view cars from a distance is AR (Augmented Reality). Specifically, open the AR camera and find a plane in the real world to prevent the pre-designed 3D model of the vehicle from being displayed. Figure 1 In addition, some event interactions are added to the 3D model, such as clicking the door of the AR vehicle model to open the door, and clicking the trunk to open the trunk of the model vehicle, as shown in the figure. Figure 2 shown.

[0004] However, the essence of this solution is just to display a 3D model, and the user experience is not good.

[0005] Another way to view cars remotely is to view them live. A video connection is established between the remote and on-site locations, and remote users can see the real vehicles on-site through real-time video.

[0006] However, this solution requires the anchor to take the lead and move the video camera along the route designed by the anchor to show the remote user the on-site vehicle. This method has high labor costs and relies on the on-site operation of the anchor. It is difficult to meet the personalized car viewing needs of different users, such as dynamically changing the camera according to the needs of remote users. Summary of the invention

[0007] In view of the above problems, embodiments of the present invention are proposed to provide a method for remotely viewing a vehicle based on augmented reality and a corresponding device for remotely viewing a vehicle based on augmented reality that overcome the above problems or at least partially solve the above problems.

[0008] In order to solve the above problems, an embodiment of the present invention discloses a method for remotely viewing a vehicle based on augmented reality, the method comprising:

[0009] Collecting a world environment data stream, and generating a first augmented reality data stream based on the world environment data stream;

[0010] Sending the first augmented reality data stream to the remote device so that the remote device generates a user collaboration data stream based on the first augmented reality data stream;

[0011] Receiving the user collaboration data stream;

[0012] performing a corresponding action based on the user collaboration data stream, and collecting a second augmented reality data stream when the corresponding action is performed on the vehicle;

[0013] The second augmented reality data stream is sent to the remote device.

[0014] In one or more embodiments, performing a corresponding action based on the user collaboration data stream and collecting a second augmented reality data stream when the corresponding action is performed on the vehicle includes:

[0015] identifying action parameters in the user collaboration data stream;

[0016] If the action parameter is opening / closing a door, opening / closing the corresponding door according to the door identifier in the action parameter value in the user collaboration data stream;

[0017] The augmented reality data stream when the corresponding door is opened / closed is collected.

[0018] In one or more embodiments, performing a corresponding action based on the user collaboration data stream and collecting a second augmented reality data stream when the corresponding action is performed on the vehicle includes:

[0019] identifying action parameters in the user collaboration data stream;

[0020] If the action parameter is to open / close the trunk, then the trunk is opened / closed; the action parameter value corresponding to the action parameter in the user collaboration data stream is empty;

[0021] The augmented reality data stream when the trunk is opened / closed is collected.

[0022] In one or more embodiments, performing a corresponding action based on the user collaboration data stream and collecting a second augmented reality data stream when the corresponding action is performed on the vehicle includes:

[0023] identifying action parameters in the user collaboration data stream;

[0024] If the action parameter is to move the lens, the lens is moved according to the moving direction value and the moving distance value in the action parameter value in the user collaborative data stream;

[0025] The augmented reality data stream when the moving lens is collected.

[0026] In one or more embodiments, before collecting the world environment data stream, the method further includes:

[0027] The augmented reality terminal device in the remote device is set to a multi-user operation mode.

[0028] In one or more embodiments, sending the first augmented reality data stream to the remote device includes:

[0029] Sending the first augmented reality data stream to the remote device using a preset frame rate;

[0030] The sending the second augmented reality data stream to the remote device comprises:

[0031] The second augmented reality data stream is sent to the remote device at a preset frame rate.

[0032] Accordingly, an embodiment of the present invention further discloses a method for remotely viewing a vehicle based on augmented reality, the method comprising:

[0033] receiving a first augmented reality data stream sent by a field device;

[0034] In response to a gesture action performed on the screen corresponding to the first augmented reality data stream, a corresponding user collaboration data stream is generated; the user collaboration data stream includes action parameters and action parameter values;

[0035] Sending the user collaboration data stream to the on-site device; so that the on-site device acquires a second augmented reality data stream based on the user collaboration data stream;

[0036] receiving the second augmented reality data stream;

[0037] The second augmented reality data stream is displayed.

[0038] In one or more embodiments, the generating corresponding user collaboration data in response to the gesture action performed on the screen corresponding to the first augmented reality data includes:

[0039] Determine the coordinates of performing the gesture action in the picture corresponding to the first augmented reality data stream, and the corresponding area of ​​the coordinates in the picture corresponding to the first augmented reality data stream;

[0040] Identify the gesture action, and obtain the action parameter and action parameter value corresponding to the gesture action in the corresponding area;

[0041] A user collaboration data stream is generated based on the action parameter and the action parameter value.

[0042] Accordingly, an embodiment of the present invention discloses a device for remotely viewing a vehicle based on augmented reality, the device comprising:

[0043] A collection module, used for collecting a world environment data stream and generating a first augmented reality data stream based on the world environment data stream;

[0044] A first sending module, configured to send the first augmented reality data stream to the remote device, so that the remote device generates a user collaboration data stream based on the first augmented reality data stream;

[0045] A first receiving module, configured to receive the user collaboration data stream;

[0046] a processing module, configured to perform a corresponding action based on the user collaboration data stream, and collect a second augmented reality data stream when the corresponding action is performed on the vehicle;

[0047] The first sending module is further used to send the second augmented reality data stream to the remote device.

[0048] In one or more embodiments, the processing module comprises:

[0049] A first identification submodule, used to identify action parameters in the user collaboration data stream;

[0050] an execution submodule, configured to, if the action parameter is to open / close a door, open / close the corresponding door according to the door identifier in the action parameter value in the user collaboration data stream;

[0051] The acquisition submodule is used to acquire the augmented reality data stream when the corresponding vehicle door is opened / closed.

[0052] In one or more embodiments, the processing module comprises:

[0053] The first identification submodule is further used to identify action parameters in the user collaboration data stream;

[0054] The execution submodule is further configured to open / close the trunk if the action parameter is to open / close the trunk; and the action parameter value corresponding to the action parameter in the user collaboration data stream is empty;

[0055] The acquisition submodule is further used to acquire the augmented reality data stream when the trunk is opened / closed.

[0056] In one or more embodiments, the processing module comprises:

[0057] The first identification submodule is further used to identify action parameters in the user collaboration data stream;

[0058] The execution submodule is further configured to move the lens according to the moving direction value and the moving distance value in the action parameter value in the user collaboration data stream if the action parameter is moving the lens;

[0059] The acquisition submodule is further used to acquire the augmented reality data stream when the lens is moved.

[0060] In one or more embodiments, further comprising:

[0061] The setting module is used to set the augmented reality terminal device in the remote device to a multi-user operation mode before collecting the world environment data stream.

[0062] In one or more embodiments, the first sending module is specifically configured to:

[0063] Sending the first augmented reality data stream to the remote device using a preset frame rate;

[0064] as well as,

[0065] The second augmented reality data stream is sent to the remote device at a preset frame rate.

[0066] Accordingly, an embodiment of the present invention further discloses a device for remotely viewing a vehicle based on augmented reality, the device comprising:

[0067] A second receiving module, configured to receive a first augmented reality data stream sent by a field device;

[0068] A generating module, configured to generate a corresponding user collaboration data stream in response to a gesture action performed on the screen corresponding to the first augmented reality data stream; the user collaboration data stream includes action parameters and action parameter values;

[0069] A second sending module, configured to send the user collaboration data stream to the on-site device, so that the on-site device acquires a second augmented reality data stream based on the user collaboration data stream;

[0070] The second receiving module is further used to receive the second augmented reality data stream;

[0071] A display module is used to display the second augmented reality data stream.

[0072] In one or more embodiments, the generating module comprises:

[0073] A determination submodule, configured to determine the coordinates of the gesture action in the picture corresponding to the first augmented reality data stream, and the corresponding area of ​​the coordinates in the picture corresponding to the first augmented reality data stream;

[0074] A second identification submodule is used to identify the gesture action and obtain the action parameter and action parameter value corresponding to the gesture action in the corresponding area;

[0075] A generating submodule is used to generate a user collaboration data stream based on the action parameter and the action parameter value.

[0076] Accordingly, an embodiment of the present invention discloses an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the various steps of the above-mentioned method embodiment for remotely viewing a vehicle based on augmented reality are implemented.

[0077] Accordingly, an embodiment of the present invention discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various steps of the above-mentioned method embodiment for remotely viewing a vehicle based on augmented reality are implemented.

[0078] The embodiments of the present invention include the following advantages:

[0079] The on-site device collects the world environment data stream, generates a first augmented reality data stream based on the world environment data stream, and sends the first augmented reality data stream to the remote device, so that the remote device generates a user collaboration data stream based on the first augmented reality data stream; receives the user collaboration data stream, then performs corresponding actions based on the user collaboration data stream, collects a second augmented reality data stream when performing corresponding actions on the vehicle, and sends the second augmented reality data stream to the remote device. In this way, users can not only remotely view real vehicles instead of virtual vehicle models, but also perform real operations on the vehicles, which improves the user experience. Moreover, the entire process does not require human participation, which greatly saves operating costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 This is a schematic diagram of the existing AR car viewing Figure 1 ;

[0081] Figure 2 This is a schematic diagram of the existing AR car viewing Figure 2 ;

[0082] Figure 3 is a flowchart of a method for remotely viewing a vehicle based on augmented reality according to a first embodiment of the present invention;

[0083] Figures 4A to 4B is a schematic diagram of the layout of the field equipment of the present invention;

[0084] Figure 5is a flowchart of the steps of Embodiment 2 of a method for remotely viewing a vehicle based on augmented reality of the present invention;

[0085] Figure 6 The present invention is a structural framework of an embodiment of a device for remotely viewing a vehicle based on augmented reality. Figure 1 ;

[0086] Figure 7 The present invention is a structural framework of an embodiment of a device for remotely viewing a vehicle based on augmented reality. Figure 2 . DETAILED DESCRIPTION

[0087] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0088] One of the core concepts of the embodiment of the present invention is that the on-site device collects the world environment data stream, generates a first augmented reality data stream based on the world environment data stream, and sends the first augmented reality data stream to the remote device, so that the remote device generates a user collaboration data stream based on the first augmented reality data stream; receives the user collaboration data stream, then performs corresponding actions based on the user collaboration data stream, and collects a second augmented reality data stream when performing the corresponding action on the vehicle, and sends the second augmented reality data stream to the remote device. In this way, users can not only remotely view the real vehicle instead of a virtual vehicle model, but also perform real operations on the vehicle, which improves the user experience. Moreover, the entire process does not require human participation, which greatly saves operating costs.

[0089] Reference Figure 3 , showing a step flow chart of a first embodiment of a method for remotely viewing a vehicle based on augmented reality of the present invention. The method is applied to on-site equipment, which can be equipment in the same space as the vehicle. The on-site equipment includes but is not limited to AR terminals and mechanical devices.

[0090] Among them, the AR terminal may be a terminal with AR-related basic functions. AR basic functions include being able to automatically identify objects in the real world under the AR terminal, adding virtual models to the real world and interacting with them. For example, AR cameras, head-mounted AR hardware, etc. In the embodiment of the present invention, there is no restriction on the specific form of the AR terminal, and it can be set according to actual needs, and the embodiment of the present invention does not limit this.

[0091] The mechanical device is used to perform corresponding actions on the vehicle according to the action instructions of the remote device, including but not limited to: opening / closing the door, opening / closing the trunk, starting, refueling, shifting gears, shutting down, etc.

[0092] Furthermore, the number of field devices may be one or more. Figure 4A The field device layout shown in the figure deploys four field devices around the vehicle so that users can switch perspectives in the remote device and view the vehicle from different angles. Figure 4B In the field device layout shown, one field device is deployed in the vehicle, and the purpose of viewing the vehicle from different angles can be achieved by manipulating the field device to move around the vehicle. Therefore, in actual applications, the number of field devices can be set according to actual needs, and the embodiment of the present invention does not limit this. For the convenience of description, the embodiment of the present invention takes one field device as an example for detailed description.

[0093] Furthermore, before establishing a connection between the on-site device and the remote device, the AR terminal in the on-site device can be set to a multi-user operation mode. In this mode, the on-site device can share the AR terminal with the remote device. During the entire sharing process, in addition to the real-time data transmission under the AR terminal, the operations of the remote device under the AR terminal can be transmitted to the on-site device, and then operate the vehicle in the real world on site.

[0094] It should be noted that in the embodiment of the present invention, the difference between the AR terminal and the ordinary terminal is that the AR terminal encapsulates the recognition capability of the real-world objects under the terminal, can mark the coordinates of the real-world objects, and can also place a virtual object on the real-world object based on the coordinates. If the remote device and the on-site device only transmit ordinary data streams, then this cannot be achieved.

[0095] The method may specifically include the following steps:

[0096] Step 301, collecting a world environment data stream, and generating a first augmented reality data stream based on the world environment data stream;

[0097] Specifically, when the AR device in the field equipment is set to a multi-user operation mode and is in a working state, a world environment data stream can be collected, wherein the world environment data stream includes but is not limited to: vehicle information in the real environment, AR three-dimensional world coordinate system data, etc. Then the world environment data stream is encapsulated to obtain augmented reality data (for the convenience of distinction, recorded as the first augmented reality data).

[0098] Step 302: sending the first augmented reality data stream to the remote device, so that the remote device generates a user collaboration data stream based on the first augmented reality;

[0099] Specifically, after obtaining the first augmented reality data, the first augmented reality data is sent to the remote device, and the remote device receives the first augmented reality data stream to generate a user collaboration data stream. The specific method of generating the user collaboration data stream will be described in detail later.

[0100] Step 303, receiving the user collaboration data stream;

[0101] Specifically, the on-site device and the remote device can be connected point-to-point without going through an intermediate device, so the on-site device can directly receive the user collaboration data stream sent by the remote device. The user collaboration data stream includes but is not limited to: the action parameters of the user gesture action and the action parameter values ​​corresponding to the action parameters.

[0102] The action parameter may be an action that the field device needs to perform, and the action parameter value may be a specific amount of the execution. For example, the action parameter may be movement, and the action parameter value may be a direction value and a distance value.

[0103] Step 304, performing a corresponding action based on the user collaboration data stream, and collecting a second augmented reality data stream when the corresponding action is performed on the vehicle;

[0104] After acquiring the user collaboration data stream, the on-site device can perform corresponding actions based on the action parameters and action parameter values ​​therein, and at the same time, collect the augmented reality data stream in the process of executing the action in real time.

[0105] In the embodiment of the present invention, the performing a corresponding action based on the user collaboration data stream and collecting a second augmented reality data stream when the corresponding action is performed on the vehicle includes:

[0106] identifying action parameters in the user collaboration data stream;

[0107] If the action parameter is opening / closing a door, opening / closing the corresponding door according to the door identifier in the action parameter value in the user collaboration data stream;

[0108] The augmented reality data stream when the corresponding door is opened / closed is collected.

[0109] Specifically, after receiving the user collaborative data stream, the action parameters therein can be identified. If the action parameter is opening / closing the door, the door identification in the action parameter value is obtained, such as left front, left rear, right front, and right rear. Then, the door corresponding to the door identification is opened / closed through the mechanical device in the on-site equipment. At the same time, starting from the receipt of the action parameters and action parameter values, all images of the process of opening / closing the door are collected in real time.

[0110] In the embodiment of the present invention, the performing of the corresponding action based on the action parameter and the action parameter value, and collecting the augmented reality data stream when the corresponding action is performed on the vehicle, includes:

[0111] identifying action parameters in the user collaboration data stream;

[0112] If the action parameter is to open / close the trunk, then the trunk is opened / closed; the action parameter value corresponding to the action parameter in the user collaboration data stream is empty;

[0113] The augmented reality data stream when the trunk is opened / closed is collected.

[0114] Specifically, after receiving the user collaborative data stream, the action parameters therein can be identified. If the action parameter is opening / closing the trunk, the trunk is opened / closed by the mechanical device in the on-site equipment. At the same time, starting from the receipt of the action parameter and the action parameter value, all images of the process of opening / closing the trunk are collected in real time.

[0115] It should be noted that, since the trunk is unique, the action parameter value for the trunk can be set to null. Of course, in practical applications, the action parameter value can be set to other values ​​according to actual needs without affecting the execution of the action by the field device, and the embodiment of the present invention does not limit this.

[0116] In the embodiment of the present invention, the performing a corresponding action based on the user collaboration data stream and collecting a second augmented reality data stream when the corresponding action is performed on the vehicle includes:

[0117] identifying action parameters in the user collaboration data stream;

[0118] If the action parameter is to move the lens, the lens is moved according to the moving direction value and the moving distance value in the action parameter value in the user collaborative data stream;

[0119] The augmented reality data stream when the moving lens is collected.

[0120] Specifically, after receiving the user collaborative data stream, the action parameters therein can be identified. If the action parameter is to move the lens, the moving direction value and the moving distance value in the action parameter value are obtained, and then the lens (that is, the AR terminal in the on-site device) is moved according to the moving direction value and the moving distance value. At the same time, starting from the receipt of the action parameter and the action parameter value, all images in the process of opening / closing the car door are collected in real time.

[0121] Step 305: Send the second augmented reality data stream to the remote device.

[0122] After the second augmented reality data stream is collected, the second augmented reality data stream is directly sent to the remote device, and the remote device can display the second augmented reality data stream to the user.

[0123] In this embodiment of the present invention, sending the first augmented reality data stream to the remote device includes:

[0124] Sending the first augmented reality data stream to the remote device using a preset frame rate;

[0125] The sending the second augmented reality data stream to the remote device comprises:

[0126] The second augmented reality data stream is sent to the remote device at a preset frame rate.

[0127] Specifically, in order to ensure the real-time data transmission between the remote device and the on-site device, the augmented reality data stream needs to be transmitted at a very high frame rate. Therefore, when sending the first augmented reality data stream and the second augmented reality data stream, a preset frame rate can be used for sending.

[0128] In an embodiment of the present invention, the on-site device collects the world environment data stream, generates a first augmented reality data stream based on the world environment data stream, and sends the first augmented reality data stream to the remote device, so that the remote device generates a user collaboration data stream based on the first augmented reality data stream; receives the user collaboration data stream, then performs corresponding actions based on the user collaboration data stream, collects a second augmented reality data stream when performing corresponding actions on the vehicle, and sends the second augmented reality data stream to the remote device. In this way, users can not only remotely view real vehicles instead of virtual vehicle models, but also perform real operations on the vehicles, which improves user experience. Moreover, the entire process does not require human participation, which greatly saves operating costs.

[0129] Reference Figure 5 , shows a flowchart of the steps of a second embodiment of a method for remotely viewing a vehicle based on augmented reality of the present invention, the method can be applied to a remote device, and the remote device can have the following features:

[0130] (1) In terms of hardware system, the device has a central processing unit, memory, input components and output components, that is, the device is often a microcomputer device with communication function. In addition, it can also have multiple input methods, such as keyboard, mouse, touch screen, microphone and camera, and the input can be adjusted as needed. At the same time, the device often has multiple output methods, such as receiver, display screen, etc., which can also be adjusted as needed;

[0131] (2) In terms of software system, the device must have an operating system, such as Windows Mobile, Symbian, Palm, Android, iOS, etc. At the same time, these operating systems are becoming more and more open, and personalized applications developed based on these open operating system platforms are emerging in an endless stream, such as address books, calendars, notepads, calculators, and various games, which greatly meet the needs of personalized users;

[0132] (3) In terms of communication capabilities, the device has flexible access methods and high-bandwidth communication performance, and can automatically adjust the selected communication method according to the selected service and the environment, so as to facilitate user use. The device can support 3GPP (3rd Generation Partnership Project), 4GPP (4th Generation Partnership Project), 5GPP (5th Generation Partnership Project), LTE (Long Term Evolution), WIMAX (World Interoperability for Microwave Access) mobile communications, computer network communications based on TCP / IP (Transmission Control Protocol / Internet Protocol), UDP (User Datagram Protocol) protocols, and short-range wireless transmission methods based on Bluetooth and infrared transmission standards. It not only supports voice services, but also supports a variety of wireless data services;

[0133] (4) In terms of functional use, the equipment pays more attention to humanization, personalization and multi-functionality. With the development of computer technology, the equipment has moved from a "device-centric" model to a "people-centric" model, integrating embedded computing, control technology, artificial intelligence technology and biometric authentication technology, fully reflecting the people-oriented purpose. Due to the development of software technology, the equipment can adjust the settings according to personal needs and become more personalized. At the same time, the equipment itself integrates a lot of software and hardware, and its functions are becoming more and more powerful.

[0134] The method may specifically include the following steps:

[0135] Step 501, receiving a first augmented reality data stream sent by a field device;

[0136] Specifically, after the on-site device establishes a connection with the remote device and the AR in the on-site device is set to a multi-user operation mode, the remote device can receive a first augmented reality data stream sent by the on-site device.

[0137] Step 502, in response to a gesture action performed on the screen corresponding to the first augmented reality data stream, generating a corresponding user collaboration data stream; the user collaboration data stream includes action parameters and action parameter values;

[0138] When the remote device detects a gesture action performed by the user on the screen corresponding to the displayed first augmented reality data stream, it can generate action parameters and action parameter values ​​corresponding to the gesture action, and generate a user collaboration data stream.

[0139] In the embodiment of the present invention, the generating corresponding user collaboration data in response to the gesture action performed on the screen corresponding to the first augmented reality data includes:

[0140] Determine the coordinates of performing the gesture action in the picture corresponding to the first augmented reality data stream, and the corresponding area of ​​the coordinates in the picture corresponding to the first augmented reality data stream;

[0141] Identify the gesture action, and obtain the action parameter and action parameter value corresponding to the gesture action in the corresponding area;

[0142] A user collaboration data stream is generated based on the action parameter and the action parameter value.

[0143] Specifically, when a gesture action is detected, the coordinates of the gesture action in the screen corresponding to the first augmented reality data stream can be determined, as well as the corresponding area of ​​the coordinates in the screen corresponding to the first augmented reality data stream, that is, it is determined which area of ​​the screen corresponding to the first augmented reality data stream the user is performing the gesture action on. Then the gesture action is identified to obtain the action parameters and action parameter values ​​corresponding to the gesture action in the corresponding area. Among them, the correspondence between the gesture action and the corresponding area, action parameters, and action parameter values ​​can be shown in Table 1:

[0144] Table 1

[0145]

[0146]

[0147] It should be noted that in the embodiment of the present invention, a mapping relationship between the sliding value on the screen and the actual value can be preset. When the gesture action is sliding, zooming in, zooming out, etc., the specific parameter value of the on-site device performing the action can be calculated according to the mapping relationship. For example, assuming that 1 pixel is mapped to 2 centimeters, when the user slides 10 pixels on the screen of the remote terminal, the on-site device moves 20 centimeters accordingly.

[0148] Alternatively, the spatial coordinates before and after the movement of the on-site device may be calculated based on the coordinates before and after the sliding on the screen, and then the on-site device may be moved to the spatial coordinates after the movement.

[0149] Of course, in addition to the above-mentioned methods, the field device may also be moved in other ways, which may be configured according to actual needs in practical applications, and the embodiments of the present invention do not limit this.

[0150] Furthermore, when identifying clicks on the car door and the trunk, a trained learning model can be used for recognition. The learning model can identify the car door area and the trunk area in the picture corresponding to the first augmented reality data stream, and when a gesture action is detected, detect whether the coordinates of the gesture action are in the car door area or the trunk area.

[0151] Step 502, sending the user collaboration data stream to the on-site device; so that the on-site device acquires a second augmented reality data stream based on the user collaboration data stream;

[0152] Step 503, receiving a second augmented reality data stream sent by the field device;

[0153] After determining the action parameters and action parameter values, the action parameters and action parameter values ​​are sent to the on-site device, so that the on-site device performs the corresponding action according to the action parameters and action parameter values, and in the process of performing the action, a second augmented reality data stream is collected and sent to the remote device. The on-site device performs the above steps with reference to steps 101 to 105, which will not be described here to avoid repetition.

[0154] Step 504: display the augmented reality data stream.

[0155] After receiving the augmented reality data stream, the remote device can display it to the user.

[0156] It should be noted that, in order to ensure user experience, the remote device can shorten the transmission frequency when sending action parameters and action parameter values ​​to the on-site device. In this way, when the user performs a gesture action in the remote device, the action parameters and action parameter values ​​in the execution process are synchronized in real time. For example, when the user slides from left to right on the screen of the remote device, in order to avoid sending the action parameters and action parameter values ​​only after the sliding stops, the action parameters and action parameter values ​​can be sent at the fastest transmission frequency during the user's sliding process.

[0157] Correspondingly, when the user is performing a gesture action, he or she will also receive and display the augmented reality data stream sent by the on-site device in real time. Therefore, the screen corresponding to the first augmented reality data stream is also updated in real time using the augmented reality data stream. The augmented reality data stream received from the on-site device is not displayed only after the user finishes the gesture action.

[0158] In an embodiment of the present invention, a remote device receives a first augmented reality data stream sent by an on-site device; generates a corresponding user collaboration data stream in response to a gesture action performed on a screen corresponding to the first augmented reality data stream; the user collaboration data stream includes action parameters and action parameter values; sends the user collaboration data stream to the on-site device; so that the on-site device acquires a second augmented reality data stream based on the user collaboration data stream; receives the second augmented reality data stream; and displays the second augmented reality data stream. In this way, users can not only remotely view real vehicles instead of virtual vehicle models, but also perform real operations on the vehicles, which improves user experience. Moreover, the entire process does not require human participation, which greatly saves operating costs.

[0159] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0160] Reference Figure 6 , shows a structural block diagram of a first embodiment of a device for remotely viewing a vehicle based on augmented reality of the present invention, which may specifically include the following modules:

[0161] The acquisition module 601 is used to acquire a world environment data stream and generate a first augmented reality data stream based on the world environment data stream;

[0162] A first sending module 602, configured to send the first augmented reality data stream to the remote device, so that the remote device generates a user collaboration data stream based on the first augmented reality data stream;

[0163] A first receiving module 603, configured to receive the user collaboration data stream;

[0164] A processing module 604 is used to perform a corresponding action based on the user collaboration data stream, and collect a second augmented reality data stream when the corresponding action is performed on the vehicle;

[0165] The first sending module is further used to send the second augmented reality data stream to the remote device.

[0166] In an embodiment of the present invention, the processing module includes:

[0167] A first identification submodule, used to identify action parameters in the user collaboration data stream;

[0168] an execution submodule, configured to, if the action parameter is to open / close a door, open / close the corresponding door according to the door identifier in the action parameter value in the user collaboration data stream;

[0169] The acquisition submodule is used to acquire the augmented reality data stream when the corresponding vehicle door is opened / closed.

[0170] In an embodiment of the present invention, the processing module includes:

[0171] The first identification submodule is further used to identify action parameters in the user collaboration data stream;

[0172] The execution submodule is further configured to open / close the trunk if the action parameter is to open / close the trunk; and the action parameter value corresponding to the action parameter in the user collaboration data stream is empty;

[0173] The acquisition submodule is further used to acquire the augmented reality data stream when the trunk is opened / closed.

[0174] In an embodiment of the present invention, the processing module includes:

[0175] The first identification submodule is further used to identify action parameters in the user collaboration data stream;

[0176] The execution submodule is further configured to move the lens according to the moving direction value and the moving distance value in the action parameter value in the user collaboration data stream if the action parameter is moving the lens;

[0177] The acquisition submodule is further used to acquire the augmented reality data stream when the lens is moved.

[0178] In an embodiment of the present invention, it also includes:

[0179] The setting module is used to set the augmented reality terminal device in the remote device to a multi-user operation mode before collecting the world environment data stream.

[0180] In the embodiment of the present invention, the first sending module is specifically used to:

[0181] Sending the first augmented reality data stream to the remote device using a preset frame rate;

[0182] as well as,

[0183] The second augmented reality data stream is sent to the remote device at a preset frame rate.

[0184] Reference Figure 7 , shows a structural block diagram of a second embodiment of a device for remotely viewing a vehicle based on augmented reality of the present invention, which may specifically include the following modules:

[0185] The second receiving module 701 is used to receive a first augmented reality data stream sent by a field device;

[0186] A generating module 702, configured to generate a corresponding user collaboration data stream in response to a gesture action performed on the screen corresponding to the first augmented reality data stream; the user collaboration data stream includes action parameters and action parameter values;

[0187] The second sending module 703 is used to send the user collaboration data stream to the field device, so that the field device acquires a second augmented reality data stream based on the user collaboration data stream;

[0188] The second receiving module is further used to receive the second augmented reality data stream;

[0189] The display module 704 is used to display the second augmented reality data stream.

[0190] In an embodiment of the present invention, the generating module includes:

[0191] A determination submodule, configured to determine the coordinates of the gesture action in the picture corresponding to the first augmented reality data stream, and the corresponding area of ​​the coordinates in the picture corresponding to the first augmented reality data stream;

[0192] A second identification submodule is used to identify the gesture action and obtain the action parameter and action parameter value corresponding to the gesture action in the corresponding area;

[0193] A generating submodule is used to generate a user collaboration data stream based on the action parameter and the action parameter value.

[0194] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0195] An embodiment of the present invention further provides an electronic device, including:

[0196] The present invention comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, each process of the above-mentioned method embodiment for remotely viewing a vehicle based on augmented reality is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0197] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned method embodiment for remotely viewing a vehicle based on augmented reality are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0198] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0199] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0200] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0201] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0202] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable terminal device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0203] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0204] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0205] The above is a detailed introduction to a method for remotely viewing a vehicle based on augmented reality and a device for remotely viewing a vehicle based on augmented reality provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for remotely viewing a vehicle based on augmented reality, characterized in that: The method comprises: Setting the augmented reality terminal device in the field device to a multi-user operation mode; When the augmented reality terminal device is in working state, a world environment data stream is collected, and a first augmented reality data stream is generated based on the world environment data stream; the world environment data stream includes: vehicle information in a real environment and AR three-dimensional world coordinate system data; Sending the first augmented reality data stream to a remote device, so that the remote device generates a user collaboration data stream based on the first augmented reality data stream; Receiving the user collaboration data stream; performing a corresponding action based on the user collaboration data stream, and collecting a second augmented reality data stream when the corresponding action is performed on the vehicle; sending the second augmented reality data stream to the remote device; The sending the first augmented reality data stream to the remote device so that the remote device generates a user collaboration data stream based on the first augmented reality data stream includes: When the remote device detects a gesture action performed by the user on the screen corresponding to the first augmented reality data stream being displayed, the coordinates of the gesture action performed in the screen corresponding to the first augmented reality data stream and the corresponding area of ​​the coordinates in the screen corresponding to the first augmented reality data stream are determined; the gesture action is identified to obtain the action parameters and action parameter values ​​corresponding to the gesture action in the corresponding area; the user collaborative data stream includes the action parameters and the action parameter values; the action parameters are the actions that need to be performed by the on-site equipment, and the action parameter values ​​are the specific amounts of execution; the action parameter values ​​are calculated based on the correspondence between the gesture action and the corresponding area, action parameters, and action parameter values.

2. The method for remotely viewing a vehicle based on augmented reality according to claim 1, characterized in that: The performing a corresponding action based on the user collaboration data stream and collecting a second augmented reality data stream when the corresponding action is performed on the vehicle includes: identifying action parameters in the user collaboration data stream; If the action parameter is opening / closing a door, opening / closing the corresponding door according to the door identifier in the action parameter value in the user collaboration data stream; The augmented reality data stream when the corresponding door is opened / closed is collected.

3. The method for remotely viewing a vehicle based on augmented reality according to claim 1, characterized in that: The performing a corresponding action based on the user collaboration data stream and collecting a second augmented reality data stream when the corresponding action is performed on the vehicle includes: identifying action parameters in the user collaboration data stream; If the action parameter is to open / close the trunk, then the trunk is opened / closed; the action parameter value corresponding to the action parameter in the user collaboration data stream is empty; The augmented reality data stream when the trunk is opened / closed is collected.

4. The method for remotely viewing a vehicle based on augmented reality according to claim 1, characterized in that: The performing a corresponding action based on the user collaboration data stream and collecting a second augmented reality data stream when the corresponding action is performed on the vehicle includes: identifying action parameters in the user collaboration data stream; If the action parameter is to move the lens, the lens is moved according to the moving direction value and the moving distance value in the action parameter value in the user collaborative data stream; The augmented reality data stream when the moving lens is collected.

5. The method for remotely viewing a vehicle based on augmented reality according to claim 1, characterized in that: The sending the first augmented reality data stream to the remote device comprises: Sending the first augmented reality data stream to the remote device using a preset frame rate; The sending the second augmented reality data stream to the remote device comprises: The second augmented reality data stream is sent to the remote device at a preset frame rate.

6. A method for remotely viewing a vehicle based on augmented reality, characterized in that: The method comprises: Receive a first augmented reality data stream sent by the on-site device; the first augmented reality data stream is generated by the augmented reality terminal device collecting a world environment data stream when the augmented reality terminal device in the on-site device is set to a multi-user operation mode and the augmented reality terminal device is in a working state, and the world environment data stream includes: vehicle information in a real environment and AR three-dimensional world coordinate system data; In response to a gesture action performed on the screen corresponding to the first augmented reality data stream, a corresponding user collaboration data stream is generated; the user collaboration data stream includes action parameters and action parameter values; Sending the user collaboration data stream to the on-site device; so that the on-site device acquires a second augmented reality data stream based on the user collaboration data stream; receiving the second augmented reality data stream; displaying the second augmented reality data stream; The step of generating a corresponding user collaboration data stream in response to a gesture action performed on a screen corresponding to the first augmented reality data stream comprises: When the remote device detects a gesture action performed by the user on the screen corresponding to the first augmented reality data stream being displayed, the coordinates of the gesture action performed in the screen corresponding to the first augmented reality data stream and the corresponding area of ​​the coordinates in the screen corresponding to the first augmented reality data stream are determined; the gesture action is identified to obtain the action parameters and action parameter values ​​corresponding to the gesture action in the corresponding area; the action parameters are the actions that need to be performed by the on-site equipment, and the action parameter values ​​are the specific amounts of execution; the action parameter values ​​are calculated based on the correspondence between the gesture action and the corresponding area, action parameters, and action parameter values.

7. The method for remotely viewing a vehicle based on augmented reality according to claim 6, characterized in that: The generating corresponding user collaboration data in response to the gesture action performed on the screen corresponding to the first augmented reality data includes: Determine the coordinates of performing the gesture action in the picture corresponding to the first augmented reality data stream, and the corresponding area of ​​the coordinates in the picture corresponding to the first augmented reality data stream; Identify the gesture action, and obtain the action parameter and action parameter value corresponding to the gesture action in the corresponding area; A user collaboration data stream is generated based on the action parameter and the action parameter value.

8. A device for remotely viewing a vehicle based on augmented reality, characterized in that: The device comprises: A first receiving module is used to receive action parameters and corresponding action parameter values ​​for remotely viewing a vehicle sent by a remote device; the action parameters and corresponding action parameter values ​​are generated according to the coordinates of performing the gesture action in the picture corresponding to the first augmented reality data stream determined when the remote device detects that the user performs a gesture action on the picture corresponding to the first augmented reality data stream displayed, and the coordinates are generated in the corresponding area in the picture corresponding to the first augmented reality data stream; the first augmented reality data stream is generated by the augmented reality terminal device collecting a world environment data stream when the augmented reality terminal device in the on-site device is set to a multi-user operation mode and the augmented reality terminal device is in a working state, and the world environment data stream includes: vehicle information in a real environment and AR three-dimensional world coordinate system data; a processing module, configured to execute a corresponding action based on the action parameter and the action parameter value, and collect an augmented reality data stream when the corresponding action is executed on the vehicle; A first sending module, configured to send the augmented reality data stream to the remote device; The action parameter is the action that the on-site device needs to perform, and the action parameter value is the specific amount of the execution; the action parameter value is calculated based on the corresponding relationship between the gesture action and the corresponding area, action parameter, and action parameter value.

9. A device for remotely viewing a vehicle based on augmented reality, characterized in that: The device comprises: A generating module, configured to generate corresponding action parameters and action parameter values ​​in response to a gesture action performed on the screen corresponding to the first augmented reality data stream; A second sending module, used for sending the action parameter and the action parameter value to the field device; A second receiving module, configured to receive an augmented reality data stream sent by the field device; A display module, used to display the augmented reality data stream; The generation module is further used to determine the coordinates of the gesture action in the picture corresponding to the first augmented reality data stream, and the corresponding area of ​​the coordinates in the picture corresponding to the first augmented reality data stream when the remote device detects a gesture action performed by the user on the picture corresponding to the first augmented reality data stream displayed; identify the gesture action, and obtain the action parameters and action parameter values ​​corresponding to the gesture action in the corresponding area; the first augmented reality data stream is generated by the augmented reality terminal device collecting the world environment data stream when the augmented reality terminal device in the on-site device is set to a multi-user operation mode and the augmented reality terminal device is in a working state, and the world environment data stream includes: vehicle information in the real environment, AR three-dimensional world coordinate system data; The action parameter is the action that the on-site device needs to perform, and the action parameter value is the specific amount of the execution; the action parameter value is calculated based on the corresponding relationship between the gesture action and the corresponding area, action parameter, and action parameter value.

10. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of the method for remotely viewing a vehicle based on augmented reality as described in any one of claims 1 to 7 are implemented.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for remotely viewing a vehicle based on augmented reality as described in any one of claims 1 to 7 are implemented.

Images