Blowing control method and device based on VR data
By obtaining image data in VR equipment and determining wind information using image and sound analysis algorithms, and generating output instructions of wind output equipment, the problems of unreal and high cost of stroke output control in the prior art are solved, realizing the blasting effect and Energy saving.
Patent Information
- Application Number
- CN202311857242.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-29
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2043-12-29
AI Technical Summary
In the absence of pre-designed air environment change parameters, it is difficult to control the air output equipment in real time and efficiently, resulting in unreal blowing effect and high cost.
By obtaining the image data prepared for playback by the target VR device, the first-person movement information, wind direction information and wind speed information corresponding to the image data is determined using image analysis and sound analysis algorithms, the output command of the wind output device is generated, and the wind output sequence and wind output speed are controlled. and the direction of air outlet, combined with dynamic programming and particle swarm algorithm to optimize output instructions to match the user's somatosensory.
It achieves a more realistic blowing effect, while reducing the cost of blowing and saving energy.
Smart Images

Figure CN117908670B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of VR data processing technology, and in particular to a blowing control method and device based on VR data. Background Art
[0002] In the existing VR devices, the control of the wind environment equipment is generally to determine the wind environment change parameters corresponding to the image in advance when designing the VR image, and to design the control parameters of the wind output device based on the designed wind environment change parameters. However, when faced with an image without pre-designed wind environment change parameters, the existing technology has no way to efficiently control the wind output device to output appropriate wind in real time, and it has certain technical limitations. It can be seen that the existing technology has defects and needs to be solved urgently. Summary of the invention
[0003] The technical problem to be solved by the present invention is to provide a blowing control method and device based on VR data, which can achieve a more realistic blowing effect on the one hand, and reduce the blowing cost to save energy on the other hand.
[0004] In order to solve the above technical problems, the first aspect of the present invention discloses a blowing control method based on VR data, the method comprising:
[0005] Obtain the image data that the target VR device is ready to play;
[0006] Determining first-person movement information and wind direction information corresponding to the image data according to an image analysis algorithm;
[0007] Determine wind speed information corresponding to the image data according to an image analysis algorithm and a sound analysis algorithm;
[0008] According to the first-person movement information, the wind direction information and the wind speed information, output instructions corresponding to multiple wind output devices of the target VR device are determined; the output instructions are used to control the wind output sequence, wind output speed and wind output direction of the wind output devices.
[0009] As an optional implementation, in the first aspect of the present invention, the image data includes a plurality of fragment image data at a plurality of different angles; the first-person movement information includes a first-person movement direction and a first-person movement speed of a first-person object; and determining the first-person movement information and wind direction information corresponding to the image data according to an image analysis algorithm includes:
[0010] Input each of the image data segments in the image data into an image type classifier algorithm model to obtain an image type and a prediction probability corresponding to each of the image data segments; the image type is a repeated background image or a target image; the image type classifier algorithm model is trained by a training data set including a plurality of training image data and corresponding image type annotations;
[0011] According to the image type and the prediction probability, all the fragment image data are grouped to obtain a set of repeated background image and a set of target image;
[0012] Determine the wind direction information corresponding to the image data according to the image analysis algorithm and the set of repeated background images;
[0013] According to the image analysis algorithm, the repeated background object image set and the target object image set, the first-person moving direction and the first-person moving speed corresponding to the image data are determined.
[0014] As an optional implementation, in the first aspect of the present invention, determining the wind direction information corresponding to the image data according to the image analysis algorithm and the set of repeated background images includes:
[0015] Based on an object recognition algorithm, each background object image in the repeated background object image set is processed to obtain a plurality of background object objects;
[0016] Calculating the tilt angle vector of each background object;
[0017] calculating an object distance between each of the background objects and the first person object;
[0018] The weighted sum of the tilt angle vectors of all the background objects is calculated to obtain the wind direction information corresponding to the image data; wherein the weight corresponding to each tilt angle vector is inversely proportional to the object distance corresponding to the corresponding background object.
[0019] As an optional implementation, in the first aspect of the present invention, determining the first-person moving direction and the first-person moving speed corresponding to the image data according to the image analysis algorithm, the repeated background image set and the target image set includes:
[0020] Calculating the background object moving direction and the background object moving speed corresponding to the repeated background object image set according to a plurality of continuous background object images in the repeated background object image set;
[0021] Calculating object relative information between the target object corresponding to each target object image in the target object image set and the first person object; the object relative information includes object relative distance and object relative direction;
[0022] Calculating the target object moving direction and target object moving speed corresponding to the target object image set according to the object relative information corresponding to a plurality of continuous target object images;
[0023] Calculating the average of the moving direction of the background object and the moving direction of the target object to obtain the first-person moving direction corresponding to the image data;
[0024] The average of the moving speeds of the background object and the target object is calculated to obtain the first-person moving speed corresponding to the image data.
[0025] As an optional implementation, in the first aspect of the present invention, determining the wind speed information corresponding to the image data according to the image analysis algorithm and the sound analysis algorithm includes:
[0026] Inputting each of the background object images in the repeated background object image set into a first wind speed prediction algorithm model to obtain a first wind speed prediction value corresponding to each of the background object images; the first wind speed prediction algorithm model is trained by a training data set including a plurality of training image data and corresponding wind speed annotations;
[0027] Calculate the magnitude of the tilt angle vector of the background object corresponding to each background image;
[0028] Inputting the synchronous audio track data corresponding to each of the background object images into the second wind speed prediction algorithm model to obtain the second wind speed prediction value corresponding to each of the background object images; the second wind speed prediction algorithm model is trained by a training data set including a plurality of training audio data and corresponding wind speed annotations;
[0029] Calculate the product of the wind speed average value of the first wind speed prediction value and the second wind speed prediction value corresponding to each background object image and the wind speed weight to obtain the wind speed parameter corresponding to each background object image; the wind speed weight is proportional to the size of the tilt angle vector corresponding to the corresponding background object image;
[0030] The average value of the wind speed parameters corresponding to all the background object images is calculated to obtain the wind speed information corresponding to the image data.
[0031] As an optional implementation, in the first aspect of the present invention, determining the output instructions corresponding to the multiple wind output devices of the target VR device according to the first-person movement information, the wind direction information, and the wind speed information includes:
[0032] Using the opposite value of the first-person moving direction as the vector direction and the first-person moving speed as the vector magnitude, a first-person vector is generated;
[0033] Generate a wind vector by using the wind direction information as the vector direction and the wind speed information as the vector magnitude;
[0034] Calculating the vector sum of the first-person vector and the wind vector based on the vector law to obtain a user body perception vector;
[0035] Determine the vector direction of the user's somatosensory vector as the user's somatosensory wind direction;
[0036] Determine the vector magnitude of the user's body-felt vector as the user's body-felt wind speed;
[0037] Based on a dynamic programming algorithm, output instructions corresponding to multiple wind output devices of the target VR device are calculated according to the user-perceived wind direction and the user-perceived wind speed.
[0038] As an optional implementation, in the first aspect of the present invention, the output instructions corresponding to the multiple wind output devices of the target VR device are calculated based on the dynamic programming algorithm according to the user's perceived wind direction and the user's perceived wind speed, including:
[0039] Determine the corresponding relationship between the instruction and the power consumption, the corresponding relationship between the instruction and the execution time, and the corresponding relationship between the instruction and the wind output effect vector corresponding to each wind output device of the target VR device;
[0040] Determine the objective function to minimize the sum of power consumption and execution time of all wind output devices when executing output instructions;
[0041] Determine the limiting condition that the sum of the wind output effect vectors of all the wind output devices when executing the output instruction matches the user's perceived wind direction and the user's perceived wind speed;
[0042] Based on the particle swarm algorithm, multiple output instruction schemes are iteratively generated, and according to the corresponding relationship between the instruction and power consumption, the corresponding relationship between the instruction and execution time, and the corresponding relationship between the instruction and the wind output effect vector corresponding to each wind output device, it is verified whether each of the output instruction schemes meets the objective function and the limiting conditions, until the optimal output instruction scheme is calculated to obtain the output instruction corresponding to each wind output device of the target VR device.
[0043] As an optional embodiment, in the first aspect of the present invention, the method further comprises:
[0044] Acquire real-time user operations performed by the user through the target VR device and real-time movement information of the target VR device;
[0045] According to the real-time user operation and the real-time movement information, the output instructions corresponding to the multiple wind output devices are modified to obtain the real-time output instructions corresponding to the multiple wind output devices;
[0046] And, the step of modifying the output instructions corresponding to the plurality of wind output devices according to the real-time user operation and the real-time movement information to obtain the real-time output instructions corresponding to the plurality of wind output devices comprises:
[0047] Determining whether the real-time user operation is an operation for controlling the movement of the first-person object, and obtaining a first determination result;
[0048] If the first judgment result is yes, determining the first-person movement information change value corresponding to the real-time user operation according to the correspondence between the real-time user operation, the preset operation, and the first-person movement information change value;
[0049] generating new first-person movement information according to the first-person movement information change value and the first-person movement information;
[0050] Determine, based on the particle swarm algorithm, the real-time output instructions corresponding to the multiple wind output devices of the target VR device according to the new first-person movement information, the wind direction information, and the wind speed information;
[0051] and / or,
[0052] Determine whether the real-time movement information changes the facing position of the user relative to the air outlet device, and obtain a second determination result;
[0053] When the second judgment result is yes, determining the current orientation of the user corresponding to the real-time movement information;
[0054] Calculate the user device direction line that is closest to the current facing angle; the user device direction line is a connecting line between the user position and the position of any of the wind output devices;
[0055] Determine that the wind output device corresponding to the direction line of the user device is a target wind output device;
[0056] Adding to the limiting condition that the air volume of the target air output device is less than a preset air volume value;
[0057] Based on the particle swarm algorithm, the objective function and the added limiting conditions, real-time output instructions corresponding to multiple wind output devices of the target VR device are determined.
[0058] The second aspect of the present invention discloses a blowing control device based on VR data, the device comprising:
[0059] The acquisition module is used to obtain the image data to be played by the target VR device;
[0060] A first determination module, used to determine first-person movement information and wind direction information corresponding to the image data according to an image analysis algorithm;
[0061] A second determination module, used to determine the wind speed information corresponding to the image data according to an image analysis algorithm and a sound analysis algorithm;
[0062] The third determination module is used to determine the output instructions corresponding to the multiple wind output devices of the target VR device according to the first-person movement information, the wind direction information and the wind speed information; the output instructions are used to control the wind output sequence, wind output speed and wind output direction of the wind output device.
[0063] As an optional implementation, in the second aspect of the present invention, the image data includes a plurality of fragment image data at a plurality of different angles; the first-person movement information includes a first-person movement direction and a first-person movement speed of a first-person object; and the first determination module determines the first-person movement information and wind direction information corresponding to the image data according to an image analysis algorithm, including:
[0064] Input each of the image data segments in the image data into an image type classifier algorithm model to obtain an image type and a prediction probability corresponding to each of the image data segments; the image type is a repeated background image or a target image; the image type classifier algorithm model is trained by a training data set including a plurality of training image data and corresponding image type annotations;
[0065] According to the image type and the prediction probability, all the fragment image data are grouped to obtain a set of repeated background image and a set of target image;
[0066] Determine the wind direction information corresponding to the image data according to the image analysis algorithm and the set of repeated background images;
[0067] According to the image analysis algorithm, the repeated background object image set and the target object image set, the first-person moving direction and the first-person moving speed corresponding to the image data are determined.
[0068] As an optional implementation, in the second aspect of the present invention, the first determination module determines the specific manner of the wind direction information corresponding to the image data according to the image analysis algorithm and the repeated background image set, including:
[0069] Based on an object recognition algorithm, each background object image in the repeated background object image set is processed to obtain a plurality of background object objects;
[0070] Calculating the tilt angle vector of each background object;
[0071] calculating an object distance between each of the background objects and the first person object;
[0072] The weighted sum of the tilt angle vectors of all the background objects is calculated to obtain the wind direction information corresponding to the image data; wherein the weight corresponding to each tilt angle vector is inversely proportional to the object distance corresponding to the corresponding background object.
[0073] As an optional implementation, in the second aspect of the present invention, the first determination module determines the specific manner of the first-person moving direction and the first-person moving speed corresponding to the image data according to the image analysis algorithm, the repeated background image set and the target image set, including:
[0074] Calculating the background object moving direction and the background object moving speed corresponding to the repeated background object image set according to a plurality of continuous background object images in the repeated background object image set;
[0075] Calculating object relative information between the target object corresponding to each target object image in the target object image set and the first person object; the object relative information includes object relative distance and object relative direction;
[0076] Calculating the target object moving direction and target object moving speed corresponding to the target object image set according to the object relative information corresponding to a plurality of continuous target object images;
[0077] Calculating the average of the moving direction of the background object and the moving direction of the target object to obtain the first-person moving direction corresponding to the image data;
[0078] The average of the moving speeds of the background object and the target object is calculated to obtain the first-person moving speed corresponding to the image data.
[0079] As an optional implementation, in the second aspect of the present invention, the specific manner in which the second determination module determines the wind speed information corresponding to the image data according to the image analysis algorithm and the sound analysis algorithm includes:
[0080] Inputting each of the background object images in the repeated background object image set into a first wind speed prediction algorithm model to obtain a first wind speed prediction value corresponding to each of the background object images; the first wind speed prediction algorithm model is trained by a training data set including a plurality of training image data and corresponding wind speed annotations;
[0081] Calculate the magnitude of the tilt angle vector of the background object corresponding to each background image;
[0082] Inputting the synchronous audio track data corresponding to each of the background object images into the second wind speed prediction algorithm model to obtain the second wind speed prediction value corresponding to each of the background object images; the second wind speed prediction algorithm model is trained by a training data set including a plurality of training audio data and corresponding wind speed annotations;
[0083] Calculate the product of the wind speed average value of the first wind speed prediction value and the second wind speed prediction value corresponding to each background object image and the wind speed weight to obtain the wind speed parameter corresponding to each background object image; the wind speed weight is proportional to the size of the tilt angle vector corresponding to the corresponding background object image;
[0084] The average value of the wind speed parameters corresponding to all the background object images is calculated to obtain the wind speed information corresponding to the image data.
[0085] As an optional implementation, in the second aspect of the present invention, the third determination module determines the specific manner of outputting instructions corresponding to the multiple wind output devices of the target VR device according to the first-person movement information, the wind direction information, and the wind speed information, including:
[0086] Using the opposite value of the first-person moving direction as the vector direction and the first-person moving speed as the vector magnitude, a first-person vector is generated;
[0087] Generate a wind vector by using the wind direction information as the vector direction and the wind speed information as the vector magnitude;
[0088] Calculating the vector sum of the first-person vector and the wind vector based on the vector law to obtain a user body perception vector;
[0089] Determine the vector direction of the user's somatosensory vector as the user's somatosensory wind direction;
[0090] Determine the vector magnitude of the user's body-felt vector as the user's body-felt wind speed;
[0091] Based on a dynamic programming algorithm, output instructions corresponding to multiple wind output devices of the target VR device are calculated according to the user-perceived wind direction and the user-perceived wind speed.
[0092] As an optional implementation, in the second aspect of the present invention, the third determination module calculates the specific manner of obtaining the output instructions corresponding to the multiple wind output devices of the target VR device based on the dynamic programming algorithm according to the user's perceived wind direction and the user's perceived wind speed, including:
[0093] Determine the corresponding relationship between the instruction and the power consumption, the corresponding relationship between the instruction and the execution time, and the corresponding relationship between the instruction and the wind output effect vector corresponding to each wind output device of the target VR device;
[0094] Determine the objective function to minimize the sum of power consumption and execution time of all wind output devices when executing output instructions;
[0095] Determine the limiting condition that the sum of the wind output effect vectors of all the wind output devices when executing the output instruction matches the user's perceived wind direction and the user's perceived wind speed;
[0096] Based on the particle swarm algorithm, multiple output instruction schemes are iteratively generated, and according to the corresponding relationship between the instruction and power consumption, the corresponding relationship between the instruction and execution time, and the corresponding relationship between the instruction and the wind output effect vector corresponding to each wind output device, it is verified whether each of the output instruction schemes meets the objective function and the limiting conditions, until the optimal output instruction scheme is calculated to obtain the output instruction corresponding to each wind output device of the target VR device.
[0097] As an optional implementation, in the second aspect of the present invention, the third determination module is further configured to perform the following steps:
[0098] Acquire real-time user operations performed by the user through the target VR device and real-time movement information of the target VR device;
[0099] According to the real-time user operation and the real-time movement information, the output instructions corresponding to the multiple wind output devices are modified to obtain the real-time output instructions corresponding to the multiple wind output devices;
[0100] And, the third determination module modifies the output instructions corresponding to the multiple wind output devices according to the real-time user operation and the real-time movement information to obtain the specific manner of the real-time output instructions corresponding to the multiple wind output devices, including:
[0101] Determining whether the real-time user operation is an operation for controlling the movement of the first-person object, and obtaining a first determination result;
[0102] If the first judgment result is yes, determining the first-person movement information change value corresponding to the real-time user operation according to the correspondence between the real-time user operation, the preset operation, and the first-person movement information change value;
[0103] generating new first-person movement information according to the first-person movement information change value and the first-person movement information;
[0104] Determine, based on the particle swarm algorithm, the real-time output instructions corresponding to the multiple wind output devices of the target VR device according to the new first-person movement information, the wind direction information, and the wind speed information;
[0105] and / or,
[0106] Determine whether the real-time movement information changes the facing position of the user relative to the air outlet device, and obtain a second determination result;
[0107] When the second judgment result is yes, determining the current orientation of the user corresponding to the real-time movement information;
[0108] Calculate the user device direction line that is closest to the current facing angle; the user device direction line is a connecting line between the user position and the position of any of the wind output devices;
[0109] Determine that the wind output device corresponding to the direction line of the user device is a target wind output device;
[0110] Adding to the limiting condition that the air volume of the target air output device is less than a preset air volume value;
[0111] Based on the particle swarm algorithm, the objective function and the added limiting conditions, real-time output instructions corresponding to multiple wind output devices of the target VR device are determined.
[0112] The third aspect of the present invention discloses another blowing control device based on VR data, the device comprising:
[0113] A memory storing executable program code;
[0114] A processor coupled to the memory; the processor is connected to a VR device; the VR device includes a plurality of wind output devices;
[0115] The processor calls the executable program code stored in the memory to execute part or all of the steps in the blowing control method based on VR data disclosed in the first aspect of the present invention.
[0116] Compared with the prior art, the present invention has the following beneficial effects:
[0117] It can be seen that the embodiments of the present invention can combine the image data to be played by the VR device, determine the corresponding movement information and wind information based on the algorithm, and determine the output instructions of the wind output device based on this, thereby achieving a more realistic blowing effect on the one hand, and reducing the blowing cost on the other hand to save energy. BRIEF DESCRIPTION OF THE DRAWINGS
[0118] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0119] Figure 1 It is a flow chart of a blowing control method based on VR data disclosed in an embodiment of the present invention.
[0120] Figure 2 It is a structural schematic diagram of a blowing control device based on VR data disclosed in an embodiment of the present invention.
[0121] Figure 3 It is a structural schematic diagram of another blowing control device based on VR data disclosed in an embodiment of the present invention. DETAILED DESCRIPTION
[0122] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0123] The terms "second", "secondary", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or equipment that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products or equipment.
[0124] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0125] The present invention discloses a blowing control method and device based on VR data, which can combine the image data to be played by the VR device, determine the corresponding movement information and wind information based on the algorithm, and determine the output instructions of the wind output device based on this, so as to achieve a more realistic blowing effect on the one hand, and reduce the blowing cost on the other hand to save energy. The following are detailed descriptions.
[0126] Embodiment 1
[0127] See also Figure 1 , Figure 1 : is a flow chart of a method for controlling air blowing based on VR data disclosed in an embodiment of the present invention. Figure 1 The described blowing control method based on VR data is applied to a data processing chip, a processing terminal or a processing server (wherein the processing server may be a local server or a cloud server). Figure 1 As shown, the blowing control method based on VR data may include the following operations:
[0128] 101. Obtain image data to be played by the target VR device.
[0129] 102. Determine first-person movement information and wind direction information corresponding to the image data according to an image analysis algorithm.
[0130] 103. Determine the wind speed information corresponding to the image data according to the image analysis algorithm and the sound analysis algorithm.
[0131] 104. Determine output instructions corresponding to multiple wind output devices of the target VR device according to the first-person movement information, wind direction information, and wind speed information.
[0132] Specifically, the output instruction is used to control the air output sequence, air output speed and air output direction of the air output device.
[0133] In a specific implementation scheme, the target VR device may include a VR display head-mounted device, a handheld device, a multi-directional walking device and multiple wind output devices, wherein the wind output device is a fan. The user can fully experience the panoramic gaming experience in the VR device, and the method of the present invention can effectively control the wind output of the fan to provide the user with a more realistic experience.
[0134] It can be seen that the above-mentioned embodiments of the invention can combine the image data to be played by the VR device, determine the corresponding movement information and wind information based on the algorithm, and determine the output instructions of the wind output device based on this, thereby achieving a more realistic blowing effect on the one hand, and reducing the blowing cost on the other hand to save energy.
[0135] As an optional embodiment, the image data includes multiple image data segments at multiple different angles. In some embodiments, the three-dimensional panoramic image data may be sampled at multiple playback time periods at multiple different angles to obtain multiple image data segments.
[0136] Specifically, the first-person movement information includes the first-person movement direction and the first-person movement speed of the first-person object, where the first-person object is a subjective object represented by the user in the gaming experience of the VR device.
[0137] Specifically, in the above steps, determining the first-person movement information and wind direction information corresponding to the image data according to the image analysis algorithm includes:
[0138] Input each segment of image data into the image type classifier algorithm model to obtain the image type and prediction probability corresponding to each segment of image data; the image type is a repeated background image or a target image; the image type classifier algorithm model is trained by a training data set including a plurality of training image data and corresponding image type annotations;
[0139] According to the image type and prediction probability, all the fragment image data are grouped to obtain a set of repeated background images and a set of target image;
[0140] According to the image analysis algorithm and the set of repeated background images, the wind direction information corresponding to the image data is determined;
[0141] According to the image analysis algorithm and the repeated background object image set and target object image set, the first-person moving direction and the first-person moving speed corresponding to the image data are determined.
[0142] Optionally, the classifier algorithm model or prediction algorithm model in the present invention can be a neural network algorithm model with a CNN structure, an RNN structure or an LTSM structure, and can be obtained by training and convergence through a corresponding gradient descent algorithm and a loss function, which is not limited by the present invention.
[0143] It can be seen that through the above embodiments, the image fragments can be first classified and grouped based on the classifier model, and then the first-person movement information and wind direction information corresponding to the image data can be determined based on the image analysis algorithm according to different types of images, so as to obtain more accurate analysis results, which will help to subsequently determine the output instructions of the wind output device based on this and achieve a more realistic blowing effect.
[0144] As an optional embodiment, in the above steps, determining the wind direction information corresponding to the image data according to the image analysis algorithm and the repeated background image set includes:
[0145] Based on the object recognition algorithm, each background object image in the repeated background object image set is processed to obtain multiple background object objects;
[0146] Calculate the tilt angle vector of each background object;
[0147] calculating an object distance between each background object and the first-person object;
[0148] The weighted sum of the tilt angle vectors of all background objects is calculated to obtain the wind direction information corresponding to the image data; wherein the weight corresponding to each tilt angle vector is inversely proportional to the object distance corresponding to the corresponding background object.
[0149] Specifically, the tilt angle vector of each background object can be represented as a line from one end of the straight line where the background object is located that deviates from the vertical direction of the coordinate system where the image data is located to a perpendicular line in the vertical direction.
[0150] Specifically, the object recognition algorithm can be implemented by an image segmentation algorithm.
[0151] Specifically, the background object may be a road railing, plants in a green belt, or houses at the edge of a street.
[0152] It can be seen that through the above embodiments, the background object can be first extracted based on the object recognition algorithm, and then the corresponding wind direction information can be calculated according to the inclination angle calculation rule to obtain a more accurate analysis result, which is helpful for subsequently determining the output instructions of the wind output device based on this and achieving a more realistic blowing effect.
[0153] As an optional embodiment, in the above steps, determining the first-person moving direction and the first-person moving speed corresponding to the image data according to the image analysis algorithm and the repeated background image set and the target image set includes:
[0154] Calculating the background object moving direction and the background object moving speed corresponding to the repeated background object image set according to a plurality of continuous background object images in the repeated background object image set;
[0155] Calculating object relative information between a target object corresponding to each target object image in the target object image set and a first-person object; the object relative information includes an object relative distance and an object relative direction;
[0156] Calculating the target moving direction and target moving speed corresponding to the target image set according to the object relative information corresponding to the plurality of continuous target images;
[0157] Calculate the average of the moving directions of the background object and the target object to obtain the first-person moving direction corresponding to the image data;
[0158] The average of the moving speeds of the background object and the target object is calculated to obtain the first-person moving speed corresponding to the image data.
[0159] Specifically, the continuous images in the present invention refer to images whose corresponding playback time points are in sequence.
[0160] Specifically, to calculate the object relative information, the target object in the target image can be identified by an object recognition algorithm, and then the coordinates of the target object and the coordinates of the first-person object are determined based on the image analysis algorithm and the coordinate system conversion algorithm, so as to calculate the object relative information.
[0161] Specifically, the target moving speed may be obtained by calculating the rate of change of the relative distance of the object corresponding to multiple consecutive target images, and the target moving direction may be obtained by calculating the direction of change of the relative direction of the object corresponding to multiple consecutive target images.
[0162] It can be seen that through the above embodiments, the moving direction and speed of the target object image and the moving direction and speed of the background object image can be calculated respectively to comprehensively correct and calculate the first-person moving direction and speed, so as to obtain a more accurate analysis result, which is helpful for subsequently determining the output instructions of the wind output device based on this and achieving a more realistic blowing effect.
[0163] As an optional embodiment, in the above steps, determining the wind speed information corresponding to the image data according to the image analysis algorithm and the sound analysis algorithm includes:
[0164] Input each background object image in the repeated background object image set into a first wind speed prediction algorithm model to obtain a first wind speed prediction value corresponding to each background object image; the first wind speed prediction algorithm model is trained by a training data set including a plurality of training image data and corresponding wind speed annotations;
[0165] Calculate the magnitude of the tilt angle vector of the background object corresponding to each background image;
[0166] Inputting the synchronous audio track data corresponding to each background object image into the second wind speed prediction algorithm model to obtain the second wind speed prediction value corresponding to each background object image; the second wind speed prediction algorithm model is trained by a training data set including a plurality of training audio data and corresponding wind speed annotations;
[0167] Calculate the product of the wind speed average value of the first wind speed prediction value and the second wind speed prediction value corresponding to each background object image and the wind speed weight to obtain the wind speed parameter corresponding to each background object image; the wind speed weight is proportional to the size of the tilt angle vector corresponding to the corresponding background object image;
[0168] Calculate the average value of the wind speed parameters corresponding to all background object images to obtain the wind speed information corresponding to the image data.
[0169] It can be seen that through the above embodiments, the wind speed prediction results of the background image and the wind speed prediction results of the audio data can be calculated separately to comprehensively correct and calculate the wind speed information corresponding to the image data, so as to obtain a more accurate analysis result, which is helpful for subsequently determining the output instructions of the wind output device based on this, and achieving a more realistic blowing effect.
[0170] As an optional embodiment, in the above step, determining the output instructions corresponding to multiple wind output devices of the target VR device according to the first-person movement information, wind direction information and wind speed information includes:
[0171] The opposite value of the first-person moving direction is used as the vector direction, and the first-person moving speed is used as the vector magnitude to generate the first-person vector;
[0172] The wind direction information is used as the vector direction, and the wind speed information is used as the vector magnitude to generate the wind vector;
[0173] The vector sum of the first-person vector and the wind vector is calculated based on the vector law to obtain the user's body perception vector;
[0174] Determine the vector direction of the user's body-felt vector as the user's body-felt wind direction;
[0175] Determine the vector magnitude of the user's body-felt vector as the user's body-felt wind speed;
[0176] Based on the dynamic programming algorithm, the output instructions corresponding to the multiple wind output devices of the target VR device are calculated according to the user's perceived wind direction and the user's perceived wind speed.
[0177] It can be seen that through the above embodiments, the user-perceived wind direction and the user-perceived wind speed can be calculated based on the first-person vector and the wind vector based on the vector calculation rules, and the output instructions corresponding to multiple wind output devices of the target VR device can be obtained based on the dynamic programming algorithm, which helps to achieve a more realistic blowing effect and reduce the cost of wind output.
[0178] As an optional embodiment, in the above steps, based on a dynamic programming algorithm, according to the user's perceived wind direction and the user's perceived wind speed, the output instructions corresponding to the multiple wind output devices of the target VR device are calculated, including:
[0179] Determine the corresponding relationship between the instruction and power consumption, the corresponding relationship between the instruction and execution time, and the corresponding relationship between the instruction and the wind output effect vector corresponding to each wind output device of the target VR device;
[0180] Determine the objective function to minimize the sum of power consumption and execution time of all wind output devices when executing output instructions;
[0181] Determine the limiting condition that the sum of the wind output effect vectors of all wind output devices when executing the output instruction matches the user's perceived wind direction and the user's perceived wind speed;
[0182] Based on the particle swarm algorithm, multiple output instruction schemes are iteratively generated, and according to the corresponding relationship between the instruction and power consumption, the corresponding relationship between the instruction and execution time, and the corresponding relationship between the instruction and the wind output effect vector corresponding to each wind output device, it is verified whether each output instruction scheme meets the objective function and the restriction conditions, until the optimal output instruction scheme is calculated to obtain the output instruction corresponding to each wind output device of the target VR device.
[0183] It can be seen that through the above embodiments, the optimal output instruction scheme can be calculated based on the particle swarm algorithm and the correspondence between the instructions and power consumption, the correspondence between the instructions and execution time, and the correspondence between the instructions and the wind output effect vector corresponding to each wind output device, which helps to achieve a more realistic blowing effect and reduce the wind output cost.
[0184] As an optional embodiment, the method further includes:
[0185] Acquire real-time user operations performed by the user through the target VR device and real-time movement information of the target VR device;
[0186] According to the real-time user operation and the real-time movement information, the output instructions corresponding to the multiple wind output devices are modified to obtain the real-time output instructions corresponding to the multiple wind output devices.
[0187] Optionally, the real-time user operation may be a user operation issued by the user through an interactive device, such as a user operation input through a key in a handheld device.
[0188] Optionally, the real-time movement information may be movement information of the user himself obtained by monitoring a head mounted device of the VR device.
[0189] It can be seen that through the above embodiments, the output instructions corresponding to multiple wind output devices can be corrected according to real-time user operations and real-time movement information to obtain real-time output instructions corresponding to multiple wind output devices, so as to correct the wind output strategy according to real-time feedback, which helps to achieve a more realistic blowing effect.
[0190] As an optional embodiment, in the above steps, the output instructions corresponding to the multiple wind output devices are modified according to the real-time user operation and the real-time movement information to obtain the real-time output instructions corresponding to the multiple wind output devices, including:
[0191] Determining whether the real-time user operation is an operation for controlling the movement of a first-person object, and obtaining a first determination result;
[0192] If the first judgment result is yes, determining the first-person movement information change value corresponding to the real-time user operation according to the correspondence between the real-time user operation, the preset operation, and the first-person movement information change value;
[0193] Generate new first-person movement information according to the first-person movement information change value and the first-person movement information;
[0194] According to the new first-person movement information, wind direction information and wind speed information, the real-time output instructions corresponding to multiple wind output devices of the target VR device are determined based on the particle swarm algorithm.
[0195] Optionally, the correspondence between the preset operation and the change value of the first-person movement information can be preset by the operator based on experimental data or experience.
[0196] Specifically, based on the particle swarm algorithm, the specific implementation method of determining the real-time output instructions corresponding to multiple wind output devices of the target VR device can refer to the details of iteratively generating multiple output instruction schemes based on the particle swarm algorithm in the above embodiment until the optimal output instruction scheme is calculated, which will not be elaborated in the present invention.
[0197] It can be seen that through the above embodiments, new first-person movement information can be generated according to real-time user operations, and the output instructions corresponding to multiple wind output devices can be corrected to obtain real-time output instructions corresponding to multiple wind output devices, so as to correct the wind output strategy according to real-time feedback, which helps to achieve a more realistic blowing effect.
[0198] As an optional embodiment, in the above step, it is determined whether the real-time movement information changes the facing position of the user relative to the air outlet device to obtain a second determination result;
[0199] When the second determination result is yes, determining the current orientation of the user corresponding to the real-time movement information;
[0200] Calculate the user device direction line that is closest to the current facing angle; the user device direction line is the connecting line between the user position and the position of any wind output device;
[0201] Determine the wind output device corresponding to the direction line of the user device as the target wind output device;
[0202] Adding the target wind output device's air volume to the limiting condition is less than the preset air volume value;
[0203] Based on the particle swarm algorithm, the objective function and the added limiting conditions, the real-time output instructions corresponding to the multiple wind output devices of the target VR device are determined.
[0204] Specifically, based on the particle swarm algorithm, the objective function and the added limiting conditions, the specific implementation method of determining the real-time output instructions corresponding to multiple wind output devices of the target VR device can refer to the details of iteratively generating multiple output instruction schemes based on the particle swarm algorithm in the above embodiment until the optimal output instruction scheme is calculated, which will not be elaborated in the present invention.
[0205] It can be seen that through the above embodiment, the new limiting conditions corresponding to the target wind output device can be determined according to the real-time movement information, and then the output instructions corresponding to the multiple wind output devices can be corrected based on the particle swarm algorithm to obtain the real-time output instructions corresponding to the multiple wind output devices, so as to reduce the amount of wind blowing directly on the user's face, which helps to improve the user experience.
[0206] Embodiment 2
[0207] See also Figure 2 , Figure 2 : is a structural schematic diagram of a blowing control device based on VR data disclosed in an embodiment of the present invention. Figure 2 The described blowing control device based on VR data is applied to a data processing chip, a processing terminal or a processing server (wherein the processing server may be a local server or a cloud server). Figure 2 As shown, the blowing control device based on VR data may include:
[0208] An acquisition module 201 is used to acquire image data to be played by a target VR device;
[0209] A first determination module 202, for determining first-person movement information and wind direction information corresponding to the image data according to an image analysis algorithm;
[0210] A second determination module 203, used to determine wind speed information corresponding to the image data according to an image analysis algorithm and a sound analysis algorithm;
[0211] The third determination module 204 is used to determine the output instructions corresponding to multiple wind output devices of the target VR device according to the first-person movement information, wind direction information and wind speed information; the output instructions are used to control the wind output sequence, wind output speed and wind output direction of the wind output devices.
[0212] As an optional embodiment, the image data includes a plurality of fragment image data at a plurality of different angles; the first-person movement information includes a first-person movement direction and a first-person movement speed of a first-person object; the first determination module 202 determines the first-person movement information and wind direction information corresponding to the image data according to the image analysis algorithm, including:
[0213] Input each segment of image data into the image type classifier algorithm model to obtain the image type and prediction probability corresponding to each segment of image data; the image type is a repeated background image or a target image; the image type classifier algorithm model is trained by a training data set including a plurality of training image data and corresponding image type annotations;
[0214] According to the image type and prediction probability, all the fragment image data are grouped to obtain a set of repeated background images and a set of target image;
[0215] According to the image analysis algorithm and the set of repeated background images, the wind direction information corresponding to the image data is determined;
[0216] According to the image analysis algorithm and the repeated background object image set and target object image set, the first-person moving direction and the first-person moving speed corresponding to the image data are determined.
[0217] As an optional embodiment, the first determination module 202 determines the specific manner of the wind direction information corresponding to the image data according to the image analysis algorithm and the repeated background image set, including:
[0218] Based on the object recognition algorithm, each background object image in the repeated background object image set is processed to obtain multiple background object objects;
[0219] Calculate the tilt angle vector of each background object;
[0220] calculating an object distance between each background object and the first-person object;
[0221] The weighted sum of the tilt angle vectors of all background objects is calculated to obtain the wind direction information corresponding to the image data; wherein the weight corresponding to each tilt angle vector is inversely proportional to the object distance corresponding to the corresponding background object.
[0222] As an optional embodiment, the first determination module 202 determines the specific manner of the first-person moving direction and the first-person moving speed corresponding to the image data according to the image analysis algorithm and the repeated background image set and the target image set, including:
[0223] Calculating the background object moving direction and the background object moving speed corresponding to the repeated background object image set according to a plurality of continuous background object images in the repeated background object image set;
[0224] Calculating object relative information between a target object corresponding to each target object image in the target object image set and a first-person object; the object relative information includes an object relative distance and an object relative direction;
[0225] Calculating the target moving direction and target moving speed corresponding to the target image set according to the object relative information corresponding to the plurality of continuous target images;
[0226] Calculate the average of the moving directions of the background object and the target object to obtain the first-person moving direction corresponding to the image data;
[0227] The average of the moving speeds of the background object and the target object is calculated to obtain the first-person moving speed corresponding to the image data.
[0228] As an optional embodiment, the second determination module 203 determines the specific manner of the wind speed information corresponding to the image data according to the image analysis algorithm and the sound analysis algorithm, including:
[0229] Input each background object image in the repeated background object image set into a first wind speed prediction algorithm model to obtain a first wind speed prediction value corresponding to each background object image; the first wind speed prediction algorithm model is trained by a training data set including a plurality of training image data and corresponding wind speed annotations;
[0230] Calculate the magnitude of the tilt angle vector of the background object corresponding to each background image;
[0231] Inputting the synchronous audio track data corresponding to each background object image into the second wind speed prediction algorithm model to obtain the second wind speed prediction value corresponding to each background object image; the second wind speed prediction algorithm model is trained by a training data set including a plurality of training audio data and corresponding wind speed annotations;
[0232] Calculate the product of the wind speed average value of the first wind speed prediction value and the second wind speed prediction value corresponding to each background object image and the wind speed weight to obtain the wind speed parameter corresponding to each background object image; the wind speed weight is proportional to the size of the tilt angle vector corresponding to the corresponding background object image;
[0233] Calculate the average value of the wind speed parameters corresponding to all background object images to obtain the wind speed information corresponding to the image data.
[0234] As an optional embodiment, the third determination module 204 determines the specific manner of outputting instructions corresponding to multiple wind output devices of the target VR device according to the first-person movement information, wind direction information, and wind speed information, including:
[0235] The opposite value of the first-person moving direction is used as the vector direction, and the first-person moving speed is used as the vector magnitude to generate the first-person vector;
[0236] The wind direction information is used as the vector direction, and the wind speed information is used as the vector magnitude to generate the wind vector;
[0237] The vector sum of the first-person vector and the wind vector is calculated based on the vector law to obtain the user's body perception vector;
[0238] Determine the vector direction of the user's body-felt vector as the user's body-felt wind direction;
[0239] Determine the vector magnitude of the user's body-felt vector as the user's body-felt wind speed;
[0240] Based on the dynamic programming algorithm, the output instructions corresponding to the multiple wind output devices of the target VR device are calculated according to the user's perceived wind direction and the user's perceived wind speed.
[0241] As an optional embodiment, the third determination module 204 calculates the specific manner of obtaining the output instructions corresponding to the multiple wind output devices of the target VR device based on the dynamic programming algorithm and the user-perceived wind direction and the user-perceived wind speed, including:
[0242] Determine the corresponding relationship between the instruction and power consumption, the corresponding relationship between the instruction and execution time, and the corresponding relationship between the instruction and the wind output effect vector corresponding to each wind output device of the target VR device;
[0243] Determine the objective function to minimize the sum of power consumption and execution time of all wind output devices when executing output instructions;
[0244] Determine the limiting condition that the sum of the wind output effect vectors of all wind output devices when executing the output instruction matches the user's perceived wind direction and the user's perceived wind speed;
[0245] Based on the particle swarm algorithm, multiple output instruction schemes are iteratively generated, and according to the corresponding relationship between the instruction and power consumption, the corresponding relationship between the instruction and execution time, and the corresponding relationship between the instruction and the wind output effect vector corresponding to each wind output device, it is verified whether each output instruction scheme meets the objective function and the restriction conditions, until the optimal output instruction scheme is calculated to obtain the output instruction corresponding to each wind output device of the target VR device.
[0246] As an optional embodiment, the third determining module 204 is further configured to perform the following steps:
[0247] Acquire real-time user operations performed by the user through the target VR device and real-time movement information of the target VR device;
[0248] According to the real-time user operation and the real-time movement information, the output instructions corresponding to the multiple wind output devices are modified to obtain the real-time output instructions corresponding to the multiple wind output devices;
[0249] And, the third determination module 204 modifies the output instructions corresponding to the multiple wind output devices according to the real-time user operation and the real-time movement information to obtain the specific manner of the real-time output instructions corresponding to the multiple wind output devices, including:
[0250] Determining whether the real-time user operation is an operation for controlling the movement of a first-person object, and obtaining a first determination result;
[0251] If the first judgment result is yes, determining the first-person movement information change value corresponding to the real-time user operation according to the correspondence between the real-time user operation, the preset operation, and the first-person movement information change value;
[0252] Generate new first-person movement information according to the first-person movement information change value and the first-person movement information;
[0253] According to the new first-person movement information, wind direction information and wind speed information, based on the particle swarm algorithm, the real-time output instructions corresponding to the multiple wind output devices of the target VR device are determined;
[0254] and / or,
[0255] Determine whether the real-time movement information changes the facing position of the user relative to the air outlet device, and obtain a second determination result;
[0256] When the second determination result is yes, determining the current orientation of the user corresponding to the real-time movement information;
[0257] Calculate the user device direction line that is closest to the current facing angle; the user device direction line is the connecting line between the user position and the position of any wind output device;
[0258] Determine the wind output device corresponding to the direction line of the user device as the target wind output device;
[0259] Adding the target wind output device's air volume to the limiting condition is less than the preset air volume value;
[0260] Based on the particle swarm algorithm, the objective function and the added limiting conditions, the real-time output instructions corresponding to the multiple wind output devices of the target VR device are determined.
[0261] The specific technical details and technical effects of the modules and steps in the above embodiments can be referred to the corresponding descriptions in Embodiment 1, which will not be repeated here.
[0262] Embodiment 3
[0263] See also Figure 3 , Figure 3 It is another blowing control device based on VR data disclosed in an embodiment of the present invention. Figure 3 The described blowing control device based on VR data is applied to a data processing chip, a processing terminal or a processing server (wherein the processing server may be a local server or a cloud server). Figure 3 As shown, the blowing control device based on VR data may include:
[0264] A memory 301 storing executable program codes;
[0265] A processor 302 coupled to the memory 301; the processor 302 can be connected to a user's VR device; the VR device includes a plurality of wind output devices;
[0266] The processor 302 calls the executable program code stored in the memory 301 to execute the steps of the blowing control method based on VR data described in the first embodiment.
[0267] Embodiment 4
[0268] An embodiment of the present invention discloses a computer-readable storage medium storing a computer program for electronic data exchange, wherein the computer program enables a computer to execute the steps of the blowing control method based on VR data described in the first embodiment.
[0269] Embodiment 5
[0270] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the steps of the blowing control method based on VR data described in the first embodiment.
[0271] The above describes specific embodiments of the present specification, and other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily have to be performed in the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0272] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and non-volatile computer-readable storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.
[0273] The apparatus, device, non-volatile computer-readable storage medium and method provided in the embodiments of this specification correspond to each other, and therefore, the apparatus, device, and non-volatile computer storage medium also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the corresponding apparatus, device, and non-volatile computer storage medium will not be repeated here.
[0274] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.
[0275] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.
[0276] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
[0277] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.
[0278] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may be in the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of this specification may be in the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0279] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0280] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0281] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0282] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0283] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0284] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0285] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0286] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media including storage devices.
[0287] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0288] Finally, it should be noted that the blowing control method and device based on VR data disclosed in the embodiment of the present invention only discloses the preferred embodiment of the present invention, which is only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.
Claims
1. A blowing control method based on VR data, characterized in that: The method comprises: Obtain the image data that the target VR device is ready to play; Determine, according to an image analysis algorithm, first-person movement information and wind direction information corresponding to the image data; the image data includes a plurality of fragment image data at a plurality of different angles; the first-person movement information includes a first-person movement direction and a first-person movement speed of a first-person object; the determining, according to the image analysis algorithm, the first-person movement information and wind direction information corresponding to the image data includes: Input each of the image data segments in the image data into an image type classifier algorithm model to obtain an image type and a prediction probability corresponding to each of the image data segments; the image type is a repeated background image or a target image; the image type classifier algorithm model is trained by a training data set including a plurality of training image data and corresponding image type annotations; According to the image type and the prediction probability, all the fragment image data are grouped to obtain a set of repeated background image and a set of target image; Based on an object recognition algorithm, each background object image in the repeated background object image set is processed to obtain a plurality of background object objects; Calculating the tilt angle vector of each background object; calculating an object distance between each of the background objects and the first person object; Calculating the weighted sum of the tilt angle vectors of all the background objects to obtain the wind direction information corresponding to the image data; wherein the weight corresponding to each tilt angle vector is inversely proportional to the object distance corresponding to the corresponding background object; Calculating the background object moving direction and the background object moving speed corresponding to the repeated background object image set according to a plurality of continuous background object images in the repeated background object image set; Calculating object relative information between the target object corresponding to each target object image in the target object image set and the first person object; the object relative information includes object relative distance and object relative direction; Calculating the target object moving direction and target object moving speed corresponding to the target object image set according to the object relative information corresponding to a plurality of continuous target object images; Calculating the average of the moving direction of the background object and the moving direction of the target object to obtain the first-person moving direction corresponding to the image data; Calculate the average of the moving speed of the background object and the moving speed of the target object to obtain the first-person moving speed corresponding to the image data; Determining wind speed information corresponding to the image data according to an image analysis algorithm and a sound analysis algorithm includes: Inputting each of the background object images in the repeated background object image set into a first wind speed prediction algorithm model to obtain a first wind speed prediction value corresponding to each of the background object images; the first wind speed prediction algorithm model is trained by a training data set including a plurality of training image data and corresponding wind speed annotations; Calculate the magnitude of the tilt angle vector of the background object corresponding to each background image; Inputting the synchronous audio track data corresponding to each of the background object images into the second wind speed prediction algorithm model to obtain the second wind speed prediction value corresponding to each of the background object images; the second wind speed prediction algorithm model is trained by a training data set including a plurality of training audio data and corresponding wind speed annotations; Calculate the product of the wind speed average value of the first wind speed prediction value and the second wind speed prediction value corresponding to each background object image and the wind speed weight to obtain the wind speed parameter corresponding to each background object image; the wind speed weight is proportional to the size of the tilt angle vector corresponding to the corresponding background object image; Calculate the average value of the wind speed parameters corresponding to all the background object images to obtain the wind speed information corresponding to the image data; According to the first-person movement information, the wind direction information and the wind speed information, output instructions corresponding to multiple wind output devices of the target VR device are determined; the output instructions are used to control the wind output sequence, wind output speed and wind output direction of the wind output devices.
2. The blowing control method based on VR data according to claim 1, characterized in that: The determining, according to the first-person movement information, the wind direction information, and the wind speed information, output instructions corresponding to the plurality of wind output devices of the target VR device includes: Using the opposite value of the first-person moving direction as the vector direction and the first-person moving speed as the vector magnitude, a first-person vector is generated; Generate a wind vector by using the wind direction information as the vector direction and the wind speed information as the vector magnitude; Calculating the vector sum of the first-person vector and the wind vector based on the vector law to obtain a user body perception vector; Determine the vector direction of the user's somatosensory vector as the user's somatosensory wind direction; Determine the vector magnitude of the user's body-felt vector as the user's body-felt wind speed; Based on a dynamic programming algorithm, output instructions corresponding to multiple wind output devices of the target VR device are calculated according to the user-perceived wind direction and the user-perceived wind speed.
3. The blowing control method based on VR data according to claim 2, characterized in that: The method of calculating output instructions corresponding to multiple wind output devices of the target VR device based on the dynamic programming algorithm according to the user's perceived wind direction and the user's perceived wind speed includes: Determine the corresponding relationship between the instruction and the power consumption, the corresponding relationship between the instruction and the execution time, and the corresponding relationship between the instruction and the wind output effect vector corresponding to each wind output device of the target VR device; Determine the objective function to minimize the sum of power consumption and execution time of all wind output devices when executing output instructions; Determine the limiting condition that the sum of the wind output effect vectors of all the wind output devices when executing the output instruction matches the user's perceived wind direction and the user's perceived wind speed; Based on the particle swarm algorithm, multiple output instruction schemes are iteratively generated, and according to the corresponding relationship between the instruction and power consumption, the corresponding relationship between the instruction and execution time, and the corresponding relationship between the instruction and the wind output effect vector corresponding to each wind output device, it is verified whether each of the output instruction schemes meets the objective function and the limiting conditions, until the optimal output instruction scheme is calculated to obtain the output instruction corresponding to each wind output device of the target VR device.
4. The blowing control method based on VR data according to claim 3 is characterized in that: The method further comprises: Acquire real-time user operations performed by the user through the target VR device and real-time movement information of the target VR device; According to the real-time user operation and the real-time movement information, the output instructions corresponding to the multiple wind output devices are modified to obtain the real-time output instructions corresponding to the multiple wind output devices; And, the step of modifying the output instructions corresponding to the plurality of wind output devices according to the real-time user operation and the real-time movement information to obtain the real-time output instructions corresponding to the plurality of wind output devices comprises: Determining whether the real-time user operation is an operation for controlling the movement of the first-person object, and obtaining a first determination result; If the first judgment result is yes, determining the first-person movement information change value corresponding to the real-time user operation according to the correspondence between the real-time user operation, the preset operation, and the first-person movement information change value; generating new first-person movement information according to the first-person movement information change value and the first-person movement information; Determine, based on the particle swarm algorithm, the real-time output instructions corresponding to the multiple wind output devices of the target VR device according to the new first-person movement information, the wind direction information, and the wind speed information; and / or, Determine whether the real-time movement information changes the facing position of the user relative to the air outlet device, and obtain a second determination result; When the second judgment result is yes, determining the current orientation of the user corresponding to the real-time movement information; Calculate the user device direction line that is closest to the current facing angle; the user device direction line is a connecting line between the user position and the position of any of the wind output devices; Determine that the wind output device corresponding to the direction line of the user device is a target wind output device; Adding to the limiting condition that the air volume of the target air output device is less than a preset air volume value; Based on the particle swarm algorithm, the objective function and the added limiting conditions, real-time output instructions corresponding to multiple wind output devices of the target VR device are determined.
5. A blowing control device based on VR data, characterized in that: The device comprises: The acquisition module is used to obtain the image data to be played by the target VR device; The first determination module is used to determine the first-person movement information and wind direction information corresponding to the image data according to the image analysis algorithm; the image data includes a plurality of fragment image data at a plurality of different angles; the first-person movement information includes a first-person movement direction and a first-person movement speed of the first-person object; the specific manner in which the first determination module determines the first-person movement information and wind direction information corresponding to the image data according to the image analysis algorithm includes: Input each of the image data segments in the image data into an image type classifier algorithm model to obtain an image type and a prediction probability corresponding to each of the image data segments; the image type is a repeated background image or a target image; the image type classifier algorithm model is trained by a training data set including a plurality of training image data and corresponding image type annotations; According to the image type and the prediction probability, all the fragment image data are grouped to obtain a set of repeated background image and a set of target image; Based on an object recognition algorithm, each background object image in the repeated background object image set is processed to obtain a plurality of background object objects; Calculating the tilt angle vector of each background object; calculating an object distance between each of the background objects and the first person object; Calculating the weighted sum of the tilt angle vectors of all the background objects to obtain the wind direction information corresponding to the image data; wherein the weight corresponding to each tilt angle vector is inversely proportional to the object distance corresponding to the corresponding background object; Calculating the background object moving direction and the background object moving speed corresponding to the repeated background object image set according to a plurality of continuous background object images in the repeated background object image set; Calculating object relative information between the target object corresponding to each target object image in the target object image set and the first person object; the object relative information includes object relative distance and object relative direction; Calculating the target object moving direction and target object moving speed corresponding to the target object image set according to the object relative information corresponding to a plurality of continuous target object images; Calculating the average of the moving direction of the background object and the moving direction of the target object to obtain the first-person moving direction corresponding to the image data; Calculate the average of the moving speed of the background object and the moving speed of the target object to obtain the first-person moving speed corresponding to the image data; The second determination module is used to determine the wind speed information corresponding to the image data according to the image analysis algorithm and the sound analysis algorithm, including: Inputting each of the background object images in the repeated background object image set into a first wind speed prediction algorithm model to obtain a first wind speed prediction value corresponding to each of the background object images; the first wind speed prediction algorithm model is trained by a training data set including a plurality of training image data and corresponding wind speed annotations; Calculate the magnitude of the tilt angle vector of the background object corresponding to each background image; Inputting the synchronous audio track data corresponding to each of the background object images into the second wind speed prediction algorithm model to obtain the second wind speed prediction value corresponding to each of the background object images; the second wind speed prediction algorithm model is trained by a training data set including a plurality of training audio data and corresponding wind speed annotations; Calculate the product of the wind speed average value of the first wind speed prediction value and the second wind speed prediction value corresponding to each background object image and the wind speed weight to obtain the wind speed parameter corresponding to each background object image; the wind speed weight is proportional to the size of the tilt angle vector corresponding to the corresponding background object image; Calculate the average value of the wind speed parameters corresponding to all the background object images to obtain the wind speed information corresponding to the image data; The third determination module is used to determine the output instructions corresponding to the multiple wind output devices of the target VR device according to the first-person movement information, the wind direction information and the wind speed information; the output instructions are used to control the wind output sequence, wind output speed and wind output direction of the wind output device.
6. A blowing control device based on VR data, characterized in that: The device comprises: A memory storing executable program code; A processor coupled to the memory; the processor is connected to a VR device; the VR device includes a plurality of wind output devices; the processor calls the executable program code stored in the memory to execute the blowing control method based on VR data as described in any one of claims 1-4.
