Information processing apparatus and method, electronic device, and computer readable medium

Abstract

The invention relates to an information processing apparatus and method, an electronic device, and a computer readable medium. According to one embodiment, an information processing apparatus includes processing circuitry configured to compare environmental data acquired by a user device with reference data, determine an adjustment to an acquisition manner of the environmental data based on the comparison, and control to notify the user device of indication information related to the adjustment.

Description

[0001] This application for invention is a divisional application of the patent application No. 201710912367.4, with the application date of September 29, 2017 and the invention title of "Information Processing Device and Method, Electronic Device, and Computer Readable Medium". Technical Field

[0002] The present disclosure generally relates to the field of information processing, and more particularly, to an information processing device, an information processing method, an electronic device for a user equipment side and an information processing method, and a computer readable medium. Background Art

[0003] Existing user equipment can acquire various environmental data such as video data, audio data, etc., and can upload the acquired environmental data to a server or the like. For example, a smart phone can take pictures and can upload the taken pictures to a social network or the like. In addition, there may be a situation where multiple users shoot the same scene or event. Summary of the Invention

[0004] A brief overview of embodiments of the present invention is given below to provide a basic understanding of certain aspects of the present invention. It should be understood that the following overview is not an exhaustive overview of the present invention. It is not intended to identify the key or important parts of the present invention, nor is it intended to limit the scope of the present invention. Its purpose is merely to present certain concepts in a simplified form as a prelude to the more detailed description to be discussed later.

[0005] According to one embodiment, there is provided an information processing device including a processing circuit configured to: compare environmental data acquired by a user equipment with reference data; determine an adjustment to a manner of acquiring the environmental data based on the comparison; and perform control to notify the user equipment of indication information related to the adjustment.

[0006] According to another embodiment, an information processing method includes: comparing environmental data acquired by a user equipment with reference data; determining an adjustment to a manner of acquiring the environmental data based on the comparison; and notifying the user equipment of indication information related to the adjustment.

[0007] According to still another embodiment, an information processing method includes: receiving environmental data related to a certain scene acquired by a plurality of user equipments, the environmental data acquired by the plurality of user equipments being at least partially different; determining the scene; and fusing the environmental data acquired by the plurality of user equipments based on the determined scene to generate scene information.

[0008] According to another embodiment, an electronic device for the user equipment side is provided, which includes a processing circuit. The processing circuit is configured to perform control to obtain environmental data; send the obtained environmental data to a control node; and receive, from the control node, indication information related to an adjustment of a way of obtaining the environmental data, where the adjustment is determined based on a comparison between the environmental data and reference data.

[0009] According to yet another embodiment, an information processing method for the user equipment side includes: obtaining environmental data; sending the obtained environmental data to a control node; and receiving, from the control node, indication information related to an adjustment of a way of obtaining the environmental data, where the adjustment is determined based on a comparison between the environmental data and reference data.

[0010] Embodiments of the present disclosure further include a computer-readable medium, which includes executable instructions that, when executed by an information processing device, cause the information processing device to execute the method according to the embodiments of the present disclosure.

[0011] According to the embodiments of the present disclosure, it is beneficial to improve the effect of obtaining environmental data. Description of the Drawings

[0012] The present invention can be better understood by referring to the following description in conjunction with the accompanying drawings, in which the same or similar reference numerals are used throughout the drawings to denote the same or similar components. The accompanying drawings, together with the following detailed description, are included in this specification and form a part of this specification, and are used to further illustrate the preferred embodiments of the present invention and explain the principles and advantages of the present invention. In the drawings:

[0013] Figure 1 is a block diagram showing a configuration example of an information processing device according to an embodiment of the present invention;

[0014] Figure 2 is a block diagram showing a configuration example of an information processing device according to another embodiment;

[0015] Figure 3 is a block diagram showing a configuration example of an information processing device according to yet another embodiment;

[0016] Figure 4 is a flowchart showing a process example of an information processing method according to an embodiment of the present invention;

[0017] Figure 5 is a block diagram showing a configuration example of an electronic device for the user equipment side according to an embodiment of the present invention;

[0018] Figure 6 is a block diagram showing a configuration example of an electronic device for the user equipment side according to another embodiment;

[0019] Figure 7 is a block diagram showing a configuration example of an electronic device on the user equipment side according to yet another embodiment;

[0020] Figure 8 is a flowchart showing a process example of an information processing method on the user equipment side according to an embodiment of the present invention;

[0021] Figure 9 is a block diagram showing a configuration example of an information processing device according to an embodiment;

[0022] Figure 10 is a block diagram showing a configuration example of an information processing device according to another embodiment;

[0023] Figure 11 is a flowchart for explaining an information processing process according to an exemplary embodiment;

[0024] Figures 12A to 12E shows an example of a process of obtaining a panoramic video through multiple user devices;

[0025] Figure 13 shows an example of presenting an adjustment instruction;

[0026] Figure 14 is a block diagram showing an exemplary structure of a computer for implementing the methods and devices of the present disclosure;

[0027] Figure 15 is a block diagram showing an example of a schematic configuration of a smart phone to which the technology of the present disclosure can be applied;

[0028] Figure 16 is a block diagram showing an example of a schematic configuration of a base station to which the technology of the present disclosure can be applied. Detailed Description of the Invention

[0029] Embodiments of the present invention will be described below with reference to the accompanying drawings. Elements and features described in one drawing or one embodiment of the present invention may be combined with elements and features shown in one or more other drawings or embodiments. It should be noted that for the sake of clarity, representations and descriptions of components and processes unrelated to the present invention and known to those of ordinary skill in the art are omitted in the drawings and the description.

[0030] As Figure 1 shown, the information processing device 100 according to the present embodiment includes a processing circuit 110. The processing circuit 110 can be implemented, for example, as a specific chip, a chipset, or a central processing unit (CPU), etc.

[0031] The information processing device 100 can be implemented as a logical entity on the network side. By way of example and not limitation, logical entities on the network side include, for example, the background servers of data sharing websites, etc. In addition, the information processing device 100 can also be implemented on the user device side, for example, a processing device in a local area network composed of certain user devices. As an application example, when multiple users form a network for game operations, the game controller can be used as the user's data acquisition device, and the game console can implement the central processing device.

[0032] The processing circuit 110 includes a comparison unit 111, a determination unit 113, and a notification unit 115. It should be noted that although the estimation unit 111, the determination unit 113, and the notification unit 115 are shown in the form of functional blocks in the drawings, it should be understood that the functions of these units can also be implemented by the processing circuit 110 as a whole, rather than necessarily by discrete actual components in the processing circuit 110. Additionally, although the processing circuit 110 is shown as a single block in the figure, the electronic device 100 can include multiple processing circuits, and the functions of the estimation unit 111, the determination unit 113, and the notification unit 115 can be distributed to multiple processing circuits, so that these functions are executed by the coordinated operation of multiple processing circuits.

[0033] The comparison unit 111 is configured to compare the environmental data acquired by the user device with reference data.

[0034] Environmental data refers to data related to the environment where the user device is located. By way of example, environmental data can include image data, sound data, video data, odor data, pressure data, magnetic field data, tactile data, etc., but the present invention is not limited thereto.

[0035] Environmental data can be obtained from the user device in various ways. For example, the user device can directly send the previously acquired or real-time acquired environmental data to the information processing device 100, or can provide the environmental data to the information processing device 100 through the forwarding of other devices.

[0036] The reference data can include pre-stored historical data (which can be acquired by the same user or different users). Alternatively, the reference data can be environmental data acquired in real time by another user device, for example. The reference data can, for example, correspond to the same or similar scenarios or objects as the environmental data acquired by the user device.

[0037] The determination unit 113 is configured to determine an adjustment to the acquisition method of the environmental data based on the comparison result of the comparison unit 111.

[0038] By way of example, the adjustment to the acquisition method of the environmental data can include changing the position, angle, parameter settings, etc. of the user device that acquires the environmental data.

[0039] According to one embodiment, the adjustment determined by the determination unit 113 may include changing the data acquisition parameters for acquiring environmental data.

[0040] More specifically, the determination unit 113 may determine, based on the user evaluations obtained from the previous environmental data, the data acquisition parameters corresponding to the environmental data with high evaluations as the target parameters.

[0041] The notification unit 115 is configured to perform control to notify the user equipment of the indication information related to the adjustment determined by the determination unit 113.

[0042] Taking image data as an example, the indication information related to the adjustment may include instructing the user to perform a new capture at a capture position, capture angle, and capture parameters (such as aperture, etc.) different from the previous capture position, capture angle, and capture parameters, so as to improve the quality or effect of the capture, or provide an image of the desired position or angle of the object or scene to obtain a more complete and comprehensive image of the object or scene.

[0043] As Figure 2 shown, in addition to including a comparison unit 211, a determination unit 213, and a notification unit 215 that are respectively similar to the comparison unit 111, the determination unit 113, and the notification unit 115 described above with reference to Figure 1 the processing circuit 210 of the information processing device 200 according to one embodiment may further include an identification unit 211.

[0044] The identification unit 211 may be configured to identify environmental data associated with the same scene from the environmental data acquired by one or more user devices.

[0045] According to a specific embodiment, the identification unit 211 may be configured to cluster the environmental data acquired by one or more user devices, and identify the environmental data clustered into the same group as being associated with the same scene.

[0046] The identification unit 211 may, for example, based on the features extracted from the environmental data (taking image data as an example, texture features, shape features, color features, etc. may be extracted), and based on the extracted features, use known clustering methods to cluster multiple environmental data into several groups.

[0047] In addition to based on the features of the environmental data itself, according to one embodiment, the identification unit 211 may be configured to identify environmental data associated with the same scene based on the acquisition time, acquisition location, etc. of the environmental data. The acquisition time, acquisition location, etc. of the environmental data may, for example, be provided to the information processing device according to this embodiment by the user device that acquires the environmental data together with the environmental data.

[0048] Accordingly, an information processing apparatus according to an embodiment may be configured to acquire one or more of the following information from a user device: the time when environmental data is acquired; the location of the user device when the environmental data is acquired; and the orientation of the user device when the environmental data is acquired.

[0049] Taking the example where the user device is implemented as a smart phone, the time when environmental data is acquired may be determined by the clock of the smart phone, the location of the user device when the environmental data is acquired may be determined by the GPS (Global Positioning System) unit of the smart phone, and the orientation of the user device when the environmental data is acquired may be determined by the gyro sensor of the smart phone.

[0050] As Figure 2 shown, according to an embodiment, the processing circuit 210 of the information processing apparatus 200 may further include a fusion unit 223.

[0051] The fusion unit 223 may be configured to fuse environmental data recognized by the recognition unit 211 as being associated with the same scene.

[0052] As an example, in the case where the environmental data includes image data, the fusion unit 223 may generate a three-dimensional image or a panoramic image of the scene based on the image data recognized as being associated with the same scene. Alternatively, the fusion unit 223 may generate a panoramic video or a stereoscopic video of the scene based on the video data recognized as being associated with the same scene.

[0053] For the example of generating a three-dimensional image, for example, images of the same object or scene at different positions and angles (which may be acquired by the same user device or different user devices) may be used as the left-eye view and the right-eye view respectively, so as to form a three-dimensional stereoscopic view of the object or scene.

[0054] For the example of generating a panoramic image, for example, images of the same object or scene at different positions and angles (which may be acquired by the same user device or different user devices) may be used to generate an image with a larger viewing angle of the object or scene through existing stitching or fusion techniques.

[0055] As another example, the fusion unit 223 may generate virtual reality data or augmented reality data of the scene based on the environmental data recognized as being associated with the same scene. The virtual reality data may include not only image data, but also the aforementioned sound data, video data, smell data, pressure data, magnetic field data, tactile data, etc. It should be noted that with the development of sensing technologies and presentation technologies, etc., the environmental data used to generate, for example, virtual reality data may include more aspects not mentioned in this article.

[0056] In addition, according to one embodiment, the fusion unit 223 may be configured to select candidate data from the environmental data associated with the same scene based on data quality, and perform fusion using the selected candidate data. Thereby, it is beneficial to further improve the quality of the fused data.

[0057] As Figure 2 shown, according to one embodiment, the processing circuit 210 of the information processing device 200 may further include a reward unit 225. The reward unit 225 is configured to perform control to give a reward to a user who provides environmental data for fusion.

[0058] As an example, the reward may include virtual currency.

[0059] As Figure 2 shown, according to one embodiment, the processing circuit 210 of the information processing device 200 may further include an access unit 227. The access unit 227 is configured to perform control to provide the user device with access to the fused environmental data.

[0060] For example, the user may be allowed to access the generated three-dimensional image, panoramic image, or virtual reality data after fusion. In addition, for example, the user who accesses the fused data may be required to pay a corresponding fee, such as virtual currency, etc.

[0061] In addition, for certain events, such as car accidents, the appearance of public figures, performances, etc., many people may use their mobile phones to take pictures and share them on various media for others to watch at the same time. In order to improve the transmission efficiency when multiple user devices upload data simultaneously, the information processing device according to one embodiment can receive the environmental data collaboratively uploaded by users.

[0062] As Figure 3 shown, in addition to including a comparison unit 311, a determination unit 313, and a notification unit 315 similar to the comparison unit 111, determination unit 113, and notification unit 115 described above with reference to Figure 1 the processing circuit 310 of the information processing device 300 according to one embodiment may further include a receiving unit 321.

[0063] The receiving unit 321 is configured to perform control to receive the environmental data collaboratively sent by two or more user devices. In this collaborative sending, one user device forwards the environmental data received from another user device using proximity services communication.

[0064] As an example, the following method may be referred to for discovering cooperative terminals.

[0065] Method A

[0066] When the terminal device has data to upload, due to the low network connection speed of the terminal itself or the lack of network connection, the terminal device can search for nearby devices with storage functions and capable of providing network connections either currently or at a certain time in the future, which are called auxiliary devices. The specific search method can be implemented through beacon signaling. The terminal device (source terminal) can periodically send distributed transmission terminal request signaling, and the user can preset the selection conditions for its auxiliary terminal according to its application and carry the selection conditions in the distributed transmission terminal request signaling and send it to other terminals. For these selection conditions, for example, the user sets that the network speed of the auxiliary terminal must be greater than 1 Mbits / S, and if it cannot provide a network connection immediately, then on average it must provide a network connection within 30 minutes according to historical records. According to these conditions, the distributed transmission control device on the nearby terminal device judges whether it meets the conditions set by the user to participate in the distributed transmission based on the historical information of the device network connection.

[0067] If the nearby auxiliary terminal device that receives the request information meets the conditions set by the user, it sends a corresponding feedback signaling. Among them, the feedback information can include but is not limited to at least one of the following: the memory capacity of the auxiliary device, network connection statistical information.

[0068] Mode B

[0069] When the terminal has data to upload, it broadcasts a signaling to notify nearby devices that the data of this user device can be read.

[0070] After finding the collaborative users, transmission settings can be performed. The setting of transmission parameters can be completed by the collaboration control device, which can be set on the public network side or installed on the terminal. When performing transmission settings, the part of the service of the source terminal that is completed by the collaborative user is set according to the external network and internal network resources of the source terminal and the collaborative terminal. In addition, the transmission settings also include various settings for the collaborative user to communicate with other terminals using the internal network after completing the transmission, such as, including route selection, transmission time (the timing of each terminal's transmission), etc.

[0071] In addition, as Figure 3 shown, according to an embodiment, the processing circuit 310 may further include a reward unit 323, configured to perform control to give rewards to users who have forwarded. For example, the rewards can include virtual currency.

[0072] Next, with reference to Figure 11 the processing procedure according to an exemplary embodiment is described as a summary of various aspects of the above embodiments. It should be noted that the embodiments of the present invention do not necessarily need to include all aspects included in this exemplary embodiment.

[0073] First, the user takes a picture of an event and transmits the captured image to the information processing device (S1). Among them, for example, by using the positioning function of the user device, location and camera direction information can be provided along with the image. In addition, the user can provide information to indicate whether the image is to participate in information fusion.

[0074] Preferably, after receiving the data, the remote server can determine the transmission speed of the user device.

[0075] The processing device can determine whether multiple users are involved in photographing the same event based on information such as the time, location, and direction of the data sent by the user, and group the users (S2).

[0076] The processing device collects the information collected by the users and determines whether the results of these information collections meet the requirements (S3), such as data volume, signal-to-noise ratio, etc.

[0077] When a single user is collecting data at a certain location or when multiple users are collecting data but not participating in information fusion, preferably, based on the information collection method and collection results stored in the historical information, it can be determined whether the current user's collection method can be improved. For example, the historical information records that a certain user's camera at a certain location took pictures with what aperture and received the most favorable comments online. The processing device can use such photo-taking information as a reference for setting the best shooting parameters for the current user to take pictures.

[0078] When multiple users are involved in data collection, the remote server can determine which users are photographing the same event based on the images and geographical location information captured by different users. For example, deep learning algorithms are used to classify the photos, and the users who photograph the same event are grouped together.

[0079] The remote server selects the users who need to upload videos among the users who have taken pictures around the same event. For example, the selected users have a relatively large non-overlapping part in the captured images. This judgment can be made using existing image processing algorithms.

[0080] Preferably, among the users who do not need to upload data, those with relatively fast transmission speeds can be selected, and these users can be controlled to be locally networked with the devices of those users who need to upload. For example, each user who does not need to upload is connected to a user who needs to upload to form a D2D connection (using the 3GPP D2D network or the WiFi network).

[0081] The users who need to upload images can upload part of the captured images through the users connected to them, and the remote server merges the data.

[0082] The remote server can perform video image stitching on the received images with different shooting angles according to the shooting angles to form a panoramic image. For example, existing image processing matching algorithms can be utilized.

[0083] The processing device sends the reference suggestions set by the shooting camera to the user so that the user can adjust the camera to optimize the shooting effect (S4).

[0084] Preferably, shooting suggestions are sent to the user according to the images shot by the user to improve the effect of the panoramic image. By using the algorithms of image processing, the central management device can confirm the overlapping parts of the images and then determine how to adjust the shooting so as to cover the largest range. For example, Figures 12A to 12E An example is shown in which videos of the same scene are acquired by multiple user devices and a panoramic video of the scene is generated based on the videos acquired by the multiple user devices. As Figure 12A shown, multiple user devices shoot the same scene at multiple positions and angles. According to the overlapping situation of the images, the central processing device may wish a certain user device to adjust the shooting position or angle so as to shoot the areas not shot by other user devices. For example, when it is determined that the shooting angles of two user devices are relatively similar (as Figure 12B shown), one of the user devices can be made to display indication information, such as Figure 12C the arrow shown on the screen in Figure 12D to notify the user to move the user device. The user can move the user device based on the indication information to shoot the parts of the scene not shot by other user devices, as Figure 12E shown. Thus, the central processing device can fuse the videos shot by multiple user devices to generate a panoramic video or image of the scene, as

[0085] In addition, after the central processing device completes the image fusion, it can publish the final video image on the network. The user can access the network to view the shooting content. For example, viewing the content requires payment with virtual currency. The users participating in the collaborative shooting and data transmission can obtain virtual currency after the central processing device fuses the images for use in payment when viewing the video.

[0086] In the foregoing description of the information processing device according to the embodiments of the present disclosure, some processes and methods are also clearly disclosed. Next, without repeating the details already described above, a description of the information processing method according to the embodiments of the present disclosure is given.

[0087] As Figure 4 shown, the information processing method according to one embodiment includes the following steps:

[0088] S410, compare the environmental data acquired by the user equipment with the reference data;

[0089] S420, based on this comparison, determine the adjustment of the acquisition method of the environmental data; and

[0090] S430, notify the user equipment of the indication information related to this adjustment.

[0091] According to another embodiment, the information processing method may include the following steps: receiving environmental data related to a certain scenario acquired by multiple user equipments, where the environmental data acquired by the multiple user equipments are at least partially different; determining the scenario; and based on the determined scenario, fusing the environmental data acquired by the multiple user equipments to generate scenario information.

[0092] The scenario information may be, but is not limited to, one or more of the three-dimensional images, panoramic images, panoramic videos, stereoscopic videos, virtual reality data, and augmented reality data based on a certain event or scenario as described above. By fusing various environmental data, rich and complete event / scenario information can be obtained, and then this rich and complete event / scenario information can be applied to various fields, such as hot event publishing, multi-user virtual games, etc.

[0093] In addition, the embodiments of the present disclosure also include an electronic device on the user equipment side.

[0094] As Figure 5 shown, an electronic device 500 for the user equipment side according to an embodiment includes a processing circuit 510, and the processing circuit 510 includes an acquisition unit 511, a sending unit 513, and a receiving unit 515.

[0095] The acquisition unit 511 is configured to be controlled to acquire environmental data. As described above, the environmental data may include, for example, image data, sound data, video data, smell data, pressure data, magnetic field data, touch data, etc., but the present invention is not limited thereto. The environmental data may be acquired through sensors provided on the user equipment, but the present invention is not limited thereto. For example, some environmental data may also be provided by the user's own perception and description.

[0096] The sending unit 513 is configured to be controlled to send the acquired environmental data to a control node (such as the information processing device described above).

[0097] The receiving unit 515 is configured to be controlled to receive from the control node the indication information related to the adjustment of the acquisition method of the environmental data. This adjustment is determined based on the comparison between the environmental data and the reference data.

[0098] In addition, according to one embodiment, the receiving unit 515 may be configured to perform control to cooperate with other user devices to transmit environmental data. In this cooperative transmission, one user device forwards the environmental data received from another user device using proximity services communication.

[0099] In addition, according to one embodiment, the receiving unit 515 may be configured to perform control to report to a control node or broadcast a message indicating a predetermined event to other user devices.

[0100] Figure 6 A configuration example of an electronic device on the user device side according to another embodiment is shown. Figure 6 The processing unit of the illustrated electronic device 600 includes, in addition to the acquisition unit 611, the transmission unit 613, and the receiving unit 615 similar to the acquisition unit 511, the transmission unit 513, and the receiving unit 515, a presentation unit 617.

[0101] The presentation unit 617 is configured to perform control to present the indication information received by the receiving unit 615. The presentation unit 617 can present the indication information received by the receiving unit 615 by, but not limited to, images, text, voice, etc.

[0102] Figure 13 An example of the presented indication information is shown. As Figure 13 shown, while shooting, a zoom indication and a rotation indication are displayed on the screen of the user device. In addition, indication information corresponding to various adjustment methods such as zoom in or out, pan (e.g., up, down, left, right, etc.), rotation (e.g., rotation about the horizontal axis, rotation about the vertical axis, etc.) can also be displayed on the screen.

[0103] Figure 7 A configuration example of an electronic device on the user device side according to another embodiment is shown. Figure 7 The processing unit of the illustrated electronic device 700 includes, in addition to the acquisition unit 711, the transmission unit 713, and the receiving unit 715 similar to the acquisition unit 511, the transmission unit 513, and the receiving unit 515, an indication unit 717.

[0104] The indication unit 717 is configured to perform control to indicate to a control node whether environmental data is to participate in fusion.

[0105] Figure 8 A process example of an information processing method on the user device side according to one embodiment is shown, and the process includes:

[0106] S810, acquiring environmental data;

[0107] S820, sending the acquired environmental data to a control node; and

[0108] S830 receives indication information related to an adjustment of a manner of acquiring environmental data from a control node, where the adjustment is determined based on a comparison between the environmental data and reference data.

[0109] In addition, an embodiment of the present disclosure further includes a computer-readable medium, which includes executable instructions that, when executed by an information processing device, cause the information processing device to execute the method according to the above embodiment.

[0110] Figure 9 An information processing device 900 according to an embodiment is shown, which includes: a comparison device 910 configured to compare environmental data acquired by a user device with reference data; a determination device 920 configured to determine an adjustment of a manner of acquiring environmental data based on the comparison; and a notification device 930 configured to control to notify the user device of indication information related to the adjustment.

[0111] Figure 10 An information processing device 1000 according to an embodiment is shown, which includes: an acquisition device 1010 configured to acquire environmental data; a sending device 1020 configured to send the acquired environmental data to a control node; and a receiving device 1030 configured to receive indication information related to an adjustment of a manner of acquiring environmental data from the control node, where the adjustment is determined based on a comparison between the environmental data and reference data.

[0112] As an example, each step of the above method and each component module and / or unit of the above device may be implemented as software, firmware, hardware, or a combination thereof. In the case of implementation by software or firmware, a program constituting software for implementing the above method may be installed from a storage medium or a network into a computer having a dedicated hardware structure (such as Figure 14 the general computer 2000 shown), and when various programs are installed in this computer, it can execute various functions and the like.

[0113] In Figure 14 , an arithmetic processing unit (i.e., CPU) 2001 executes various processes according to a program stored in a read-only memory (ROM) 2002 or a program loaded from a storage section 2008 into a random access memory (RAM) 2003. In the RAM 2003, data required when the CPU 2001 executes various processes and the like is also stored as needed. The CPU 2001, the ROM 2002, and the RAM 2003 are linked to each other via a bus 2004. An input / output interface 2005 is also linked to the bus 2004.

[0114] The following components are linked to the input / output interface 2005: an input section 2006 (including a keyboard, a mouse, etc.), an output section 2007 (including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.), a storage section 2008 (including a hard disk, etc.), and a communication section 2009 (including a network interface card such as a LAN card, a modem, etc.). The communication section 2009 performs communication processing via a network such as the Internet. As needed, a drive 2010 may also be linked to the input / output interface 2005. A removable medium 2011 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is mounted on the drive 2010 as needed, so that a computer program read therefrom is installed in the storage section 2008 as needed.

[0115] In the case where the above-described series of processes are implemented by software, a program constituting the software is installed from a network such as the Internet or a storage medium such as the removable medium 2011.

[0116] Those skilled in the art should understand that such a storage medium is not limited to Figure 14 the removable medium 2011 shown in which a program is stored and distributed separately from the device to provide the program to the user. Examples of the removable medium 2011 include a magnetic disk (including a floppy disk (registered trademark)), an optical disk (including a compact disc read only memory (CD-ROM) and a digital versatile disc (DVD)), a magneto-optical disk (including a mini disc (MD) (registered trademark)), and a semiconductor memory. Alternatively, the storage medium may be a ROM 2002, a hard disk included in the storage section 2008, etc., in which a program is stored and distributed to the user together with the device including them.

[0117] An embodiment of the present invention also relates to a program product storing machine-readable instruction codes. When the instruction codes are read and executed by a machine, the method according to the embodiment of the present invention described above can be executed.

[0118] Accordingly, a storage medium for carrying the program product storing the machine-readable instruction codes described above is also included in the disclosure of the present invention. The storage medium includes but is not limited to a floppy disk, an optical disk, a magneto-optical disk, a memory card, a memory stick, etc.

[0119] Embodiments of the present application also relate to the following electronic devices. When the electronic device is used on the base station side, the electronic device can be implemented as any type of gNB, evolved Node B (eNB), such as a macro eNB and a small eNB. A small eNB can be an eNB that covers a cell smaller than a macro cell, such as a pico eNB, a micro eNB, and a home (femto) eNB. Instead, the electronic device can be implemented as any other type of base station, such as a NodeB and a base transceiver station (BTS). The electronic device can include: a main body (also referred to as a base station device) configured to control wireless communication; and one or more remote radio heads (RRHs) provided in a place different from the main body. In addition, various types of terminals described below can all operate as a base station by temporarily or semi-persistently executing base station functions.

[0120] When the electronic device is used on the user equipment side, it can be implemented as a mobile terminal (such as a smart phone, a tablet personal computer (PC), a notebook PC, a portable game terminal, a portable / dongle-type mobile router, and a digital imaging device) or a vehicle-mounted terminal (such as a car navigation device). In addition, the electronic device can be a wireless communication module (such as an integrated circuit module including a single or multiple wafers) installed on each of the above terminals.

[0121] [Examples of Application of Terminal Devices]

[0122] Figure 15 FIG. is a block diagram showing an example of a schematic configuration of a smart phone 2500 to which the technology of the present disclosure can be applied. The smart phone 2500 includes a processor 2501, a memory 2502, a storage device 2503, an external connection interface 2504, a camera device 2506, a sensor 2507, a microphone 2508, an input device 2509, a display device 2510, a speaker 2511, a wireless communication interface 2512, one or more antenna switches 2515, one or more antennas 2516, a bus 2517, a battery 2518, and an auxiliary controller 2519.

[0123] The processor 2501 can be, for example, a CPU or a system on chip (SoC), and controls the functions of the application layer and other layers of the smart phone 2500. The memory 2502 includes a RAM and a ROM, and stores data and programs executed by the processor 2501. The storage device 2503 can include a storage medium, such as a semiconductor memory and a hard disk. The external connection interface 2504 is an interface for connecting an external device (such as a memory card and a universal serial bus (USB) device) to the smart phone 2500.

[0124] The imaging device 2506 includes an image sensor (such as a charge-coupled device (CCD) and a complementary metal-oxide semiconductor (CMOS)), and generates a captured image. The sensor 2507 may include a set of sensors, such as a measurement sensor, a gyro sensor, a geomagnetic sensor, and an acceleration sensor. The microphone 2508 converts the sound input to the smart phone 2500 into an audio signal. The input device 2509 includes, for example, a touch sensor configured to detect a touch on the screen of the display device 2510, a keypad, a keyboard, a button, or a switch, and receives operations or information input from the user. The display device 2510 includes a screen (such as a liquid crystal display (LCD) and an organic light-emitting diode (OLED) display), and displays the output image of the smart phone 2500. The speaker 2511 converts the audio signal output from the smart phone 2500 into sound.

[0125] The wireless communication interface 2512 supports any cellular communication scheme (such as LTE and LTE-Advanced), and performs wireless communication. The wireless communication interface 2512 generally may include, for example, a baseband (BB) processor 2513 and a radio frequency (RF) circuit 2514. The BB processor 2513 may perform, for example, encoding / decoding, modulation / demodulation, and multiplexing / demultiplexing, and perform various types of signal processing for wireless communication. At the same time, the RF circuit 2514 may include, for example, a mixer, a filter, and an amplifier, and transmit and receive wireless signals via the antenna 2516. The wireless communication interface 2512 may be one chip module on which the BB processor 2513 and the RF circuit 2514 are integrated. As Figure 15 shown, the wireless communication interface 2512 may include multiple BB processors 2513 and multiple RF circuits 2514. Although Figure 15 an example in which the wireless communication interface 2512 includes multiple BB processors 2513 and multiple RF circuits 2514 is shown, the wireless communication interface 2512 may also include a single BB processor 2513 or a single RF circuit 2514.

[0126] In addition, in addition to the cellular communication scheme, the wireless communication interface 2512 may support other types of wireless communication schemes, such as a short-range wireless communication scheme, a near-field communication scheme, and a wireless local area network (LAN) scheme. In this case, the wireless communication interface 2512 may include a BB processor 2513 and an RF circuit 2514 for each wireless communication scheme.

[0127] Each of the antenna switches 2515 switches the connection destination of the antenna 2516 among a plurality of circuits included in the wireless communication interface 2512 (for example, circuits for different wireless communication schemes).

[0128] Each of the antennas 2516 includes single or multiple antenna elements (such as multiple antenna elements included in a MIMO antenna), and is used for the wireless communication interface 2512 to transmit and receive wireless signals. As Figure 15 shown, the smart phone 2500 may include multiple antennas 2516. Although Figure 15 an example where the smart phone 2500 includes multiple antennas 2516 is shown, the smart phone 2500 may also include a single antenna 2516.

[0129] In addition, the smart phone 2500 may include an antenna 2516 for each wireless communication scheme. In this case, the antenna switch 2515 may be omitted from the configuration of the smart phone 2500.

[0130] The bus 2517 connects the processor 2501, the memory 2502, the storage device 2503, the external connection interface 2504, the imaging device 2506, the sensor 2507, the microphone 2508, the input device 2509, the display device 2510, the speaker 2511, the wireless communication interface 2512, and the auxiliary controller 2519 to each other. The battery 2518 supplies power to Figure 15 the respective blocks of the smart phone 2500 shown via a feeder line, which is partially shown as a dashed line in the figure. The auxiliary controller 2519 operates the minimum necessary functions of the smart phone 2500, for example, in the sleep mode.

[0131] In Figure 15 the smart phone 2500 shown, the transceiver device of the device on the user equipment side according to an embodiment of the present invention may be implemented by the wireless communication interface 2512. At least a part of the processing circuit and / or the functions of each unit of the electronic device or information processing device on the user equipment side according to an embodiment of the present invention may also be implemented by the processor 2501 or the auxiliary controller 2519. For example, the power consumption of the battery 2518 may be reduced by the auxiliary controller 2519 executing part of the functions of the processor 2501. In addition, the processor 2501 or the auxiliary controller 2519 may execute at least a part of the processing circuit and / or the functions of each unit of the electronic device or information processing device on the user equipment side according to an embodiment of the present invention by executing a program stored in the memory 2502 or the storage device 2503.

[0132] [Application Example Regarding Base Station]

[0133] Figure 16 is a block diagram showing an example of a schematic configuration of a base station, for example, an evolved Node B (eNB), to which the technology of the present disclosure can be applied. The eNB 2300 includes one or more antennas 2310 and a base station device 2320. The base station device 2320 and each antenna 2310 may be connected to each other via a radio frequency (RF) cable.

[0134] Each of the antennas 2310 includes single or multiple antenna elements (such as multiple antenna elements included in a multiple-input multiple-output (MIMO) antenna), and is used to transmit and receive wireless signals for the base station device 2320. As Figure 16 shown, the eNB 2300 may include multiple antennas 2310. For example, the multiple antennas 2310 may be compatible with multiple frequency bands used by the eNB 2300. Although Figure 16 an example where the eNB 2300 includes multiple antennas 2310 is shown, the eNB 2300 may also include a single antenna 2310.

[0135] The base station device 2320 includes a controller 2321, a memory 2322, a network interface 2323, and a wireless communication interface 2325.

[0136] The controller 2321 may be, for example, a CPU or a DSP, and operates various functions at the higher layers of the base station device 2320. For example, the controller 2321 generates data packets based on the data in the signals processed by the wireless communication interface 2325, and transfers the generated packets via the network interface 2323. The controller 2321 may bundle data from multiple baseband processors to generate a bundled packet, and transfer the generated bundled packet. The controller 2321 may have a logical function to perform controls such as radio resource control, radio bearer control, mobility management, admission control, and scheduling. This control may be performed in conjunction with a nearby eNB or a core network node. The memory 2322 includes a RAM and a ROM, and stores programs executed by the controller 2321 and various types of control data (such as a terminal list, transmission power data, and scheduling data).

[0137] The network interface 2323 is a communication interface for connecting the base station device 2320 to the core network 2324. The controller 2321 may communicate with a core network node or another eNB via the network interface 2323. In this case, the eNB 2300 and the core network node or other eNBs may be connected to each other through logical interfaces (such as the S1 interface and the X2 interface). The network interface 2323 may also be a wired communication interface or a wireless communication interface for a wireless backhaul line. If the network interface 2323 is a wireless communication interface, compared with the frequency band used by the wireless communication interface 2325, the network interface 2323 may use a higher frequency band for wireless communication.

[0138] The wireless communication interface 2325 supports any cellular communication scheme (such as Long Term Evolution (LTE) and LTE-Advanced), and provides a wireless connection to terminals in the cell located at the eNB 2300 via the antenna 2310. The wireless communication interface 2325 generally may include, for example, a BB processor 2326 and an RF circuit 2327. The BB processor 2326 may perform, for example, encoding / decoding, modulation / demodulation, and multiplexing / demultiplexing, and perform various types of signal processing of layers (such as L1, Medium Access Control (MAC), Radio Link Control (RLC), and Packet Data Convergence Protocol (PDCP)). Instead of the controller 2321, the BB processor 2326 may have a part or all of the above-described logical functions. The BB processor 2326 may be a memory storing a communication control program, or a module including a processor configured to execute a program and related circuits. The update program may change the functions of the BB processor 2326. The module may be a card or blade inserted into a slot of the base station device 2320. Alternatively, the module may also be a chip mounted on a card or blade. At the same time, the RF circuit 2327 may include, for example, mixers, filters, and amplifiers, and transmit and receive wireless signals via the antenna 2310.

[0139] As Figure 16 shown, the wireless communication interface 2325 may include a plurality of BB processors 2326. For example, the plurality of BB processors 2326 may be compatible with a plurality of frequency bands used by the eNB 2300. As Figure 16 shown, the wireless communication interface 2325 may include a plurality of RF circuits 2327. For example, the plurality of RF circuits 2327 may be compatible with a plurality of antenna elements. Although Figure 16 illustrates an example in which the wireless communication interface 2325 includes a plurality of BB processors 2326 and a plurality of RF circuits 2327, the wireless communication interface 2325 may also include a single BB processor 2326 or a single RF circuit 2327.

[0140] In Figure 16 the eNB 2300 shown, the transceiver device of the base station-side device according to an embodiment of the present invention may be implemented by the wireless communication interface 2325. At least a part of the processing circuit and / or the functions of each unit of the base station-side electronic device or information processing device according to an embodiment of the present invention may also be implemented by the controller 2321. For example, the controller 2321 may execute at least a part of the processing circuit and / or the functions of each unit of the base station-side electronic device or information processing device according to an embodiment of the present invention by executing a program stored in the memory 2322.

[0141] In the above description of the specific embodiments of the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, combined with features in other embodiments, or substituted for features in other embodiments.

[0142] It should be emphasized that the term "comprising / including" as used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0143] In the above embodiments and examples, reference numerals composed of numbers are used to represent each step and / or unit. Those of ordinary skill in the art should understand that these reference numerals are only for the convenience of narration and drawing, and do not represent their order or any other limitation.

[0144] In addition, the method of the present invention is not limited to being executed in the time sequence described in the specification, and can also be executed in other time sequences, in parallel or independently. Therefore, the execution sequence of the method described in this specification does not limit the technical scope of the present invention.

[0145] Although the present invention has been disclosed above through the description of the specific embodiments of the present invention, it should be understood that all the above embodiments and examples are exemplary, not restrictive. Those skilled in the art can design various modifications, improvements or equivalents of the present invention within the spirit and scope of the appended claims. These modifications, improvements or equivalents should also be considered to be included within the protection scope of the present invention.

[0146] In addition, the present technology can also be configured as follows.

[0147] (1) An information processing device, comprising:

[0148] A processing circuit, the processing circuit being configured to

[0149] Compare the environmental data acquired by the user device with reference data;

[0150] Based on the comparison, determine an adjustment to the acquisition method of the environmental data; and

[0151] Perform control to notify the user device of indication information related to the adjustment.

[0152] (2) The information processing device according to (1), wherein the environmental data includes one or more of image data, sound data, video data, odor data, pressure data, magnetic field data, and touch data.

[0153] (3) The information processing device according to (1), wherein the reference data includes pre-stored historical data and / or environmental data acquired by another user device.

[0154] (4) The information processing device according to (1), wherein the adjustment includes: changing the position, angle, and / or parameter settings of the user device that acquires the environmental data.

[0155] (5) The information processing device according to (1), wherein the adjustment includes: changing the data acquisition parameters for acquiring the environmental data.

[0156] (6) The information processing device according to (5), wherein the determination of the adjustment includes:

[0157] Based on the user evaluation obtained from the previous environmental data, determining the data acquisition parameters corresponding to the environmental data with high evaluation as the target parameters.

[0158] (7) The information processing device according to (1), the processing circuit is further configured to:

[0159] Identify environmental data associated with the same scene from the environmental data acquired by one or more user devices.

[0160] (8) The information processing device according to (7), wherein the identification includes:

[0161] Cluster the environmental data acquired by one or more user devices, and identify the environmental data clustered into the same group as being associated with the same scene.

[0162] (9) The information processing device according to (7), wherein the identification includes:

[0163] Identify environmental data associated with the same scene based on the acquisition time and / or acquisition location of the environmental data.

[0164] (10) The information processing device according to (1), the processing circuit is further configured to acquire:

[0165] The time when the environmental data is acquired;

[0166] The position of the user device when the environmental data is acquired; and / or

[0167] The orientation of the user device when the environmental data is acquired.

[0168] (11) The information processing device according to (7), the processing circuit is further configured to: fuse the environmental data identified as being associated with the same scene.

[0169] (12) The information processing device according to (11), wherein the processing circuit is configured to:

[0170] select candidate data from the environmental data associated with the same scene based on data quality; and

[0171] perform the fusion using the selected candidate data.

[0172] (13) The information processing device according to (11), wherein the fusion includes:

[0173] generate a three-dimensional image or a panoramic image of the scene based on the image data recognized as being associated with the same scene;

[0174] generate a panoramic video or a stereoscopic video of the scene based on the video data recognized as being associated with the same scene; or

[0175] generate virtual reality data or augmented reality data of the scene based on the environmental data recognized as being associated with the same scene.

[0176] (14) The information processing device according to (11), the processing circuit is further configured to: perform control to give a reward to a user who provides the environmental data for the fusion.

[0177] (15) The information processing device according to (11), wherein the processing circuit is further configured to: perform control to provide access to the fused environmental data to a user device.

[0178] (16) The information processing device according to (1), the processing circuit is configured to perform control to receive environmental data jointly sent by two or more user devices,

[0179] in the joint sending, one user device forwards the environmental data received from another user device using proximity services communication.

[0180] (17) The information processing device according to (16), the processing circuit is further configured to perform control to give a reward to the user who performs the forwarding.

[0181] (18) The information processing device according to (14) or (17), wherein the reward includes virtual currency.

[0182] (19) An information processing method, comprising:

[0183] receive environmental data related to a certain scene acquired by a plurality of user devices, and the environmental data acquired by the plurality of user devices is at least partially different;

[0184] Determine the scenario; and

[0185] Based on the determined scenario, fuse the environmental data acquired by the multiple user devices to generate scenario information.

[0186] (20) An electronic device for the user equipment side, comprising:

[0187] A processing circuit configured to control to

[0188] Acquire environmental data;

[0189] Send the acquired environmental data to a control node; and

[0190] Receive, from the control node, indication information related to an adjustment of the acquisition manner of the environmental data, where the adjustment is determined based on a comparison between the environmental data and reference data.

[0191] (21) The electronic device according to (20), wherein the processing circuit is further configured to control to present the indication information.

[0192] (22) The electronic device according to (20), wherein the processing circuit is further configured to control to indicate to the control node whether the environmental data is to participate in the fusion.

[0193] (23) The electronic device according to (20), wherein the processing circuit is further configured to control to cooperate with other user devices to send environmental data,

[0194] In the cooperative sending, one user device forwards the environmental data received from another user device by using proximity services communication.

[0195] (24) The electronic device according to (20), wherein the processing circuit is further configured to control to report to the control node or broadcast to other user devices a message indicating a predetermined event.

[0196] (25) An information processing method for the user equipment side, comprising:

[0197] Acquire environmental data;

[0198] Send the acquired environmental data to a control node; and

[0199] Receive, from the control node, indication information related to an adjustment of the acquisition manner of the environmental data, where the adjustment is determined based on a comparison between the environmental data and reference data.

[0200] (26) A computer-readable medium comprising executable instructions that, when executed by an information processing device, cause the information processing device to perform the method according to (19) or (25).

Claims

1. An information processing device, comprising: a processing circuit configured to compare environmental data acquired by a user device with reference data; based on the comparison, determine an adjustment to the acquisition method of the environmental data; and perform control to notify the user device of indication information related to the adjustment.

2. The information processing device according to claim 1, wherein the environmental data includes one or more of image data, sound data, video data, odor data, pressure data, magnetic field data, and touch data.

3. The information processing device according to claim 1, wherein the reference data includes pre-stored historical data and / or environmental data acquired by another user device.

4. The information processing device according to claim 1, wherein the adjustment includes: changing the position, angle, and / or parameter settings of the user device that acquires the environmental data.

5. The information processing device according to claim 1, wherein the adjustment includes: changing the data acquisition parameters for acquiring the environmental data.

6. The information processing device according to claim 5, wherein the determination of the adjustment includes: based on user evaluations obtained from previous environmental data, determining the data acquisition parameters corresponding to the environmental data that received high evaluations as target parameters.

7. The information processing device according to claim 1, the processing circuit is further configured to: identify environmental data associated with the same scene from environmental data acquired by one or more user devices.

8. The information processing device according to claim 7, wherein the identification includes: clustering environmental data acquired by one or more user devices, and identifying the environmental data clustered into the same group as being associated with the same scene.

9. The information processing device according to claim 7, wherein the identification includes: identifying environmental data associated with the same scene based on the acquisition time and / or acquisition location of the environmental data.

10. The information processing device according to claim 1, the processing circuit is further configured to acquire: the time when the environmental data is acquired; the position of the user device when the environmental data is acquired; and / or the orientation of the user device when the environmental data is acquired.

11. The information processing device according to claim 7, the processing circuit is further configured to: fuse environmental data identified as being associated with the same scene.

12. The information processing device according to claim 11, wherein the processing circuit is configured to: select candidate data from environmental data associated with the same scene based on data quality; and perform the fusion using the selected candidate data.

13. The information processing device according to claim 11, wherein the fusion includes: generating a three-dimensional image or panoramic image of the scene based on the image data identified as being associated with the same scene; generating a panoramic video or stereoscopic video of the scene based on the video data identified as being associated with the same scene; or Generate virtual reality data or augmented reality data of the scene based on environmental data recognized as being associated with the same scene.

14. The information processing apparatus according to claim 11, wherein the processing circuit is further configured to: perform control to give a reward to a user who provides environmental data for the fusion.

15. The information processing apparatus according to claim 11, wherein, the processing circuit is further configured to: perform control to provide access to the fused environmental data to a user device.

16. The information processing apparatus according to claim 1, wherein the processing circuit is configured to perform control to receive environmental data jointly transmitted by two or more user devices, in the joint transmission, one user device forwards environmental data received from another user device using proximity services communication.

17. The information processing apparatus according to claim 16, wherein the processing circuit is further configured to perform control to give a reward to a user who has performed the forwarding.

18. The information processing apparatus according to claim 14 or claim 17, wherein, the reward includes virtual currency.

19. An information processing method, comprising: receiving environmental data related to a certain scene acquired by a plurality of user devices, and the environmental data acquired by the plurality of user devices is at least partially different; determining the scene; and based on the determined scene, fusing the environmental data acquired by the plurality of user devices to generate scene information.

20. An electronic device on the user device side, comprising: a processing circuit, the processing circuit being configured to perform control to acquire environmental data; send the acquired environmental data to a control node; and receive indication information related to an adjustment of the acquisition manner of the environmental data from the control node, wherein the adjustment is determined based on a comparison between the environmental data and reference data.

21. The electronic device according to claim 20, wherein the processing circuit is further configured to perform control to present the indication information.

22. The electronic device according to claim 20, wherein the processing circuit is further configured to perform control to indicate to the control node whether the environmental data is to participate in the fusion.

23. The electronic device according to claim 20, wherein the processing circuit is further configured to perform control to jointly transmit environmental data with other user devices, in the joint transmission, one user device forwards environmental data received from another user device using proximity services communication.

24. The electronic device according to claim 20, wherein the processing circuit is further configured to perform control to report to the control node or broadcast a message indicating a predetermined event to other user devices.

25. An information processing method on the user device side, comprising: acquiring environmental data; sending the acquired environmental data to a control node; and receiving indication information related to an adjustment of the acquisition manner of the environmental data from the control node, wherein the adjustment is determined based on a comparison between the environmental data and reference data.

26. A computer-readable medium comprising executable instructions that, when executed by an information processing device, cause the information processing device to perform the method according to claim 19 or claim 25.

Images