Data processing method and apparatus, intelligent camera, electronic device, and storage medium

Abstract

This invention provides a data processing method, apparatus, smart camera, electronic device, and storage medium. The data processing method, applied to a smart camera, includes: sending a knowledge service request, the knowledge service request including a preset multimedia display mode; receiving knowledge service data, the knowledge service data being generated by calling a knowledge service interface matching the preset multimedia display mode; and displaying the knowledge service data in the preset multimedia display mode. In this embodiment of the invention, because the knowledge service data is generated by calling a knowledge service interface matching the preset multimedia display mode, displaying the knowledge service data in the preset multimedia display mode achieves the display of knowledge service data under a specific mode, thus enhancing its educational attributes.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a data processing method, apparatus, smart camera, electronic device and storage medium. Background Technology

[0002] Most existing smart cameras are not truly smart; they can only take photos, record videos, and review them. Existing technology includes applications based on visual recognition algorithms installed on smart terminals such as smartphones. Users can photograph plants, upload the photos to the cloud for identification, and finally receive a textual description of the plant's encyclopedic information.

[0003] However, there is still room for improvement in the educational aspects of this application. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a data processing method, apparatus, smart camera, electronic device, and storage medium to solve or alleviate the above-mentioned problems.

[0005] According to a first aspect of the present invention, a data processing method is provided, comprising: sending a knowledge service request, the knowledge service request including a preset multimedia display mode; receiving knowledge service data, the knowledge service data being generated by calling a knowledge service interface matching the preset multimedia display mode; and displaying the knowledge service data in the preset multimedia display mode.

[0006] According to a second aspect of the present invention, a data synchronization method is provided, comprising: establishing a network connection with a smart camera on the same network, wherein the smart camera stores captured images; and, in response to the establishment of the network connection, receiving the captured images synchronously transmitted by the smart camera to a local folder.

[0007] According to a third aspect of the present invention, a data processing method is provided, comprising: receiving a knowledge service request, the knowledge service request including a preset multimedia display mode; according to the knowledge service request, invoking a knowledge service interface corresponding to the preset multimedia display mode to generate knowledge service data; and returning the knowledge service data.

[0008] According to a fourth aspect of the present invention, a smart camera is provided, comprising: a request sending module for sending a knowledge service request, the knowledge service request including a preset multimedia display mode; a data receiving module for receiving knowledge service data, the knowledge service data being generated by calling a knowledge service interface matching the preset multimedia display mode; and a data display module for displaying the knowledge service data in the preset multimedia display mode.

[0009] According to a fifth aspect of the present invention, an electronic device is provided, comprising: a connection establishment module for establishing a network connection with a smart camera on the same network, wherein the smart camera stores captured images; and a synchronization transmission module for receiving the captured images from the smart camera and synchronously transmitting them to a local folder in response to the establishment of the network connection.

[0010] According to a sixth aspect of the present invention, a data processing apparatus is provided, comprising: a request receiving module for receiving a knowledge service request, the knowledge service request including a preset multimedia display mode; a data generation module for generating knowledge service data by invoking a knowledge service interface corresponding to the preset multimedia display mode according to the knowledge service request; and a data return module for returning the knowledge service data.

[0011] According to a seventh aspect of the present invention, an electronic device is provided, the device comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, the executable instruction causing the processor to perform an operation corresponding to any one of the first to third aspects.

[0012] According to an eighth aspect of the present invention, a storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method described in any one of the first to third aspects.

[0013] In the embodiment of the present invention, since the knowledge service data is generated by calling the knowledge service interface that matches the preset multimedia display mode, the knowledge service data is displayed in the preset multimedia display mode, thereby realizing the display of knowledge service data in a specific mode and improving its educational attributes. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0015] Figure 1 This is a schematic diagram of a data processing method according to an embodiment of the present invention;

[0016] Figure 2A This is a schematic flowchart illustrating a data processing method according to another embodiment of the present invention.

[0017] Figure 2BThis is a schematic diagram of a data processing method according to another embodiment of the present invention;

[0018] Figure 2C This is a schematic diagram of a data processing method according to another embodiment of the present invention;

[0019] Figure 3A This is a schematic diagram of a data processing method according to another embodiment of the present invention;

[0020] Figure 3B This is a schematic diagram of a data processing method according to another embodiment of the present invention;

[0021] Figure 4A This is a schematic flowchart illustrating a data processing method according to another embodiment of the present invention.

[0022] Figure 4B This is a schematic diagram of a data processing method according to another embodiment of the present invention;

[0023] Figure 5 A schematic block diagram of a smart camera according to another embodiment of the present invention;

[0024] Figure 6 This is a schematic block diagram of an electronic device according to another embodiment of the present invention;

[0025] Figure 7 This is a schematic block diagram of a data processing apparatus according to another embodiment of the present invention;

[0026] Figure 8 The hardware structure of an electronic device is shown in another embodiment of the present invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art should fall within the protection scope of the present invention.

[0028] The specific implementation of the embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0029] Figure 1 This is a schematic diagram of a data processing method according to an embodiment of the present invention. As shown, the mobile terminal 110 and the server 120 can communicate via a network such as the Internet. The mobile terminal 110 includes a shooting module 111, a presentation module 112, and a network module 113. The server 120 includes an identification module 121, a reasoning model 122, and a network module 123.

[0030] Furthermore, the shooting module 111 can be implemented using the camera of the mobile terminal 110. The shooting module 111 can take pictures of target objects, such as specific plants. Network modules 113 and 123 realize data transmission and network communication between the mobile terminal 110 and the server 120. The inference model 122 can perform image recognition, for example, recognizing the image of the specific plant mentioned above. The server 120 returns the recognized plant name information and classification information to the mobile terminal 110 through the network module 123, and the mobile terminal 110 receives the plant name information and classification information through the network module 113.

[0031] Figure 2A This is a schematic flowchart illustrating a data processing method according to another embodiment of the present invention. Figure 2A The data processing method can be applied to any suitable electronic device with data processing capabilities, including but not limited to smart devices such as smart cameras, mobile terminals (such as mobile phones, tablets, etc.), PCs, and devices with shooting or networking capabilities. Figure 2A The methods include:

[0032] 210: Send a knowledge service request, which includes a preset multimedia display mode.

[0033] It should be understood that preset multimedia display modes may include at least one of the following: encyclopedia knowledge display mode, English content (e.g., children's English content) display mode, travel mode, or extended definition mode. Sending a knowledge service request may be in response to user instructions (e.g., voice instructions, gesture instructions, posture instructions, facial expression instructions, physical key instructions, etc.), obtaining locally preset multimedia display mode information, and encapsulating the preset multimedia display mode into the knowledge service request for transmission; or, it may encapsulate information or identifiers indicating the preset multimedia display mode into the knowledge service request for transmission.

[0034] Specifically, after acquiring the captured image, a knowledge service request can be triggered via any of the following methods: camera physical button, screen button, or voice control. This could include, for example, triggering AI (artificial intelligence) recognition. It should also be understood that the knowledge service request may include information related to the display mode. For example, in an encyclopedia knowledge display mode, the captured image can be included in the knowledge service request. Similarly, in an English content display mode, the captured image can be included in the knowledge service request, allowing the identification of the target audience for the English content service. In a travel mode, the location information of the executing device can be included in the knowledge service request.

[0035] 220: Receive knowledge service data. The knowledge service data is generated by calling the knowledge service interface that matches the preset multimedia display mode.

[0036] It should be understood that knowledge service data can be any form of multimedia data, such as at least one of text data, audio data, image data, and video data. The knowledge service interface can directly return knowledge service data, or it can return the raw data of the knowledge service data, and the knowledge service data can be generated based on the raw data. The aforementioned raw data can be a type of multimedia data. For example, knowledge service data can be generated based on raw text data and raw audio data; for example, knowledge service data can be generated from multiple types of data, such as raw text data, raw audio data, raw image data, and raw video data. The knowledge service interface can also call other external interfaces (e.g., external resource servers) to obtain the aforementioned data. This embodiment of the invention does not limit this.

[0037] 230: Display knowledge service data using a preset multimedia display mode.

[0038] It should be understood that text data can be displayed as text, read aloud, etc. Audio data can be displayed as playback or text. Image data can be displayed statically or dynamically at a specific resolution. Video data can be displayed as playback, subtitles, etc. The preset knowledge display modes in this document can include combinations of the above display methods.

[0039] In the embodiment of the present invention, since the knowledge service data is generated by calling the knowledge service interface that matches the preset multimedia display mode, the knowledge service data is displayed in the preset multimedia display mode, thereby realizing the display of knowledge service data in a specific mode and improving its educational attributes.

[0040] In another implementation of the present invention, the knowledge service request further includes image data of the captured image. The knowledge service data is generated by calling a knowledge service interface that matches a preset multimedia display mode based on the recognition result of the image data. The knowledge service data is displayed in the preset multimedia display mode, including: displaying the knowledge service data in the preset multimedia display mode when displaying the captured image.

[0041] Since the knowledge service data is generated by calling the knowledge service interface that matches the preset multimedia display mode based on the recognition results of the image data, the knowledge service data more comprehensively supplements the information of the captured image itself. In addition, when displaying the captured image, the knowledge service data is displayed in the preset multimedia display mode. Therefore, the educational attributes of the captured image are improved by using a more comprehensive display method.

[0042] In one example, the preset multimedia display mode is an encyclopedia multimedia display mode, and the knowledge service data includes text descriptions and / or audio descriptions that match the recognition results of the image data. When displaying the captured image, displaying the knowledge service data in the preset multimedia display mode includes: displaying the text descriptions and / or playing the audio descriptions when displaying the captured image.

[0043] In another example, the preset multimedia display mode is a foreign language multimedia display mode, and the knowledge service data includes foreign language text and / or foreign language audio that match the recognition results of the image data. The step of displaying the knowledge service data in the preset multimedia display mode when displaying the captured image includes: displaying the foreign language text and / or playing the foreign language audio when displaying the captured image.

[0044] Specifically, preprocessed image data can be transmitted over a network to the server. Image algorithms deployed on the server can process image data uploaded from various terminals (e.g., various smart cameras) in parallel. During processing (before obtaining the processing results), no data is returned to the terminals (no interface is provided for terminal access), thus maximizing user privacy and security. The image algorithm processing described in this paper can include object recognition and scene recognition, among others.

[0045] In addition, the knowledge service request can include specific scene or object subcategories and names from the images uploaded by the camera. If the image does not have a clearly defined subject, a scene category can be output, such as street view / office / beach / living room. If the image has a clearly defined subject, such as an animal or building, this step will output a detailed object category and name. The detailed object category can refer to specific object categories such as animal / plant / furniture, etc. The name can refer to the specific name attribute of the object, such as gardenia, rose, table, etc.

[0046] Furthermore, based on the object recognition or scene recognition results described above, related knowledge services (interfaces) can be invoked. These knowledge service interfaces can be customized according to user settings. For example, in encyclopedia knowledge mode, the corresponding encyclopedia knowledge service interface can be invoked to obtain encyclopedia content for the current object (e.g., children's encyclopedia content). In English immersion mode, a children's English education content interface can be invoked to obtain English vocabulary, phrases, example sentences, and audio files related to the current scene or object. It should be understood that more related services can be extended as needed. For example, extended interfaces can be invoked to implement extended knowledge service capabilities. These extended interfaces can call camera servers or other external resource servers to obtain external data in order to generate extended knowledge service data.

[0047] In another implementation of the present invention, the method further includes: performing downsampling processing and / or feature extraction processing on the captured image to obtain image data of the captured image.

[0048] Downsampling and / or feature extraction facilitate data transmission and / or data recognition, thereby improving data processing efficiency and further enhancing the display efficiency of knowledge service data.

[0049] Specifically, captured images can be preprocessed locally within the camera's built-in processor to reduce subsequent network transmission bandwidth and server-side computational costs. The aforementioned image downsampling or basic feature extraction operations can be configured based on the camera's hardware configuration and cost. Alternatively, the smart camera can determine the appropriate data preprocessing mode based on camera configuration information (hardware and / or software configuration information) acquired locally or from a camera server.

[0050] In another implementation of the present invention, the knowledge service data includes descriptive text and / or descriptive audio that match the recognition results of the image data. When displaying the captured image, the knowledge service data is displayed in a preset multimedia display mode, including: presenting descriptive text and / or playing descriptive audio when displaying the captured image.

[0051] In another implementation of the present invention, the method further includes: in response to the establishment of a network connection with the screen device, synchronizing the captured images to a target folder in the screen device.

[0052] Efficient image management is achieved by establishing a network connection between the synchronization process and the screen-equipped device.

[0053] In another implementation of the present invention, in response to the establishment of a network connection with the screen device, the captured images are synchronized to a target folder in the screen device, including: in response to the establishment of a network connection with the screen device, displaying an image synchronization confirmation control, the prompt information of the image synchronization confirmation control including the identifier of the screen device; and in response to the triggering of the image synchronization confirmation control, synchronizing the captured images to the target folder in the screen device.

[0054] By providing users with the option to synchronize images through a synchronization confirmation control, efficient image management is achieved while giving users choice, avoiding the limitation of a single image management method.

[0055] In another implementation of the present invention, the knowledge service request also includes the location information of the smart camera, the preset multimedia display mode includes a travel mode, the knowledge service data includes the geographical knowledge introduction corresponding to the location information, the geographical knowledge introduction is generated based on the location information by calling the knowledge service interface that matches the travel mode, and the knowledge service data is displayed in the preset multimedia display mode, including: displaying the geographical knowledge introduction in the travel mode.

[0056] Because the introduction of geographical knowledge is generated and displayed based on the knowledge service interface that matches the tourism pattern and is based on location information, the efficiency, flexibility, or fun of knowledge service provision is achieved, thereby improving the user's learning efficiency.

[0057] Specifically, devices that incorporate GPS (Global Positioning System) positioning into smart cameras can provide users (e.g., children) with travel geography encyclopedia services. For example, upon entering travel mode, the device provides geographical encyclopedia information about the current attraction based on GPS location (e.g., a children's geography encyclopedia). Furthermore, the arrangement of encyclopedic content and the tone of the audio presentation can be tailored to the characteristics of specific groups (e.g., children). For children, this can be achieved through presentation methods that are engaging and easy for them to understand.

[0058] In another implementation of the present invention, sending the knowledge service request includes: responding to a voice command and sending the knowledge service request, wherein the knowledge service request includes voice data indicating the voice command, and the knowledge service data is generated by calling a knowledge service interface that matches the preset multimedia display mode based on the recognition result of the voice data.

[0059] By responding to voice commands to send knowledge service requests and generating knowledge service data by calling the knowledge service interface based on the recognition results of the voice data, knowledge service is achieved through voice interaction.

[0060] In one example, a speech recognition service interface can be called to recognize speech data and obtain the speech recognition result. A knowledge service interface can be called to generate knowledge service data based on the speech recognition result.

[0061] In another example, the knowledge service interface is configured in the knowledge server. The knowledge server responds to knowledge service requests from the knowledge client device by invoking the voice server's service to obtain the recognition results of the voice data. Based on the voice recognition results, the knowledge server generates knowledge service data. The knowledge client device then plays at least the voice content of the knowledge service data.

[0062] In another implementation of the present invention, displaying knowledge service data in a preset multimedia display mode includes: performing voice interaction based on the knowledge service data in a voice interaction display mode.

[0063] Because voice interaction based on the knowledge service data is possible through a voice-interactive display mode, further user interaction regarding knowledge points is achieved, improving the efficiency of users' in-depth learning of these knowledge points.

[0064] In one example, in the voice interaction display mode, the knowledge client device sends a voice interaction knowledge service request to the knowledge server. The content in this request can be associated with previously displayed knowledge service data. The knowledge server responds to the request, recognizes the content, and retrieves the current knowledge service data associated with the recognition result based on the previously displayed knowledge service data. The knowledge server returns at least the voice playback data of the current knowledge service data to the knowledge client.

[0065] In another example, the knowledge server responds to the voice interaction knowledge service request by sending a request to the speech recognition server to recognize the above content; alternatively, it can call the local speech recognition service interface to recognize the above content.

[0066] Figure 2B This is a schematic diagram of a data processing method according to another embodiment of the present invention. As shown in the figure, the description of each step in this example is exemplary. In other examples, more or fewer steps may be included, and the order of the steps may also vary.

[0067] In step 211, the user triggers the photo-taking command.

[0068] In step 212, the smart camera responds to the photo-taking trigger command and takes a photo.

[0069] In step 213, the user triggers a knowledge service operation command.

[0070] In step 214, the smart camera responds to the knowledge service operation instruction and sends a knowledge service request to the smart camera server. The knowledge service request includes the photo data of the captured photo.

[0071] In step 215, the smart camera server returns knowledge service data to the smart camera client.

[0072] In step 216, the smart camera displays knowledge service data to the user.

[0073] Figure 2CThis is a schematic diagram of a data processing method according to another embodiment of the present invention. As shown in the figure, the description of each step in this example is exemplary. In other examples, more or fewer steps may be included, and the order of the steps may also vary.

[0074] In step 221, the smart camera obtains geographic location information from the location service according to the travel mode.

[0075] In step 222, the smart camera sends a knowledge service request, including geographic location information, to the smart camera server.

[0076] In step 223, the smart camera server returns knowledge service data to the smart camera.

[0077] In step 224, the smart camera displays geographic knowledge service data to the user in a travel mode.

[0078] Figure 3A This is a schematic diagram of a data processing method according to another embodiment of the present invention. Figure 3A The data processing method can be applied to any suitable electronic device with data processing capabilities, including but not limited to mobile terminals (such as mobile phones, tablets, etc.) and PCs. Figure 3A The methods include:

[0079] 310: Establishes a network connection with the smart camera on the same network, and the smart camera stores the captured images.

[0080] 320: Response to the establishment of network connection, receiving images synchronously transmitted from the smart camera to the local folder.

[0081] In the embodiment of the present invention, the system can respond to the establishment of a connection with the same network and receive images synchronously transmitted from the smart camera to the local folder, thus enabling quick saving of images and saving local storage space.

[0082] In another implementation of the present invention, the local folder belongs to the first application, and the method further includes: in response to an image sharing operation on the captured image, switching from the first application to a second application, and in the second application, publishing the captured image in association with the identifier of the smart camera.

[0083] By associating captured images with the smart camera's identifier in the second application, the smart camera is promoted, thus linking it with the mobile terminal's service functions.

[0084] For example, the electronic device can be a mobile device, and the mobile server of the mobile device can respond to the target application platform's receipt of the captured image and associate the captured image with the knowledge service platform.

[0085] Figure 3B This is a schematic diagram of a data processing method according to another embodiment of the present invention. As shown in the figure, the description of each step in this example is exemplary. In other examples, more or fewer steps may be included, and the order of the steps may also vary.

[0086] In step 311, the smart camera responds to the network connection and synchronously transmits the captured images to the mobile terminal.

[0087] In step 312, the mobile terminal publishes the captured image to the target application platform.

[0088] In step 313, the mobile server responds to the target application platform's receipt of the captured image and associates the captured image with the knowledge service platform. It should be understood that the mobile server can operate or manage data from at least one application platform, including the target application platform, and the captured image may display an identifier related to the smart camera's services on the target application platform (e.g., a second application). Other mobile terminals can trigger access to the smart camera's service platform based on this identifier to obtain product information about the smart camera, access the knowledge service interface, or visit the knowledge service introduction on the application platform.

[0089] Figure 4A This is a schematic flowchart illustrating a data processing method according to another embodiment of the present invention. It can be applied to any suitable electronic device with data processing capabilities, including but not limited to servers or devices with service capabilities. Specifically, it can be a cloud server such as a public cloud, private cloud, dedicated cloud, or hybrid cloud. Figure 4A The methods include:

[0090] 410: Receive knowledge service requests, which include preset multimedia display modes.

[0091] 420: Based on the knowledge service request, call the knowledge service interface corresponding to the preset multimedia display mode to generate knowledge service data.

[0092] 430: Return knowledge service data.

[0093] In the embodiment of the present invention, since the knowledge service data is generated by calling the knowledge service interface that matches the preset multimedia display mode, the knowledge service data is displayed in the preset multimedia display mode, thereby realizing the display of knowledge service data in a specific mode and improving its educational attributes.

[0094] In another implementation of the present invention, the knowledge service request further includes image data of the captured image. The process of generating knowledge service data by calling a knowledge service interface corresponding to a preset multimedia display mode according to the knowledge service request includes: recognizing the image data according to the knowledge service request to obtain a recognition result of the image data; and using the recognition result, calling the knowledge service interface corresponding to the preset multimedia display mode to generate the knowledge service data.

[0095] By calling the knowledge service interface based on the recognition results, knowledge service data is generated, resulting in richer knowledge service data. In addition, the knowledge service interface corresponds to the preset multimedia display mode, which realizes the simplicity and efficiency of interface calls, and further facilitates the maintenance and updating of the algorithm interface.

[0096] In another implementation of the present invention, the image data is identified according to the knowledge service request to obtain the identification result of the image data, including: subject identification of the image data according to the knowledge service request to obtain the subject classification and / or subject name of the image data; or scene identification of the image data according to the knowledge service request to obtain the scene classification of the image data.

[0097] By performing subject identification or scene identification on image data based on knowledge service requests, high-value related information about the captured images is provided, thus improving the user's learning efficiency.

[0098] Specifically, based on the object recognition or scene recognition results mentioned above, related knowledge services (interfaces) can be invoked. These knowledge service interfaces can be customized according to user settings. For example, in encyclopedia knowledge mode, the corresponding encyclopedia knowledge service interface can be invoked to obtain encyclopedia content for the current object (e.g., children's encyclopedia content). In English immersion mode, a children's English education content interface can be invoked to obtain English vocabulary, phrases, example sentences, and audio files related to the current scene or object. It should be understood that more related services can be extended as needed. For example, extended interfaces can be invoked to implement extended knowledge service capabilities. These extended interfaces can call camera servers or other external resource servers to obtain external data in order to generate extended knowledge service data.

[0099] In another implementation of the present invention, knowledge service data is generated by calling the knowledge service interface corresponding to the preset multimedia display mode based on the recognition result, including: calling the knowledge service interface corresponding to the preset multimedia display mode based on the recognition result to obtain the description text corresponding to the recognition result; generating description audio based on the description text; and generating knowledge service data based on the description text and description audio.

[0100] Because knowledge service data is generated by describing text and audio, it is easy to display multimodal knowledge service data.

[0101] In another implementation of the present invention, generating descriptive audio based on descriptive text includes: performing automatic text editing on the descriptive text to obtain simplified text; and performing speech processing on the simplified text to obtain target style audio.

[0102] By automatically editing the descriptive text to obtain simplified text, the efficiency of users acquiring knowledge is improved. In addition, by performing speech processing on the simplified text to obtain audio in the target style, the efficiency of users acquiring knowledge is further improved.

[0103] Specifically, it can automatically generate descriptive text and corresponding audio. For example, it can perform automatic text editing, that is, automatically edit excessively long descriptive text to obtain relatively concise text. In addition, it can convert simplified text into audio suitable for children based on TTS (text-to-speech) algorithms, such as simulating the tone of an animated character. The simplified text and corresponding audio can be transmitted to a smart camera, and the text can be displayed on the camera screen while the audio plays.

[0104] In another implementation of the present invention, the knowledge service request includes voice data indicating the voice command, wherein the step of calling the knowledge service interface corresponding to the preset multimedia display mode to generate knowledge service data according to the knowledge service request includes: responding to the knowledge service request, recognizing the voice data to obtain a recognition result of the voice data; and calling the knowledge service interface corresponding to the preset multimedia display mode to generate knowledge service data based on the recognition result.

[0105] In one example, a speech recognition service interface can be called to recognize speech data and obtain the speech recognition result. A knowledge service interface can be called to generate knowledge service data based on the speech recognition result.

[0106] In another example, the knowledge service interface is configured in the knowledge server. The knowledge server responds to knowledge service requests from the knowledge client device by invoking the voice server's service to obtain the recognition results of the voice data. Based on the voice recognition results, the knowledge server generates knowledge service data. The knowledge client device then plays at least the voice content of the knowledge service data.

[0107] In one example, in the voice interaction display mode, the knowledge client device sends a voice interaction knowledge service request to the knowledge server. The content in this request can be associated with previously displayed knowledge service data. The knowledge server responds to the request, recognizes the content, and retrieves the current knowledge service data associated with the recognition result based on the previously displayed knowledge service data. The knowledge server returns at least the voice playback data of the current knowledge service data to the knowledge client.

[0108] Because voice interaction based on the knowledge service data is possible through a voice-interactive display mode, further user interaction regarding knowledge points is achieved, improving the efficiency of users' in-depth learning of these knowledge points.

[0109] In another example, the knowledge server responds to the voice interaction knowledge service request by sending a request to the speech recognition server to recognize the above content; alternatively, it can call the local speech recognition service interface to recognize the above content.

[0110] Figure 4B This is a schematic diagram of a data processing method according to another embodiment of the present invention. As shown in the figure, the description of each step in this example is exemplary. In other examples, more or fewer steps may be included, and the order of the steps may also vary.

[0111] In step 411, the smart camera sends a knowledge service request, including a knowledge display mode, to the smart camera server.

[0112] In step 412, the smart camera server calls the knowledge service interface corresponding to the knowledge display mode.

[0113] In step 413, the smart camera server obtains knowledge service data through the knowledge service interface.

[0114] In step 414, the smart camera server returns knowledge service data to the smart camera.

[0115] In step 415, the smart camera displays the knowledge service data in a knowledge display mode.

[0116] Figure 5 This is a schematic block diagram of a smart camera according to another embodiment of the present invention. Figure 5 Smart cameras include:

[0117] The request sending module 510 sends a knowledge service request, which includes a preset multimedia display mode.

[0118] The data receiving module 520 receives knowledge service data, which is generated by calling a knowledge service interface that matches a preset multimedia display mode.

[0119] The data display module 530 displays knowledge service data in a preset multimedia display mode.

[0120] In the embodiment of the present invention, since the knowledge service data is generated by calling the knowledge service interface that matches the preset multimedia display mode, the knowledge service data is displayed in the preset multimedia display mode, thereby realizing the display of knowledge service data in a specific mode and improving its educational attributes.

[0121] In another implementation of the present invention, the knowledge service request also includes image data of the captured image. The knowledge service data is generated by calling a knowledge service interface that matches a preset multimedia display mode based on the recognition result of the image data. The data display module is specifically used to: display the knowledge service data in a preset multimedia display mode when displaying the captured image.

[0122] In another implementation of the present invention, the smart camera further includes a preprocessing module, which performs downsampling processing and / or feature extraction processing on the captured images to obtain image data of the captured images.

[0123] In another implementation of the present invention, the knowledge service data includes descriptive text and / or descriptive audio that match the recognition results of the image data, and the data display module is specifically used to: present descriptive text and / or play descriptive audio when displaying the captured image.

[0124] In another implementation of the present invention, the smart camera further includes a data synchronization module, which, in response to the establishment of a network connection with a screen device, synchronizes the captured images to a target folder in the screen device.

[0125] In another implementation of the present invention, the data synchronization module is specifically used to: respond to the establishment of a network connection with the screen device, display an image synchronization confirmation control, the prompt information of which includes the identifier of the screen device; and respond to the triggering of the image synchronization confirmation control to synchronize the captured images to the target folder in the screen device.

[0126] In another implementation of the present invention, the knowledge service request also includes the location information of the smart camera, the preset multimedia display mode includes a tourism mode, the knowledge service data includes the geographical knowledge introduction corresponding to the location information, the geographical knowledge introduction is generated based on the location information by calling the knowledge service interface that matches the tourism mode, and the data display module is specifically used to: display the geographical knowledge introduction in the tourism mode.

[0127] The smart camera in this embodiment is used to implement the corresponding methods in the foregoing method embodiments and has the beneficial effects of the corresponding method embodiments, which will not be repeated here. Furthermore, the functional implementation of each module in the smart camera of this embodiment can be referred to the description of the corresponding part in the foregoing method embodiments, which will also not be repeated here.

[0128] Figure 6 This is a schematic block diagram of an electronic device according to another embodiment of the present invention. Figure 6 The electronic devices can be applied to any suitable electronic device with data processing capabilities, including but not limited to: servers, mobile terminals (such as mobile phones, PADs, etc.) and PCs. Figure 6 Electronic devices include:

[0129] The connection establishment module 610 establishes a network connection with the smart camera on the same network, and the smart camera stores the captured pictures.

[0130] The synchronous transmission module 620 responds to the establishment of a network connection and receives images captured by the smart camera and transmits them to a local folder synchronously.

[0131] In the embodiment of the present invention, the system can respond to the establishment of a connection with the same network and receive images synchronously transmitted from the smart camera to the local folder, thus enabling quick saving of images and saving local storage space.

[0132] In another implementation of the invention, the local folder belongs to the first application, and the electronic device further includes: a publishing module that, in response to an image sharing operation on the captured image, switches from the first application to a second application, and publishes the captured image in the second application in association with the identifier of the smart camera.

[0133] The electronic device of this embodiment is used to implement the corresponding methods in the foregoing method embodiments and has the beneficial effects of the corresponding method embodiments, which will not be repeated here. Furthermore, the functional implementation of each module in the electronic device of this embodiment can be referred to the description of the corresponding part in the foregoing method embodiments, which will also not be repeated here.

[0134] Figure 7 This is a schematic block diagram of a data processing apparatus according to another embodiment of the present invention. Figure 7 The data processing device can be any suitable electronic device with data processing capabilities, including but not limited to servers or devices with service capabilities. Specifically, it can be a cloud server such as a public cloud, private cloud, dedicated cloud, or hybrid cloud. Figure 7 The device includes:

[0135] The request receiving module 710 receives knowledge service requests, which include preset multimedia display modes.

[0136] The data generation module 720, based on the knowledge service request, calls the knowledge service interface corresponding to the preset multimedia display mode to generate knowledge service data.

[0137] Data return module 730 returns knowledge service data.

[0138] In the embodiment of the present invention, since the knowledge service data is generated by calling the knowledge service interface that matches the preset multimedia display mode, the knowledge service data is displayed in the preset multimedia display mode, thereby realizing the display of knowledge service data in a specific mode and improving its educational attributes.

[0139] In another implementation of the present invention, the knowledge service request further includes image data of the captured image. Specifically, the data generation module is used to: identify the image data according to the knowledge service request to obtain the identification result of the image data; and, based on the identification result, call the knowledge service interface corresponding to the preset multimedia display mode to generate knowledge service data.

[0140] In another implementation of the present invention, the data generation module is specifically used to: perform subject identification on the image data according to the knowledge service request to obtain the subject classification and / or subject name of the image data; or, perform scene identification on the image data according to the knowledge service request to obtain the scene classification of the image data.

[0141] In another implementation of the present invention, the data generation module is specifically used to: call the knowledge service interface corresponding to the preset multimedia display mode based on the recognition result, and obtain the description text corresponding to the recognition result; generate description audio based on the description text; and generate knowledge service data based on the description text and description audio.

[0142] In another implementation of the present invention, the data generation module is specifically used to: perform automatic text editing on the descriptive text to obtain simplified text; and perform speech processing on the simplified text to obtain target style audio.

[0143] The apparatus of this embodiment is used to implement the corresponding methods in the foregoing method embodiments and has the beneficial effects of the corresponding method embodiments, which will not be repeated here. Furthermore, the functional implementation of each module in the apparatus of this embodiment can be referred to the description of the corresponding part in the foregoing method embodiments, which will also not be repeated here.

[0144] Figure 8 The hardware structure of an electronic device according to another embodiment of the present invention; such as Figure 8 As shown, the hardware structure of the electronic device may include: a processor 801, a communication interface 802, a storage medium 803, and a communication bus 804.

[0145] The processor 801, communication interface 802, and storage medium 803 communicate with each other through the communication bus 804.

[0146] Optionally, the communication interface 802 can be an interface of a communication module;

[0147] Specifically, the processor 801 can be configured to: send a knowledge service request, the knowledge service request including a preset multimedia display mode; receive knowledge service data, the knowledge service data being generated by calling a knowledge service interface that matches the preset multimedia display mode; and display the knowledge service data in the preset multimedia display mode.

[0148] Alternatively, a network connection can be established with a smart camera that stores captured images; in response to the establishment of the network connection, the captured images can be synchronously transmitted from the smart camera to a local folder.

[0149] Alternatively, receive a knowledge service request, which includes a preset multimedia display mode; based on the knowledge service request, call the knowledge service interface corresponding to the preset multimedia display mode to generate knowledge service data; and return the knowledge service data.

[0150] The processor 801 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0151] The storage medium 803 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0152] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code configured to perform the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), it performs the functions defined in the methods of this invention. It should be noted that the computer-readable medium described in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. The computer-readable medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access storage media (RAM), read-only storage media (ROM), erasable programmable read-only storage media (EPROM or flash memory), optical fibers, portable compact disk read-only storage media (CD-ROM), optical storage media, magnetic storage media, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program configured for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0153] Computer program code configured to perform the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0154] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions configured to perform a specified logical function. Specific sequences are present in the above specific embodiments, but these sequences are merely exemplary, and in actual implementations, these steps may be fewer, more, or executed in a different order. That is, in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0155] The modules described in the embodiments of this invention can be implemented in software or hardware. The names of these modules do not necessarily limit the module itself.

[0156] In another aspect, the present invention also provides a computer-readable medium having a computer program stored thereon that, when executed by a processor, implements the methods described in the above embodiments.

[0157] In another aspect, the present invention also provides a computer-readable medium, which may be included in the apparatus described in the above embodiments; or it may exist independently and not assembled into the apparatus. The computer-readable medium carries one or more programs that, when executed by the apparatus, cause the apparatus to: send a knowledge service request, the knowledge service request including a preset multimedia display mode; receive knowledge service data, the knowledge service data being generated by calling a knowledge service interface matching the preset multimedia display mode; and display the knowledge service data in the preset multimedia display mode.

[0158] Alternatively, a network connection can be established with a smart camera that stores captured images; in response to the establishment of the network connection, the captured images can be synchronously transmitted from the smart camera to a local folder.

[0159] Alternatively, receive a knowledge service request, which includes a preset multimedia display mode; based on the knowledge service request, call the knowledge service interface corresponding to the preset multimedia display mode to generate knowledge service data; and return the knowledge service data.

[0160] The terms "first," "second," "first," or "second" as used in the various embodiments of this disclosure may modify various components regardless of their order and / or importance, but these terms do not limit the corresponding components. The above terms are configured only for the purpose of distinguishing an element from other elements. For example, "first user equipment" and "second user equipment" refer to different user equipments, although both are user equipment. For example, without departing from the scope of this disclosure, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element.

[0161] When a component (e.g., a first component) is referred to as being "(operably or communicatively) coupled" or "(operably or communicatively) coupled to" or "connected to" another component (e.g., a second component), it should be understood that the first component is directly connected to the second component or that the first component is indirectly connected to the second component via yet another component (e.g., a third component). Conversely, it can be understood that when a component (e.g., a first component) is referred to as being "directly connected" or "directly coupled" to another component (the second component), no component (e.g., a third component) is inserted between the two.

[0162] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention is not limited to the specific combination of the above-described technical features, but also includes other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in this invention.

Claims

1. A data processing method, comprising: Send a knowledge service request, which includes image data of the captured image and a preset multimedia display mode; Receive knowledge service data, which is generated by calling a knowledge service interface that matches the preset multimedia display mode based on the recognition result of the image data; When displaying the captured images, the knowledge service data is displayed in the preset multimedia display mode; The step of displaying the knowledge service data in the preset multimedia display mode includes: performing voice interaction based on the knowledge service data in a voice interaction display mode; in the voice interaction display mode, sending a voice interaction knowledge service request, wherein the content of the voice interaction knowledge service request is associated with the previously displayed knowledge service data, and receiving voice playback data of the current knowledge service data associated with the previously displayed knowledge service data; The knowledge service request also includes the location information of the smart camera, the preset multimedia display mode includes a travel mode, the current knowledge service data includes the geographical knowledge introduction corresponding to the location information, the geographical knowledge introduction is generated based on the location information by calling the knowledge service interface that matches the travel mode, and the display of the knowledge service data in the preset multimedia display mode also includes: displaying the geographical knowledge introduction in the travel mode.

2. The method according to claim 1, wherein, The method further includes: The captured image is downsampled and / or feature extracted to obtain the image data of the captured image.

3. The method according to claim 1, wherein, The knowledge service data includes descriptive text and / or descriptive audio that match the recognition results of the image data, wherein displaying the knowledge service data in the preset multimedia display mode when displaying the captured image includes: When displaying the captured images, the descriptive text is presented and / or the descriptive audio is played.

4. The method according to claim 1, wherein, The method further includes: In response to the establishment of a network connection with the device with a screen, the captured images are synchronized to the target folder on the device with a screen.

5. The method according to claim 4, wherein, The response establishes a network connection with the screen device, and synchronizes the captured images to a target folder on the screen device, including: In response to the establishment of a network connection with the screen device, an image synchronization confirmation control is displayed, and the prompt information of the image synchronization confirmation control includes the identifier of the screen device; In response to the triggering of the image synchronization confirmation control, the captured image is synchronized to the target folder in the device with a screen.

6. The method according to claim 1, wherein, Sending the knowledge service request includes: In response to a voice command, a knowledge service request is sent, which includes voice data indicating the voice command. The knowledge service data is generated by calling a knowledge service interface that matches the preset multimedia display mode based on the recognition result of the voice data.

7. A data processing method, comprising: Receive a knowledge service request, which includes image data of the captured image and a preset multimedia display mode; Based on the recognition results of the image data, the knowledge service interface corresponding to the preset multimedia display mode is invoked to generate knowledge service data. Return the knowledge service data so that when the knowledge service requester displays the captured image, the knowledge service data is displayed in the preset multimedia display mode; The step of displaying the knowledge service data in the preset multimedia display mode includes: performing voice interaction based on the knowledge service data in a voice interaction display mode; in the voice interaction display mode, sending a voice interaction knowledge service request, wherein the content of the voice interaction knowledge service request is associated with the previously displayed knowledge service data, and receiving voice playback data of the current knowledge service data associated with the previously displayed knowledge service data; The knowledge service request also includes the location information of the smart camera, the preset multimedia display mode includes a travel mode, the current knowledge service data includes the geographical knowledge introduction corresponding to the location information, the geographical knowledge introduction is generated based on the location information by calling the knowledge service interface that matches the travel mode, and the display of the knowledge service data in the preset multimedia display mode also includes: displaying the geographical knowledge introduction in the travel mode.

8. The method according to claim 7, further comprising: Based on the knowledge service request, perform subject identification on the image data to obtain the subject classification and / or subject name of the image data, or... Based on the knowledge service request, scene recognition is performed on the image data to obtain the scene classification of the image data.

9. The method according to claim 7, wherein, Based on the recognition results, the knowledge service interface corresponding to the preset multimedia display mode is invoked to generate knowledge service data, including: Based on the recognition results, the knowledge service interface corresponding to the preset multimedia display mode is invoked to obtain the descriptive text corresponding to the recognition results; Based on the description text, generate description audio; The knowledge service data is generated based on the description text and the description audio.

10. The method according to claim 9, wherein, The step of generating descriptive audio based on the descriptive text includes: The description text is automatically edited to obtain simplified text; The simplified text is processed by speech to obtain the target style audio.

11. The method according to claim 7, wherein, The knowledge service request includes voice data indicating voice commands, wherein the knowledge service data is generated by calling the knowledge service interface corresponding to the preset multimedia display mode, including: In response to the knowledge service request, the voice data is recognized to obtain the recognition result of the voice data; The knowledge service interface corresponding to the preset multimedia display mode is invoked, and knowledge service data is generated based on the recognition results.

12. A smart camera, comprising: The request sending module sends a knowledge service request, which includes image data of the captured image and a preset multimedia display mode. The data receiving module receives knowledge service data, which is generated by calling a knowledge service interface that matches the preset multimedia display mode based on the recognition result of the image data. The data display module displays the knowledge service data in the preset multimedia display mode when displaying the captured images; The step of displaying the knowledge service data in the preset multimedia display mode includes: performing voice interaction based on the knowledge service data in a voice interaction display mode; in the voice interaction display mode, sending a voice interaction knowledge service request, wherein the content of the voice interaction knowledge service request is associated with the previously displayed knowledge service data, and receiving voice playback data of the current knowledge service data associated with the previously displayed knowledge service data; The knowledge service request also includes the location information of the smart camera, the preset multimedia display mode includes a travel mode, the current knowledge service data includes the geographical knowledge introduction corresponding to the location information, the geographical knowledge introduction is generated based on the location information by calling the knowledge service interface that matches the travel mode, and the display of the knowledge service data in the preset multimedia display mode also includes: displaying the geographical knowledge introduction in the travel mode.

13. A data processing apparatus, comprising: The request receiving module receives knowledge service requests, which include image data of captured images and preset multimedia display modes. The data generation module, based on the recognition results of the image data, calls the knowledge service interface corresponding to the preset multimedia display mode to generate knowledge service data; The data return module returns the knowledge service data so that when the knowledge service requester displays the captured image, the knowledge service data is displayed in the preset multimedia display mode. The step of displaying the knowledge service data in the preset multimedia display mode includes: performing voice interaction based on the knowledge service data in a voice interaction display mode; in the voice interaction display mode, sending a voice interaction knowledge service request, wherein the content of the voice interaction knowledge service request is associated with the previously displayed knowledge service data, and receiving voice playback data of the current knowledge service data associated with the previously displayed knowledge service data; The knowledge service request also includes the location information of the smart camera, the preset multimedia display mode includes a travel mode, the current knowledge service data includes the geographical knowledge introduction corresponding to the location information, the geographical knowledge introduction is generated based on the location information by calling the knowledge service interface that matches the travel mode, and the display of the knowledge service data in the preset multimedia display mode also includes: displaying the geographical knowledge introduction in the travel mode.

14. An electronic device, the device comprising: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction that causes the processor to perform the operation corresponding to the method as described in any one of claims 1-11.

15. A storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as claimed in any one of claims 1-11.

Images