Picture display method, image data processing method, device and equipment

Abstract

The invention relates to a picture display method, an image data processing method and device and equipment. The method comprises the following steps: displaying an event entrance area, wherein the event entrance area comprises event snapshots corresponding to events; the event snapshot graph is generated based on images related to each event and identified from the acquired image data; and in response to a viewing operation for the selected target event snapshot, displaying a picture display page, and displaying a target event picture corresponding to the event of the target event snapshot in the picture display page.

Description

Screen display method, image data processing method, device and equipment Technical Field

[0001] The present application relates to the technical field of electronic equipment, and in particular to a screen display method, an image data processing method, an apparatus and a device. Background Art

[0002] With the development of Internet of Things (IoT) technology and the improvement of people's safety awareness, more and more video surveillance devices are being applied to people's production and life. These video surveillance devices can record surveillance videos of specific areas, so that people can check the surveillance videos to determine whether there are any abnormalities or safety hazards.

[0003] However, in practice, video surveillance equipment usually records continuously for a long time, so the duration of the surveillance video is relatively long. However, people are often only interested in certain events in the entire video. However, the long video is not convenient for people to view the corresponding events.

[0004] Summary of the Invention

[0005] Based on this, it is necessary to provide a screen display method, image data processing method, device and equipment that can facilitate users to view specific events in the video in order to address the above technical problems.

[0006] In a first aspect, an embodiment of the present application provides a screen display method, which is executed by a user terminal electronic device, and the method includes:

[0007] Displaying an event entry area, the event entry area includes an event snapshot corresponding to each event; the event snapshot is generated based on an image related to each event identified from the collected image data;

[0008] In response to a viewing operation on a selected target event snapshot image, a screen display page is displayed, and a target event screen corresponding to the event of the target event snapshot image is displayed on the screen display page.

[0009] In a second aspect, an embodiment of the present application provides an image data processing method, performed by an electronic device, the method comprising:

[0010] Perform event recognition on the collected image data to obtain event images corresponding to each event;

[0011] From the event screen corresponding to each event, an image corresponding to each event is extracted, and an event snapshot corresponding to each event is generated based on the image; the event snapshot is used for index display in the event entry area, and by responding to the viewing operation of the event snapshot in the event entry area, the target event screen corresponding to the event of the event snapshot is displayed.

[0012] In a third aspect, an embodiment of the present application further provides a screen display device, comprising:

[0013] A first display module is configured to display an event entry area, wherein the event entry area includes an event snapshot corresponding to each event; the event snapshot is generated based on an image related to each event identified from the collected image data;

[0014] The second display module is configured to display a picture display page in response to a viewing operation on the selected target event snapshot picture, and to display a target event picture corresponding to the event of the target event snapshot picture on the picture display page.

[0015] In a fourth aspect, an embodiment of the present application further provides an image data processing device, the device comprising:

[0016] The recognition module is used to perform event recognition on the collected image data and obtain event images corresponding to each event;

[0017] A generation module is used to extract images corresponding to each event from the event screen corresponding to each event, and generate event snapshots corresponding to each event based on the images; the event snapshots are used for index display in the event entry area, and by responding to the viewing operation of the event snapshots in the event entry area, the target event screen corresponding to the event of the event snapshot is displayed.

[0018] In a fifth aspect, an embodiment of the present application further provides an electronic device comprising at least one processor and at least one memory, wherein the memory stores computer-readable instructions, wherein when the computer-readable instructions are executed by the processor, the steps of the screen display method as described in the first aspect are implemented, or the steps of the image data processing method as described in the second aspect are implemented.

[0019] In a sixth aspect, an embodiment of the present application further provides a computer-readable storage medium having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by one or more processors, the steps of the screen display method described in the first aspect are implemented, or the steps of the image data processing method described in the second aspect are implemented.

[0020] In the seventh aspect, an embodiment of the present application also provides a computer program product or a computer program, wherein the computer program product or the computer program includes computer-readable instructions, and the computer-readable instructions are stored in a computer-readable storage medium; the processor of the computer device reads the computer-readable instructions from the computer-readable storage medium, and when the processor executes the computer-readable instructions, it implements the steps of the screen display method described in the first aspect, or implements the steps of the image data processing method described in the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] To more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following briefly introduces the drawings required for describing the embodiments of the present disclosure. Obviously, the drawings described below are only some embodiments of the present application. Those skilled in the art can also derive other drawings based on these drawings without inventive effort.

[0022] FIG1 is a diagram illustrating an implementation environment of a screen display method and an image data processing method according to an embodiment;

[0023] FIG2 is a schematic diagram of a flow chart of a screen display method in one embodiment;

[0024] FIG3 is a schematic flow chart of an image data processing method according to an embodiment;

[0025] FIG4 is a flow chart of a method for displaying images and a method for processing image data in one embodiment;

[0026] FIG5 is a schematic diagram of a home page of an application program according to an embodiment;

[0027] FIG6 is a schematic diagram of an event configuration page of an application program in one embodiment;

[0028] FIG7 is a second schematic diagram of the home page of an application program in one embodiment;

[0029] FIG8 is a schematic diagram of a camera album page of an application program in one embodiment;

[0030] FIG9 is a second schematic diagram of a camera album page of an application program in one embodiment;

[0031] FIG10 is a schematic diagram of an intermediate page of an application program in one embodiment;

[0032] FIG11 is a schematic diagram of a photo album page of an application in one embodiment;

[0033] FIG12 is a schematic diagram of an application display magnified view in one embodiment;

[0034] FIG13 is a second schematic diagram of an album page of an application program in one embodiment;

[0035] FIG14 is a schematic diagram of a play page in one embodiment;

[0036] FIG15 is a structural block diagram of a picture display device according to an embodiment;

[0037] FIG16 is a block diagram of the structure of an image data processing device according to an embodiment;

[0038] FIG17 is a schematic structural diagram of an electronic device in one embodiment. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0040] Application Overview

[0041] As mentioned in the background section, with the development of the Internet of Things (IoT), video surveillance equipment is becoming increasingly common in people's daily lives and production. These devices can transmit captured video footage to IoT platforms, allowing users to access the platforms using devices such as mobile phones and computers and view the captured footage at any time. Furthermore, captured video footage can be stored as video files for users to access later as needed.

[0042] To facilitate understanding, let's use an example. For example, consider a video surveillance device, such as a smart camera installed in a user's home and connected to an IoT platform. Once activated, the smart camera transmits the home video footage to the IoT platform in real time. Users can access the IoT platform via their mobile phone and check what's happening at home at any time. Furthermore, the video footage captured by the smart camera can be stored as a video file. This allows users to review any missed events using the stored video file. Typically, to avoid omissions, smart cameras continue recording after being activated, capturing all events during the recording period and ultimately generating a longer video file.

[0043] In some scenarios, users need to view specific video clips corresponding to a specific event or events using the aforementioned technology. In such cases, users can only slowly scroll through the timeline of the video file to find the corresponding video clips. This process is cumbersome, inconvenient, and often time-consuming and laborious.

[0044] To address the above issues, the present application provides a screen display method, an image data processing method, an apparatus, and a device, the purpose of which is to quickly present the video screen corresponding to the event the user is looking for, effectively reducing the user's operation steps. The specific solution is described in detail below through examples.

[0045] Exemplary Implementation Environment

[0046] The screen display method and image data processing method provided in this application can be applied to the implementation environment shown in Figure 1. The implementation environment can be an Internet of Things platform, wherein one or more smart devices 110 can be connected to the gateway 120 via WIFI, Bluetooth ZIGBEE (Purple Bee), etc., the gateway 120 is connected to the router 130 via WIFI, etc., and the router 130 is connected to the cloud server 140 via WIFI, etc.; In addition, in some scenarios, the router 130 is also connected to the user terminal 150 via WIFI, etc., and the user terminal 150 is also connected to the cloud server 140 via 2G / 3G / 4G / 5G / WIFI, etc. It should be understood that the communication connection of the above-mentioned devices is not limited to the connection methods listed above, and can be adjusted accordingly according to actual needs. After the above-mentioned devices are connected to each other in communication, a communication process can be realized, and the communication process includes but is not limited to data interaction and instruction transmission.

[0047] Smart devices 110 may include, but are not limited to, smart curtain motors, smart door locks, smart printers, smart fax machines, smart cameras, smart air conditioners, smart TVs, smart refrigerators, or devices equipped with communication modules (e.g., ZIGBEE modules, Wi-Fi modules, Bluetooth communication modules, etc.), human sensors, door and window sensors, temperature and humidity sensors, water immersion sensors, natural gas alarms, smoke alarms, wall switches, wall sockets, wireless switches, wireless wall switches, Rubik's Cube controllers, and the like, without limitation herein. Gateway 120 may be a network interconnection device with hub capabilities or a camera device with gateway functionality.

[0048] The user terminal 150 may be a smart phone, a laptop, a personal computer, a tablet computer, a smart control panel, or other electronic device capable of network connection, which is not limited herein. The cloud server 140 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0049] Specifically, a smart device 110 or gateway 120 with a camera function can collect image data within a corresponding range. The collected image data is then used by the device, server, or user terminal to perform event recognition on the collected image data to obtain event screens corresponding to each event. From the event screens corresponding to each event, images corresponding to each event are extracted, and event snapshots corresponding to each event are generated based on the images. The event snapshots are used for indexing and displaying in the event entry area of ​​the user terminal interface. By responding to a viewing operation on the event snapshot in the event entry area, the target event screen corresponding to the event in the selected event snapshot can be quickly viewed without viewing the entire video. This can effectively reduce the user's operation steps, thereby improving the user experience and also increasing the viewing rate of the video screen.

[0050] It is understood that the above implementation environment is not the only implementation environment for the screen display method and image data processing method provided by this application, but is one of the environments in which the methods provided by this application can be implemented. Here, the screen display method and image data processing method provided by this application are described using only the Internet of Things platform as an example.

[0051] Exemplary Methods

[0052] Referring to FIG. 2 , in an exemplary embodiment, a screen display method is provided, which is described using an example of the method being applied to a user terminal. Specifically, the user terminal may be user terminal 150 in the implementation environment shown in FIG. At least one smart device 110 in the implementation environment shown in FIG. 1 may capture and generate a video file, and store the video file in gateway 120 and / or cloud server 140. User terminal 150 may access the video file through gateway 120 and / or cloud server 140.

[0053] The screen display method can be executed by the user terminal, as shown in FIG2 , and may include the following steps:

[0054] Step S201: The user terminal displays an event entry area, which includes an event snapshot corresponding to each event; the event snapshot is generated based on images related to each event identified from collected image data.

[0055] Among them, the event can be an event in which a preset situation occurs. The event entry area is all or part of the area in the current display interface of the user terminal, and the event entry area can display data such as images, Chinese characters, letters, numbers, special symbols, etc. Assuming that an application provided by the Internet of Things platform is installed on the user terminal, the event entry area can be a partial area of ​​any page of the application. For example, the event entry area can be a predetermined area of ​​the homepage and / or camera album page of the application. When the user terminal displays the homepage or camera album page of the application, the event snapshot corresponding to each event is displayed through the predetermined area of ​​the homepage or camera album page.

[0056] It can be understood that an event snapshot image can refer to a diagram that reflects the event status occurring at a certain moment through a screen. Specifically, it can be one of the images in multiple frames of image data that can better describe the occurrence of a preset event monitored at a certain moment as the event snapshot image corresponding to the event.

[0057] Based on this, since event snapshot images are generated from images associated with each event identified from the collected image data, the image content of the event snapshot image is associated with the event corresponding to the event snapshot image. In some embodiments, the event corresponding to the event snapshot image may be an event that has occurred or is currently occurring, as represented by the image content of the event snapshot image. For example, if the image content of an event snapshot image is an open window, the event corresponding to the event snapshot image may be a window opening event.

[0058] It is understandable that users can easily know which event the event snapshot image corresponds to through the event snapshot image in the event entry area. Therefore, displaying the event snapshot image corresponding to each event in the event entry area can facilitate users to select the event of interest and perform subsequent operations according to their needs.

[0059] In addition, the number of event snapshot images in the event entry area is not limited to one. In some embodiments, if there are multiple event snapshot images, each event snapshot image is dynamically displayed in the event entry area.

[0060] Specifically, when the event entry area includes multiple event snapshots, each event snapshot corresponds to an event. Moreover, the image content of different event snapshots may be the same or different, and the events corresponding to different event snapshots may also be the same or different. In some scenarios, the events corresponding to multiple event snapshots are not exactly the same, or even completely different. Since different users may pay attention to different events, even the events that the same user pays attention to in different situations may be different. Therefore, the user may operate on any event snapshot. In order to avoid affecting the use of some users due to the failure to display one or some event snapshots.

[0061] In this embodiment, the user terminal will display each event snapshot in the event entry area separately, and display each event snapshot to the user in sequence in a dynamic display manner. By displaying multiple event snapshots in the event entry area, it is possible to avoid the situation where the normal use of some users is affected due to the failure to display one or some event snapshots. Furthermore, compared to displaying all event snapshots at the same time, this embodiment uses a dynamic display method to display each event snapshot in sequence, so that the event snapshot occupies a smaller area in the screen display page. At the same time, it breaks through the limit on the number of event snapshots imposed by the area of ​​the screen display page, and a larger number of event snapshots can be displayed in the screen display page.

[0062] In some embodiments, when the user terminal dynamically displays each event snapshot, each event snapshot may be displayed cyclically in the event entry area of ​​the user terminal, and each event snapshot will be displayed once in each cycle. The cycle may be a relatively short duration.

[0063] Specifically, in one embodiment, the cycle period can be a shorter fixed duration, and no matter how many event snapshot images exist, they are all displayed in a cycle according to the cycle period. The cycle period can be, for example, 5 seconds, but is not limited thereto. In another embodiment, the display interval between two adjacent event snapshot images can be a shorter fixed duration, and the cycle period varies with the number of event snapshot images. Assume that the display interval between two adjacent event snapshot images is 1 second. When the number of event snapshot images is 4, the cycle period is 3 seconds. When the number of event snapshot images is 5, the cycle period is 4 seconds. It can be understood that the display interval between two adjacent event snapshot images is not limited to 1 second, and can be set according to actual needs.

[0064] In some embodiments, the user terminal can also dynamically display each event snapshot image based on the user's operation in the event entry area. In other words, the user can manually switch the display of different event snapshot images. For example, each event snapshot image is displayed in the form of a carousel image. The user can slide the event snapshot image currently displayed in the event entry area to make the event entry area display other event snapshot images.

[0065] Step S202: In response to the viewing operation on the selected target event snapshot image, the user terminal displays a screen display page, and displays the target event screen corresponding to the event of the target event snapshot image on the screen display page.

[0066] Among them, the target event snapshot is the event snapshot selected in the event entry area. Specifically, in the case where the event entry area includes multiple event snapshots, the target event snapshot can be any one of the multiple event snapshots. That is to say, the user can freely select the target event snapshot on the user terminal according to his or her needs. The viewing operation can be a click, slide, long press and other operations, but is not limited to this. The user terminal responds when receiving the viewing operation and displays the screen display page. Among them, the screen display page is a page that can display an image screen, and the image screen can be a picture, video, etc. Continuing with the above-mentioned user terminal with an application provided by the Internet of Things platform installed as an example, the screen display page can be any page of the application. For example, the screen display page can be the camera playback page of the application, but is not limited to this.

[0067] Furthermore, the target event screen is the event screen corresponding to the target event, and the target event is the event corresponding to the target event snapshot image. Thus, when a user is focused on the target event, by selecting and viewing the target event snapshot image corresponding to the target event, the event screen corresponding to the target event can be viewed on the user terminal. Therefore, the image screen displayed on the screen display page should match the event snapshot image targeted by the viewing operation. For example, the target event is a window opening event, and the target event snapshot image is an image of an open window. Accordingly, the target event screen can be at least one screen during the window opening process.

[0068] In some embodiments, the event image is an image related to the event in the image data collected in step S201. Accordingly, the target event image is an image related to the target event in the image data collected in step S201. Thus, when a user is interested in a target event, they can select and view the target event snapshot. The user terminal can then directly locate and display the image related to the target event in the image data collected in step S201.

[0069] In some embodiments, in step S202, the user terminal displays the target event screen corresponding to the event of the target event snapshot on the screen display page, including: the user terminal displays the enlarged image and / or video clip corresponding to the target event snapshot on the screen display page.

[0070] Specifically, each event snapshot image corresponds to an enlarged image having the same image content and / or a video clip recording the event corresponding to the event snapshot image. The enlarged image is larger than the event snapshot image. In some embodiments, the resolution and / or clarity of the enlarged image is also higher than that of the event snapshot image. The video clip corresponding to the target event snapshot image records the target event. In some embodiments, the video clip recording the target event is a video clip composed of images related to the target event in the image data collected in step S201 above.

[0071] In one embodiment, the events corresponding to each event snapshot are recorded in one video file, so that the video clips corresponding to each event snapshot are different video clips in the same video file. In this way, the number of video files can be reduced. In another embodiment, the events corresponding to different event snapshots are recorded in different video files, and each video file serves as a video clip. In this way, although the number of video files increases, each video file can be distinguished by the recorded events, which is more convenient for management. For example, 10 video files each record 10 different events. When the user no longer pays attention to some of the events, the user can delete the video files recording these events. Only the video files recording the events that the user may pay attention to are retained.

[0072] It is understood that magnified images and video clips can meet different levels of user needs. In some scenarios, users only need to view the magnified image to meet their needs. In such cases, simply providing the magnified image to the user is sufficient. In other scenarios, the magnified image provides limited information and cannot meet user needs. In such cases, the user terminal needs to provide the video clip to the user to meet more in-depth needs. This embodiment displays at least one of the magnified image corresponding to the target event snapshot image and the video clip corresponding to the target event snapshot image as the target event image on the image display page. Displaying the magnified image can provide the user with an enhanced visual experience of the image content. Displaying the video clip can also provide the user with more information related to the event. It can even be possible to display the magnified image and video clip simultaneously, providing the user with both the image content and more information related to the event. Thus, by flexibly selecting the data content to be displayed on the image display page from the magnified image and video clip, users' needs for events at different levels can be met.

[0073] In some embodiments, depending on the viewing operation performed on the target event snapshot image, the target event image displayed on the screen display page of the user terminal may also be different. For example, if the viewing operation performed on the target event snapshot image is a first viewing operation, the target event image displayed on the screen display page of the user terminal is an enlarged image corresponding to the target event snapshot image. If the viewing operation performed on the target event snapshot image is a second viewing operation, the target event image displayed on the screen display page of the user terminal is a video clip corresponding to the target event snapshot image.

[0074] The first viewing operation can be a single operation or a series of operations. The second viewing operation can be a single operation or a series of operations. The first viewing operation and the second viewing operation can be different operations. For example, the first viewing operation can be a short press operation, and the second viewing operation can be a long press operation, but the present invention is not limited thereto.

[0075] In the above-mentioned screen display method, by displaying an event entry area including event snapshot images corresponding to each event, and in response to a viewing operation on a selected target event snapshot image, displaying a screen display page, and displaying a target event screen corresponding to the event of the target event snapshot image on the screen display page, the screen corresponding to a specific event can be quickly displayed. That is, the user can quickly view the target event screen corresponding to the event of the selected event snapshot image by viewing the event entry area displayed by the device and selecting and viewing the event snapshot images corresponding to each event included in the event entry area, without having to view the entire video. Therefore, the user's operation steps can be effectively reduced, thereby improving the user experience, and at the same time, the viewing rate of the video screen can be increased.

[0076] Based on the above-mentioned embodiments, the screen display method provided in this embodiment may also include: the user terminal displays a dynamic album page in response to a trigger operation on the event entry area; the dynamic album page includes event snapshots corresponding to each generated event.

[0077] In some embodiments, when the event entry area includes event snapshots corresponding to multiple events, the event entry area can usually only display a small number of event snapshots at the same time. In some scenarios, the user needs to quickly select the target event snapshot from multiple event snapshots. At this time, if you still select the target event snapshot in the event entry area, you may encounter a situation where the target event snapshot is not displayed in the current event entry area. Once this happens, you need to wait until the event entry area displays the target event snapshot before you can select the target event snapshot. In this way, the user cannot quickly select the target event snapshot from multiple event snapshots.

[0078] In response to the above-mentioned possible problems, a dynamic album page can be set up. The dynamic album page is used to display the event snapshot images corresponding to each event that has been generated. In this way, when the user needs to quickly select the target event snapshot image from multiple event snapshot images, the user can first trigger the event entry area. By displaying the dynamic album page, the user terminal can display the event snapshot images corresponding to each event that has been generated to the user. The user can then quickly select the target event snapshot image from the dynamic album page. Therefore, compared to viewing the event snapshot images in the event entry area, this embodiment can first provide the user with more choices, allowing the user to choose to view the event snapshot images on the dynamic album page. At the same time, the event snapshot images corresponding to each event that has been generated are all displayed on the dynamic album page, so that the user can quickly select the target event snapshot image from them.

[0079] In some embodiments, the user terminal has an application provided by the Internet of Things platform installed, and the dynamic photo album page can be any page of the application. The user terminal can directly display the dynamic photo album page on the page displaying the event entrance area.

[0080] In the case where there are a large number of event snapshot images corresponding to the generated events, the user terminal may not need to display the event snapshot images corresponding to all the generated events at the same time. In one embodiment, the dynamic album page includes a page that can load the event snapshot images corresponding to all the generated events. The user terminal can only display the event snapshot images corresponding to some events at the same time. By sliding the dynamic album page, the event snapshot image currently being displayed can be changed. In another embodiment, the dynamic album page includes multiple pages that can jump to each other, and each page can load the event snapshot images corresponding to some of the generated events. The user terminal can display different event snapshot images to the user by displaying different pages.

[0081] In other embodiments, the event entry area is an entry for indicating event images captured by the camera device; the method also includes: the user terminal displays the album page corresponding to the camera device in response to a trigger operation on the event entry area; the album page includes at least a dynamic album entry; the user terminal displays the dynamic album page in response to a trigger operation on the dynamic album entry; the dynamic album page includes event snapshot images corresponding to each generated event.

[0082] Specifically, in some embodiments, an application provided by an IoT platform is installed on the user terminal, and the album page and the dynamic album page are different pages of the application. The user terminal can directly display the album page on the page displaying the event entry area, and then jump from the album page to the dynamic album page based on the user's triggering operation on the dynamic album entry. It is understandable that the dynamic album page in this embodiment is the same as the dynamic album page in the previous embodiment and will not be repeated here.

[0083] Through the method of this embodiment, not only can a dynamic album page be provided to display the event snapshot images corresponding to each generated event to the user. An album page can also be provided as an intermediate page before jumping to the dynamic album page. Therefore, compared to viewing the event snapshot images in the event entry area, this embodiment can first provide users with more choices, so that users can choose to view the event snapshot images on the dynamic album page. At the same time, the event snapshot images corresponding to each generated event are all displayed on the dynamic album page, so that users can quickly select the target event snapshot image from them. In addition, using the album page as a buffer between the current page displaying the event entry area and the dynamic album page can avoid triggering operations based on the event entry area and directly jump to the dynamic album page.

[0084] Based on the above-mentioned embodiments, the screen display method provided in this embodiment may also include: the user terminal displays the event label information corresponding to each event snapshot image while displaying the event entry area; and / or, the user terminal displays the event label information corresponding to the target event screen while displaying the target event screen in the screen display page; wherein the event label information includes at least one of spatial information, time information and event information.

[0085] Specifically, in some embodiments, when the user terminal displays various event snapshots through the event entry area, the user can select an event snapshot as a target event snapshot according to their needs in order to view the target event screen. However, in some scenarios, the user's needs are relatively complex and involve multiple pieces of information. Moreover, not all of this information can be obtained from the image content of the event snapshot. For example, the user's need is for a certain event to occur at a certain location within a certain time period. In this case, the user terminal will not be able to select a target event snapshot that meets the user's needs based solely on the image content of the event snapshot.

[0086] To address the above-mentioned possible problems, the user terminal can display the event label information corresponding to each event snapshot while displaying the event entry area, so that the user can select the target event snapshot that meets his or her needs based on the image content of the event snapshot and the event label information corresponding to the event snapshot.

[0087] The spatial information in the event label information corresponding to the event snapshot image may be the location where the event corresponding to the event snapshot image occurred. The location of occurrence may be, for example, geographic coordinates of longitude and latitude, a room identifier, etc. The time information in the event label information corresponding to the event snapshot image may be the time of occurrence of the event corresponding to the event snapshot image. The time of occurrence may be, for example, a timestamp, a time in any time zone, etc. The event information in the event label information corresponding to the event snapshot image may be relevant information about the event corresponding to the event snapshot image. The relevant information about the event may be, for example, facial information, pet information, face tagging, stranger tagging, etc.

[0088] In other embodiments, when a user terminal displays a target event screen via a screen display page, the user can obtain the desired information by viewing the target event screen. However, in some scenarios, the user may not be able to obtain all the desired information from the target event screen, or may require other information as a reference when viewing the target event screen. For example, the user may want information about the time a certain event occurred at a certain location. In this case, simply displaying the target event screen may not meet the user's needs.

[0089] To address the above potential issues, the user terminal can display the target event screen on the screen display page, along with the event tag information corresponding to the target event screen. This allows the user to obtain the desired information based on the target event screen and the event tag information corresponding to the target event screen. The event tag information here is the same as the above event tag information and will not be repeated here.

[0090] It should be noted that in this embodiment, the user terminal can choose to display the event label information corresponding to each event snapshot image at the same time as displaying the event entry area, thereby resolving the problem that the user may be unable to select the target event snapshot image that meets their needs based on the image content of the event snapshot image. The user terminal can also choose to display the event label information corresponding to the target event image at the same time as displaying the target event image on the image display page, thereby resolving the problem that displaying only the target event image may not meet the user's needs. It is also possible to select both of the aforementioned options, thereby resolving the above two possible problems at the same time.

[0091] In this embodiment, the event label information corresponding to each event snapshot image can be displayed at the same time as the event entry area is displayed, thereby helping users understand each event snapshot image and allowing users to quickly select the target event snapshot image that meets their needs. Alternatively, when displaying the target event image on the image display page, the event label information corresponding to the target event image can be displayed, thereby helping users understand the target event image in more detail.

[0092] By displaying the event label information corresponding to each event snapshot, users can select the target event snapshot that meets their needs. The event label information corresponding to the target event screen can be displayed to help users obtain more information.

[0093] In addition, based on the above-mentioned embodiments, in this embodiment, before displaying the event entry area, the screen display method also includes: the user terminal displays the event configuration page in response to the configuration operation triggered on the event entry area; the event configuration page includes selection controls corresponding to multiple event types; the user terminal obtains the selected event type in response to the triggering operation of the selection control corresponding to each event type, and binds the selected event type to the event entry area; the bound event type is used to perform event recognition on the image data collected by the camera device to obtain event screens corresponding to each event.

[0094] Specifically, event types are related to events, and events under different event types are clearly distinct. In some embodiments, there is only one event under each event type, and therefore, the event type is the event itself. Event types include, but are not limited to, human recognition, cat and dog recognition, face recognition, motion detection, door and window opening, and door and window closing. It is understandable that the event types that different users may be interested in are generally not exactly the same, and even the event types that the same user may be interested in under different circumstances may not be completely different.

[0095] In some embodiments, to avoid missing event types that users may be interested in, the user terminal typically provides a large number of event snapshots corresponding to events of different event types for the user to use, which inevitably include event types that the user is not interested in. For event types that users are not interested in, users generally do not view event screens for events of that event type. Therefore, this approach is not targeted and requires processing a large number of event types, resulting in a waste of resources.

[0096] In response to the above-mentioned possible problems, in this embodiment, the user terminal can provide an event configuration page, which is a page for users to freely configure the event types that they may be interested in. Each selection control in the event configuration page corresponds to an event type, and the identification of the corresponding event type is displayed on the selection control. Among them, the identification of the event type is used to distinguish different event types, which can be text, pictures, etc. For example, if the event type is a face recognition type, the identification of the event type can be: "Face Recognition" or a picture of a face. The user selects the event type that he may be interested in from the event configuration page. Then, the user terminal binds the event type that the user may be interested in with the event entry area. In this way, only the event type that the user may be interested in needs to be processed, which is highly targeted for different users, thereby reducing the waste of resources.

[0097] In one embodiment, the event types corresponding to each selection control in the event configuration page are all event types that the image acquisition device can recognize, so as to provide the user with the widest range of choices. In another embodiment, the event types corresponding to each selection control in the event configuration page are commonly used event types, where the commonly used event types can be event types commonly used by most users obtained by counting a large number of different users, or can be event types commonly used by the current user obtained by counting the current user's historical records.

[0098] In this embodiment, multiple event types can be provided for users to choose from through the event configuration page. Users can select the event types that they may be interested in and bind them to the event entry area, so that they only need to process the event types that the users may be interested in, reducing waste of resources.

[0099] The above embodiments provide exemplary explanations of the process of the user terminal displaying the event screen and the process of the user interacting with the user terminal from the user's perspective. Correspondingly, the embodiments of the present application also provide a process and method for generating the event screen displayed by the user terminal.

[0100] Please refer to Figure 3. In an exemplary embodiment, an image data processing method is provided, and the method is described by taking the application of the method to an electronic device as an example. The electronic device can specifically be an electronic device such as the smart device 110, gateway 120, user terminal 150 or cloud server 140 in the implementation environment shown in Figure 1, or a camera device with specific image data processing capabilities. These electronic devices can all independently execute the image data processing method provided in this embodiment to provide the user terminal with an event snapshot image and a target event screen. For the sake of simplicity of description, the following uses electronic devices as a general term for devices such as smart devices 110, gateway 120, user terminal 150 or cloud server 140. Specifically, the image data processing method can be executed by an electronic device, as shown in Figure 3, and can include the following steps:

[0101] Step S301: The electronic device performs event recognition on the collected image data to obtain event screens corresponding to each event.

[0102] Among them, the image data is the video data captured by the camera device. The electronic device is communicatively connected to the camera device, and then the electronic device can obtain the video data from the camera device and perform event recognition on the obtained video data. In the implementation environment shown in Figure 1, the electronic device can be a smart device 110, and can also be a user terminal 150, a cloud server 140 or a gateway 120 with a camera function. The camera device can be a smart device 110, and can also be an electronic device such as a gateway 120 with a camera function. It can be understood that the electronic device and the camera device can be the same device. Event recognition refers to the use of image recognition technology to identify events recorded in image data. The event screen corresponding to the event is the part of the image data that records the event.

[0103] For ease of understanding, here we take the specific event of human figure recognition as an example. Suppose that there is no one entering or leaving a certain doorway from 10 a.m. on January 1, 2023 to 2 p.m. on the same day, but people are constantly entering and leaving from 2 p.m. on that day to 3 p.m. on the same day. The camera is facing the doorway, and starts shooting from 10 a.m. on January 1, 2023 to 3 p.m. on the same day, and generates a video file. The video file is the image data collected in this step. By performing human figure recognition on the video file, the electronic device can identify that the human figure recognition event is recorded within a certain time period of the video file, and the video data within this time period is the event screen corresponding to the human figure recognition event.

[0104] In some embodiments, in the above step S301, the electronic device performs event recognition on the collected image data to obtain event pictures corresponding to each event, which may include: the electronic device obtains image data, and performs event recognition on multiple frames of images in the image data in turn, and extracts video clips corresponding to each event according to the recognition results, respectively serving as event pictures corresponding to each event; or, obtains image data, and performs event recognition on multiple frames of images in the image data in turn, marks the video clips corresponding to each event in the image data according to the recognition results, and obtains video clips corresponding to each event, respectively serving as event pictures corresponding to each event.

[0105] Specifically, the image data captured by the camera device can be a complete video recording multiple events. It can also be multiple video segments, each recording a single event. Different processing methods are required for different situations. In some embodiments, the image data captured by the camera device is a complete video recording multiple events. In this case, the electronic device needs to determine which segment of the video corresponds to each event based on event recognition. The segment corresponding to each event is then extracted from the video as the event screen corresponding to each event. In one embodiment, for this complete video, the electronic device extracts one frame per second to obtain multiple frames of images. The electronic device then performs event recognition on the multiple frames of images to determine whether each frame records an event and which event it records. The electronic device counts the time periods corresponding to the multiple consecutive frames of images recording the same event and, for each time period, determines the video segment within that time period as the event screen corresponding to the corresponding event. It will be understood that the method by which the electronic device extracts images from the video is not limited to the above-mentioned method of extracting one frame per second. In another embodiment, the electronic device can also extract multiple frames of images per second, and after performing event recognition on the multiple frames of images extracted per second, select one of them as a representative to perform subsequent statistical records of the time period corresponding to the continuous multiple frames of images with the same event.

[0106] The solution of this embodiment is applicable to a camera device having only an image data acquisition function, that is, a camera device that can only take pictures. In this case, the camera device is only responsible for acquiring complete image data and then sending the acquired image data to the electronic device for subsequent processing.

[0107] In other embodiments, the image data captured by the camera device is a plurality of videos, each recording a single event. In this case, since each video records a single event, it is sufficient to extract at least one frame from each video for event recognition, and then determine the specific event recorded in the video based on the event recognition result and label it.

[0108] The solution of this embodiment is applicable to video cameras with both image data acquisition and event recognition capabilities, that is, devices such as cameras that can only analyze and process captured video. In this case, the video camera can identify the captured video. If it identifies an event or scene that meets the requirements, the corresponding video is stored, resulting in a video recording a single event. Other videos that do not meet the requirements are discarded and not stored. The video camera can then send the stored video and other image data to an electronic device for subsequent processing.

[0109] Therefore, regardless of whether the image data represents a complete video recording multiple events or multiple videos each recording a single event, it is sufficient to identify the video segments corresponding to each event after event recognition. These segments can then be extracted and processed as new videos, thus avoiding impacting the original video. Alternatively, the segments can be labeled within the original video and processed directly using the labeled original video, eliminating the need to generate additional videos.

[0110] Step S302: The electronic device extracts images corresponding to each event from the event screen corresponding to each event, and generates event snapshots corresponding to each event based on the images; the event snapshots are used for index display in the event entry area, and by responding to the viewing operation of the event snapshot in the event entry area, the target event screen corresponding to the event in the event snapshot is displayed.

[0111] The event screen corresponding to the event typically includes multiple frames of images, and the event snapshot image corresponding to the event is one of the multiple frames. It should be noted that the index display of the event snapshot image in the event entry area, the response to the viewing operation of the event snapshot image in the event entry area, and the display of the target event screen corresponding to the event of the event snapshot image in this step can all be referred to the embodiment of the screen display method described above, and will not be repeated here.

[0112] In some embodiments, in the above step S402, the electronic device extracts images corresponding to each event from the event screen corresponding to each event, and generates event snapshots corresponding to each event based on the images, which may include: the electronic device extracts each frame image included in the video clip corresponding to the current event, and parses each frame image corresponding to the current event to obtain multiple frames of images that meet the extraction conditions; wherein the extraction conditions include the clarity of the event subject being greater than a preset clarity, and / or the proportion of the event subject being greater than a preset proportion; and selecting one frame image from the multiple frame images that meet the extraction conditions as the event snapshot of the current event.

[0113] Specifically, for each event, while all frames of the video clip corresponding to the event record the event, the quality of each frame varies. Images serving as event snapshots need to be displayed to users. To help users accurately understand the image content of event snapshots, it is necessary to select high-quality images as event snapshots.

[0114] It is understandable that the higher the clarity of the event subject, the clearer the event subject. Accordingly, the easier it is for the user to distinguish the event subject in the image, the better the quality of the image to which the event subject belongs. The larger the proportion of the event subject, the more obvious the event subject. Accordingly, the easier it is for the user to distinguish the event subject in the image, the better the quality of the image to which the event subject belongs. Therefore, parameters for measuring image quality include but are not limited to the clarity of the event subject and the proportion of the event subject.

[0115] In this embodiment, the preset clarity is a predetermined clarity threshold. This preset clarity is used to measure whether an image is clear or blurry. If the clarity of an image exceeds this preset clarity threshold, the image can be considered clear. The preset ratio is a predetermined ratio threshold. This preset ratio is used to measure whether the subject of an event is conspicuous. If the proportion of the subject of an event in an image exceeds this preset ratio, the subject of the event can be considered conspicuous in the image.

[0116] In this embodiment, by screening the video clips corresponding to the event based on clarity and / or the proportion of the event subject, an image with clear image and / or obvious event subject can be obtained, which is then used as the event snapshot, thereby ensuring the image quality of the event snapshot.

[0117] In some embodiments, after determining the event snapshot image, the electronic device may further annotate the event snapshot image with event tag information. Here, the event tag information includes at least one of spatial information, temporal information, and event information. The spatial information may be the location of the event corresponding to the event snapshot image. The location may be, for example, geographic coordinates of latitude and longitude, a room identifier, etc. The temporal information may be, for example, the time at which the event corresponding to the event snapshot image occurred.

[0118] The occurrence time may be, for example, a timestamp or a time in any time zone. The event information may be information related to the event corresponding to the event snapshot. The event-related information may include facial information, pet information, face tags, stranger tags, etc. In some other embodiments, the event tag information may also include the name and index address of the event screen corresponding to the event snapshot.

[0119] In the above-mentioned image data processing method, the electronic device obtains event pictures corresponding to each event by performing event recognition on the collected image data. Then, the electronic device extracts the image corresponding to each event from the event picture corresponding to each event, and generates an event snapshot picture corresponding to each event based on the image. The user terminal can help users quickly locate the picture corresponding to a specific event by indexing and displaying the event snapshot picture in the event entry area. That is, the user can select and view the event snapshot pictures corresponding to each event included in the event entry area through the event entry area displayed by the user terminal, so as to quickly view the target event picture corresponding to the event of the selected event snapshot picture without viewing the complete video. Therefore, the user's operation steps can be effectively reduced, thereby improving the user experience, and at the same time, the viewing rate of the video picture can also be improved.

[0120] Based on the above-mentioned embodiments, in this embodiment, the image data processing method also includes: the electronic device merges at least a portion of the event snapshot images corresponding to each event to generate an event snapshot image set; the event snapshot image set is used to dynamically display in the event entrance area, and by responding to the viewing operation of the event snapshot image currently displayed in the event entrance area, the target event screen corresponding to the event of the event snapshot image is displayed.

[0121] Specifically, the user terminal can dynamically and rotate the event snapshots corresponding to some or all events in the event entrance area in the form of a collection. In some embodiments, when the user terminal selects an event snapshot to enter the event snapshot collection, it can select the event snapshot obtained based on the recently taken event screen to add to the event snapshot collection according to the shooting time of the event screen. For example, from the event snapshots corresponding to each event, filter out the event snapshots obtained based on the event screen taken within the last month, merge them, and generate an event snapshot collection. Of course, the shooting time is not limited to the last month. Users can also set the shooting time according to their own needs.

[0122] In this embodiment, event snapshots corresponding to some events can be selected and dynamically displayed in the event entry area in turn, which not only makes it convenient for users to select from various event snapshots; but also makes the user's selection range more flexible and varied.

[0123] Example application scenarios

[0124] Referring to Figure 4, the present application also provides an application scenario, which can be combined with the above-mentioned screen display method and image data processing method, so that users can view the corresponding event screen according to their needs. Taking the electronic device as a camera device as an example, the camera device can record video and store the recorded video in the camera device storage unit. The camera device and the user terminal are both communicatively connected to the network device. The network device can be a cloud server or a gateway. The gateway can refer to a home hub in the Internet of Things platform, which has a strong processing capability. It should be noted that the gateway can be a different device in the Internet of Things platform provided by different merchants, and this is not limited here. An application is installed on the user terminal, and the user terminal can access the data stored on the network device through the application.

[0125] As shown in Figure 4, the application of the above method in this application scenario is as follows:

[0126] Step S401: The user configures the dynamic photo album.

[0127] In this step, the user can configure one or more events that he or she may be interested in in the application of the user terminal. Specifically, if the dynamic album has not been configured, the user can enter the event configuration page through the dynamic album control in the application, and then the user can configure the dynamic album on the event configuration page according to his or her needs.

[0128] As shown in FIG5 , a dynamic album control 51 may be provided on the homepage of the application. If the dynamic album has not been configured, the dynamic album control 51 may display a message prompting the user to configure the photo, such as the message “Click to configure key events to take photos” in FIG5 .

[0129] By clicking on the dynamic album control 51, the event configuration page shown in Figure 6 can be displayed. The event configuration page includes a first selection control 61 for human recognition, a second selection control 62 for cat and dog recognition, a third selection control 63 for face recognition, a fourth selection control 64 for motion detection, a fifth selection control 65 for opening doors and windows, and a sixth selection control 66 for closing doors and windows. After selecting the first selection control 61 and the second selection control 62, the user clicks the completion control 67 to complete the configuration of the dynamic album. Of course, Figure 6 is only an example, and the event configuration page of this application includes but is not limited to the selection controls shown in Figure 6.

[0130] After completing the configuration of the dynamic album, the dynamic album control on the home page is shown in Figure 7, where a snapshot image 71 and video information 72 of the video corresponding to the snapshot image are displayed on the dynamic album control 70. The words "Click to configure the key event to take a photo" disappear.

[0131] Similarly, as shown in FIG8 , a dynamic album control 81 can also be provided on the camera album page. If a dynamic album has not yet been configured, a message prompting the user to configure it will be displayed on the dynamic album control 81, such as the message "Not Configured" in FIG8 . Clicking the dynamic album control 81 will also display the event configuration page shown in FIG7 . It is worth noting that after the dynamic album is configured, the dynamic album control changes accordingly. As shown in FIG9 , the message "Not Configured" in the dynamic album control 91 disappears.

[0132] Step S402: The camera device triggers video recording according to the configuration information.

[0133] After the dynamic album is configured, the camera device will begin recording video after detecting the configured event and stop recording after the event ends, thereby obtaining a video recording of the event. The event detected here is the event configured in step S401 above. After the user configures the relevant event on the user terminal, the configuration information can be transmitted to the camera device. For example, the configuration information is transmitted to the camera device via a network device.

[0134] Specifically, the key frame images captured by the camera device are subjected to algorithmic recognition processing through the local artificial intelligence recognition algorithm of the camera device. For example, three video frames can be taken within 1 second, and then the images of the video frames can be recognized to determine whether the events configured by the user appear in the images of the video frames. The artificial intelligence recognition algorithm here is a model algorithm obtained in advance through model training, which can be used to identify various events. When it is determined that any event configured by the user appears in the image of the video frame, the recording function is triggered. Since the data transmission process takes time, there will be a millisecond-level delay after the signal is received, and thus the camera device will be at risk of not being able to capture the image in the first time. The camera device of this embodiment is locally provided with a 2s buffer queue. After the recording function is triggered, past records can be found in the buffer queue, which can greatly reduce the risk of discarding important images.

[0135] Step S403: The camera device performs event marking and stores the video.

[0136] The camera stores recorded videos locally or on a cloud server. Each video is then tagged with an event. This tagged information includes, but is not limited to, the event ID, timestamp, video name, storage path, and index. Finally, each video and the tagged information are sent to a network device.

[0137] Step S404: The network device performs event marking and generates a snapshot graph.

[0138] The network device processes each video received separately to obtain a snapshot corresponding to each video. Here we only take the processing of one of the videos as an example to illustrate. First, the video is subjected to artificial intelligence recognition to extract the video frame in which the event subject in the video is the clearest and the event subject occupies the largest proportion of the screen. The extracted video frame is then used as the snapshot corresponding to the video. Among them, if the number of extracted video frames is greater than 1, any one of them can be selected as the snapshot corresponding to the video. Here, when judging the clarity of the event subject, the edge jaggedness of the event subject in the screen can be calculated by a preset algorithm, and the edge jaggedness of the event subject can be used to determine the clarity of the event subject. The proportion of the event subject in the screen is the percentage of the size of the event subject and the size of the overall screen of the video frame to which it belongs.

[0139] The network device can automatically merge all snapshots of the day every 24 hours, extract the snapshot details corresponding to each snapshot, and then merge these independent snapshots and store them in a file to generate a dynamic album and an array in the order of the snapshots. The array is recorded in a JSON (JS Object Notation) file. The elements of the array contain the snapshot details of the corresponding snapshot. The snapshot details include event tags, the local path and index of the video to which it belongs, and the corresponding timestamp. The event tag can contain spatial information (room, longitude and latitude), time information (timestamp, local time zone), and event information (face information, pet information, tagged faces or strangers, etc.). After the dynamic album is generated, the information can be pushed to the notification bar of the user's terminal.

[0140] Step S405: The user opens the application.

[0141] Step S406: The user terminal pulls the recently generated snapshot image from the network device.

[0142] The "recent" period can be the last day, week, etc. Here, the recently stored dynamic albums and their corresponding JSON files on the network device will be pulled. Specifically, when the user clicks the dynamic album notification information on the user terminal notification bar or opens the application to trigger the dynamic album control, the recently stored dynamic albums and their corresponding JSON files will be obtained from the network device.

[0143] Step S407: Generate and display a dynamic photo album based on the recently generated snapshots.

[0144] Here, each snapshot in the dynamic album obtained in step S406 is displayed in the dynamic album control of the application program. In this way, the user can know which events are recently recorded by viewing the dynamic album control.

[0145] Step S408: The user clicks on the target snapshot image in the dynamic album control.

[0146] Here, since each snapshot image in the dynamic album control is dynamically displayed, the user can select a snapshot image and click it according to his or her needs. The snapshot image selected by the user is the target snapshot image.

[0147] Step S409: The user terminal sends the snapshot image details corresponding to the target snapshot image in the JSON file to the network device, and the network device uses the snapshot image details to retrieve the corresponding target video.

[0148] Step S410: The network device returns the retrieved target video to the user terminal.

[0149] Step S411: The user terminal displays the target video to the user.

[0150] It should be noted that in order to avoid users from accidentally triggering the snapshot image in the dynamic album control. After the user clicks the target snapshot image, a prompt message can pop up, asking the user whether he needs to view the target video corresponding to the target snapshot image. When the user clicks to view the target video, the corresponding target video file is found by the camera device storage unit according to the video storage path and index in the snapshot image details information. The found target video is then played on the user terminal. When the user clicks to jump to the recording position, the corresponding position of the timeline can be located according to the timestamp to play back the event. Preferably, when displaying the target video, the corresponding position of the target video can be found according to the timestamp in the snapshot image details information corresponding to the target snapshot image, and then the target video can be played from that position. For example, if the timestamp indicates the seventh minute of the target video, then the target video will be played from the seventh minute when playing.

[0151] In some scenarios, users may simply need to view snapshots in detail. Therefore, you can also provide users with a portal to carefully review snapshots. Specifically, if a dynamic album is configured, you can use the dynamic album control on the homepage to provide a portal to view snapshots. Alternatively, you can use the dynamic album control on the camera album page to provide a portal to view snapshots. You can also set up a dedicated function control on the homepage to provide a portal to view snapshots.

[0152] For example, when a user opens the page shown in Figure 7, long-pressing the dynamic album control 70 can display the page shown in Figure 10, further asking the user whether to enter the album page. After the user clicks to view the album control 101, the album page shown in Figure 11 is displayed. The album page simultaneously displays multiple snapshots for the user to view and select, such as event snapshot 111 corresponding to the "Door and Window Open" event, event snapshot 112 corresponding to the "Human Figure Recognition" event, event snapshot 113 corresponding to the "Cat and Dog Recognition" event, event snapshot 114 corresponding to another "Human Figure Recognition" event, and event snapshots 115 and 116 corresponding to two other events. After the user clicks on one of the snapshots, an enlarged version of that snapshot is displayed for the user to view. For example, after the user clicks on event snapshot 111, the page shown in Figure 12 is displayed, followed by an enlarged version of the corresponding snapshot of the door and window opening. For another example, when the user opens the page shown in Figure 9, clicking the dynamic album control 91 can display the page shown in Figure 10. For details about Figure 10, please refer to the above description and will not be repeated here.

[0153] It should be noted that when snapshot images exist, the album page, as shown in Figure 11, can display the existing snapshot images. When no snapshot images exist, the album page, as shown in Figure 13, displays the words "No Data" in the target area 131. Preferably, as shown in the setting control 117 in Figure 11 and the setting control 132 in Figure 13, by designing a setting control in the upper right corner of the album page, it is possible to enter the event configuration page shown in Figure 6 from the album page to continue configuration.

[0154] In some embodiments, when the user opens the page shown in Figure 7, long pressing the dynamic album control 70 can provide the user with an entrance to carefully view the snapshot image. Clicking the dynamic album control 70 can enter the video playback page, play the video recording the event, and adjust the playback progress to the location where the event occurred. Assume that the user clicks the dynamic album control 70 when the dynamic album control 70 in Figure 7 displays the first event snapshot image. Among them, the first event snapshot image is an event snapshot image corresponding to the "doors and windows open" event. After clicking, you can jump to the playback page, play the video recording the "doors and windows open" event, and adjust the playback progress to the location where the "doors and windows open" event occurs.

[0155] For example, as shown in Figure 14 , the video frame 141 displayed on the play page shows a window 142 as open. Furthermore, based on the specific data displayed in the play area 143, it can be determined that the current playback position of the video frame 141 is 03:12:02 within the entire video. In other words, the playback progress has been adjusted to the point where the "door and window opened" event occurred.

[0156] In this embodiment, users can select key events of their interest, and then use event recognition to mark and store the key events captured by the camera in the image. Based on the video frames in the video, event snapshots of each event are generated, and dynamic albums are generated using each event snapshot and displayed to the user. Each event snapshot can be automatically played without user operation, and when the user clicks on the event snapshot in the dynamic album, the corresponding video screen can be quickly indexed without viewing the entire video. This can effectively reduce the user's operation steps, thereby improving the user experience and also increasing the viewing rate of the video screen.

[0157] It should be understood that, although the various steps in the flowcharts of Figures 2-4 are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in Figures 2-4 may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these sub-steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0158] Exemplary devices

[0159] In one embodiment, as shown in FIG15 , a picture display device is provided, the device comprising:

[0160] The first display module 151 is used to display the event entry area, which includes an event snapshot corresponding to each event; the event snapshot is generated based on an image related to each event identified from the collected image data;

[0161] The second display module 152 is configured to display a screen display page in response to a viewing operation on the selected target event snapshot image, and to display a target event screen corresponding to the event of the target event snapshot image on the screen display page.

[0162] In one embodiment, if there are multiple event snapshot images, the first display module 151 dynamically displays each event snapshot image in the event entry area.

[0163] In one embodiment, the second display module 152 is further configured to: display a dynamic album page in response to a triggering operation on the event entry area; the dynamic album page includes event snapshots corresponding to each generated event.

[0164] In one embodiment, the second display module 152 is also used to: display the album page corresponding to the camera device in response to a trigger operation on the event entry area; the album page includes at least a dynamic album entry; display the dynamic album page in response to a trigger operation on the dynamic album entry; the dynamic album page includes event snapshots corresponding to each generated event.

[0165] In one embodiment, when the second display module 152 displays the target event screen corresponding to the event of the target event snapshot on the screen display page, it is specifically used to: display the enlarged image and / or video clip corresponding to the target event snapshot on the screen display page.

[0166] In one embodiment, the first display module 151 is also used to: display the event label information corresponding to each event snapshot while displaying the event entry area; and / or, display the event label information corresponding to the target event screen while displaying the target event screen in the screen display page; wherein the event label information includes at least one of spatial information, time information and event information.

[0167] In one embodiment, the device also includes: a third display module, which is used to display an event configuration page in response to a configuration operation triggered on an event entry area; the event configuration page includes selection controls corresponding to multiple event types; a binding module, which is used to obtain a selected event type in response to a triggering operation on a selection control corresponding to each event type, and bind the selected event type to the event entry area; the bound event type is used to perform event recognition on image data collected by a camera device to obtain event pictures corresponding to each event.

[0168] For the specific definition of the screen display device, please refer to the definition of the screen display method above, which will not be repeated here. The various modules in the above-mentioned screen display device can be implemented in whole or in part by software, hardware, or a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0169] In addition, in one embodiment, as shown in FIG16 , an image data processing device is provided, the device comprising:

[0170] The recognition module 161 is used to perform event recognition on the collected image data to obtain event images corresponding to each event;

[0171] Generation module 162 is used to extract images corresponding to each event from the event screen corresponding to each event, and generate event snapshots corresponding to each event based on the images; the event snapshots are used to be indexed and displayed in the event entry area, and by responding to the viewing operation of the event snapshots in the event entry area, the target event screen corresponding to the event of the event snapshot is displayed.

[0172] In one embodiment, when the recognition module 161 performs event recognition on the collected image data and obtains event pictures corresponding to each event, it is specifically used to: obtain image data, and perform event recognition on multiple frames of images in the image data in turn, and extract video clips corresponding to each event according to the recognition results, respectively serving as event pictures corresponding to each event; or, obtain image data, and perform event recognition on multiple frames of images in the image data in turn, mark the video clips corresponding to each event in the image data according to the recognition results, and obtain video clips corresponding to each event, respectively serving as event pictures corresponding to each event.

[0173] In one embodiment, when the generation module 162 extracts images corresponding to each event from the event screen corresponding to each event, and generates event snapshots corresponding to each event based on the images, it is specifically used to: extract each frame image included in the video clip corresponding to the current event, and parse each frame image corresponding to the current event to obtain multiple frame images that meet the extraction conditions; wherein the extraction conditions include the clarity of the event subject being greater than a preset clarity, and / or the proportion of the event subject being greater than a preset proportion; and select one frame image from the multiple frame images that meet the extraction conditions as the event snapshot of the current event.

[0174] In one embodiment, the generation module 162 is further used to: merge at least a portion of the event snapshot images corresponding to each of the events to generate an event snapshot image set; the event snapshot image set is used to dynamically display in the event entry area, and to display the target event screen corresponding to the event of the event snapshot image by responding to the viewing operation of the event snapshot image currently displayed in the event entry area.

[0175] The specific definition of the image data processing device can be found in the definition of the image data processing method above and will not be repeated here. The various modules in the above-mentioned image data processing device can be implemented in whole or in part through software, hardware, or a combination thereof. The above-mentioned modules can be embedded in or independent of the processor of the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each of the above modules.

[0176] Exemplary electronic devices

[0177] In one embodiment, an electronic device is provided, comprising at least one processor and at least one memory, wherein the memory stores computer-readable instructions; when the computer-readable instructions are executed by the processor, the processor implements the steps in the above-mentioned method embodiments.

[0178] Another embodiment of the present application further proposes an electronic device, the internal structure of which may be as shown in FIG17 , wherein the electronic device includes a processor, a memory, a network interface, and an input device connected via a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, the steps in the above-mentioned method embodiments are implemented.

[0179] The processor may include a main processor, and may also include a baseband chip, a modem, etc.

[0180] The memory stores a computer program that executes the technical solution of the present application, and may also store an operating system and other key services. Specifically, the computer program may include program code, and the program code includes computer operating instructions. More specifically, the memory may include read-only memory (ROM), other types of static storage devices that can store static information and instructions, random access memory (RAM), other types of dynamic storage devices that can store information and instructions, disk storage, flash, etc.

[0181] The processor may be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, or an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present application. It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component.

[0182] Input devices may include devices that receive data and information input by a user, such as a keyboard, mouse, camera, scanner, light pen, voice input device, touch screen, pedometer, or gravity sensor.

[0183] Output devices may include means that allow information to be output to a user, such as display screens, printers, speakers, and the like.

[0184] The communication interface may include any device such as a transceiver to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0185] The processor executes the computer program stored in the memory and calls other devices, which can be used to implement the steps in the above-mentioned method embodiments of the present application.

[0186] The electronic device may further include a display component and a voice component. The display component may be a liquid crystal display or an electronic ink display. The input device of the electronic device may be a touch layer covering the display component, or a button, trackball or touchpad provided on the housing of the electronic device, or an external keyboard, touchpad or mouse.

[0187] Those skilled in the art will understand that the structure shown in Figure 17 is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0188] Exemplary computer program products and computer-readable storage media

[0189] In one embodiment, a computer program product or computer program is provided, which includes computer-readable instructions, and the computer-readable instructions are stored in a computer-readable storage medium; a processor of a computer device reads the computer-readable instructions from the computer-readable storage medium, and when the processor executes the computer-readable instructions, the steps in the screen display method or image data processing method of each embodiment of the present application are implemented.

[0190] The computer program product may be written in any combination of one or more programming languages ​​to implement the operations of the embodiments of the present application. The programming languages ​​include object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the device or as a standalone software package.

[0191] In one embodiment, a computer-readable storage medium is provided, on which computer-readable instructions are stored. When the computer-readable instructions are executed by one or more processors, the various steps of the screen display method or image data processing method shown in the various embodiments of the above exemplary method part are implemented.

[0192] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through computer-readable instructions, and the computer-readable instructions can be stored in a non-volatile computer-readable storage medium. When the computer-readable instructions are executed, they may include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0193] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0194] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A method for displaying a picture, wherein: The method is performed by a user terminal electronic device, and includes: Displaying an event entry area, the event entry area includes an event snapshot corresponding to each event; the event snapshot is generated based on an image related to each event identified from the collected image data; In response to a viewing operation on a selected target event snapshot, a screen display page is displayed, and a target event screen corresponding to the event of the target event snapshot is displayed on the screen display page.

2. The method according to claim 1, characterized in that: If there are multiple event snapshots, each of the event snapshots is dynamically displayed in the event entry area.

3. The method according to claim 1, characterized in that The method further comprises: In response to a trigger operation on the event entry area, a dynamic album page is displayed; the dynamic album page includes event snapshots corresponding to each generated event.

4. The method according to claim 1, characterized in that: The event entry area is used to indicate the entry for the event image collected by the camera device; the method also includes: In response to a triggering operation on the event entry area, displaying an album page corresponding to the camera device; the album page at least includes a dynamic album entry; In response to a trigger operation on the dynamic album entry, a dynamic album page is displayed; the dynamic album page includes event snapshots corresponding to each generated event.

5. The method according to claim 1, characterized in that The displaying of the target event screen corresponding to the event of the target event snapshot image in the screen display page includes: The enlarged image and / or video clip corresponding to the target event snapshot image is displayed in the image display page.

6. The method according to claim 1, characterized in that The method further comprises: While displaying the event entry area, displaying the event label information corresponding to each event snapshot; and / or, while displaying the target event screen in the screen display page, displaying event tag information corresponding to the target event screen; The event tag information includes at least one of spatial information, time information and event information.

7. The method according to any one of claims 1 to 6, characterized in that: Before displaying the event entrance area, the method further includes: In response to a configuration operation triggered on the event entry area, an event configuration page is displayed; the event configuration page includes selection controls corresponding to a plurality of event types; In response to the triggering operation of the selection control corresponding to each of the event types, the selected event type is obtained and bound to the event entry area; the bound event type is used to perform event recognition on the image data collected by the camera device to obtain event pictures corresponding to each event.

8. A method for processing image data, wherein: Executed by an electronic device, the method includes: Perform event recognition on the collected image data to obtain event pictures corresponding to each event; Extract images corresponding to each event from the event screen corresponding to each event, and generate event snapshots corresponding to each event based on the images; the event snapshots are used for index display in the event entry area, and by responding to the viewing operation of the event snapshots in the event entry area, the target event corresponding to the event in the event snapshot is displayed. picture.

9. The method according to claim 8, characterized in that The event recognition is performed on the collected image data to obtain event pictures corresponding to each event, including: Acquire image data, and perform event recognition on multiple frames of images in the image data in sequence, and extract video clips corresponding to each event according to the recognition results as event pictures corresponding to each event; Alternatively, image data is acquired, and event recognition is performed on multiple frames of images in the image data in turn, and video clips corresponding to each event in the image data are marked according to the recognition results to obtain video clips corresponding to each event as event pictures corresponding to each event.

10. The method according to claim 8, characterized in that The step of extracting images corresponding to each event from event screens corresponding to each event, and generating event snapshots corresponding to each event according to the images includes: Extracting each frame of the video clip corresponding to the current event, and parsing each frame of the video clip corresponding to the current event to obtain multiple frames of images that meet extraction conditions; wherein the extraction conditions include that the clarity of the event subject is greater than a preset clarity, and / or the proportion of the event subject is greater than a preset proportion; One frame of image is selected from multiple frames of images that meet the extraction condition as an event snapshot image of the current event.

11. The method according to any one of claims 8 to 10, characterized in that: The method further comprises: At least a portion of the event snapshot images corresponding to each of the events are merged to generate a set of event snapshot images; the set of event snapshot images is used for dynamic and rotational display in the event entry area, and by responding to a viewing operation on the event snapshot image currently displayed in the event entry area, a target event screen corresponding to the event of the event snapshot image is displayed.

12. A picture display device, wherein: The device comprises: A first display module is used to display an event entry area, wherein the event entry area includes an event snapshot corresponding to each event; the event snapshot is generated based on an image related to each event identified from the collected image data; The second display module is used to display a picture display page in response to a viewing operation on the selected target event snapshot picture, and to display a target event picture corresponding to the event of the target event snapshot picture in the picture display page.

13. An image data processing device, wherein: The device comprises: The recognition module is used to perform event recognition on the collected image data to obtain event pictures corresponding to each event; A generation module is used to extract images corresponding to each event from event screens corresponding to each event, and generate event snapshots corresponding to each event based on the images; the event snapshots are used to be indexed and displayed in an event entry area, and by responding to a viewing operation on the event snapshots in the event entry area, a target event screen corresponding to the event in the event snapshot is displayed.

14. An electronic device comprising at least one processor and at least one memory, wherein the memory stores computer-readable instructions, wherein: When the computer-readable instructions are executed by the processor, the steps of the screen display method described in any one of claims 1 to 7 are implemented, or the steps of the image data processing method described in any one of claims 8 to 11 are implemented.

15. A computer-readable storage medium having computer-readable instructions stored thereon, wherein: When the computer-readable instructions are executed by one or more processors, the steps of the screen display method described in any one of claims 1 to 7 are implemented, or the steps of the image data processing method described in any one of claims 8 to 11 are implemented.

Images