Environmental event detection method and related device

By acquiring the difference data of the reference image sequence and the target image captured by the camera, the events within the camera shooting range are detected, and the problem of high error detection rate in the prior art is solved, and the accuracy and robustness of event detection are improved.

CN119091341APending Publication Date: 2024-12-06TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310659029.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-06-05
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The existing camera-based environmental event detection methods are poor in actual applications and are prone to misdetection, especially when background interference or lighting changes.

Method used

By acquiring the reference image sequence corresponding to the target image captured by the camera, combining the image difference between the adjacent reference image and the adjacent image every two frames, the first and second reference data are constructed for detecting the camera shooting range. Whether the target trigger event occurred.

Benefits of technology

It effectively avoids false detection caused by basic dynamic situations (such as environmental interference and lighting changes), and improves the accuracy and robustness of event detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091341A_ABST
    Figure CN119091341A_ABST
Patent Text Reader

Abstract

The embodiment of the invention discloses an environment event detection method and a related device, and the method comprises the steps: obtaining a reference image sequence corresponding to a target image shot by a camera, and the reference image sequence comprises n frames of reference images shot by the camera before the target image is shot, the time difference between the shooting time of the reference image and the shooting time of the target image meets a preset time condition; determining first reference data according to the image difference between every two adjacent frames of reference images in the reference image sequence; determining second reference data according to the image difference between the target image and the corresponding adjacent image; and based on the first reference data and the second reference data, detecting whether a target trigger event occurs in the shooting range of the camera when the camera shoots the target image. According to the method, the occurrence of false detection can be reduced, and the accuracy and robustness of event detection are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an environmental event detection method and related devices. Background Art

[0002] Nowadays, camera-based environmental event detection methods are widely used in various scenarios, such as security environment monitoring, home environment monitoring, etc. Such camera-based environmental event detection methods are often used to detect whether specific objects (such as people, animals, vehicles, etc.) appear within the camera's shooting range.

[0003] In the related art, an inter-frame difference algorithm is usually used to detect environmental events based on images captured by a camera. Specifically, the difference between two adjacent frames of images captured by the camera can be calculated. If the difference is greater than a preset difference threshold, it is determined that a trigger event has occurred within the camera's shooting range. However, in actual applications, the accuracy and robustness of the above environmental event detection methods are poor, and false detections are very likely to occur. For example, when there is interference or light changes in the shooting background, it is easy to falsely detect that a trigger event has occurred. Summary of the invention

[0004] The embodiments of the present application provide an environmental event detection method and related devices, which can reduce the occurrence of false detections and improve the accuracy and robustness of event detection.

[0005] In view of this, the first aspect of the present application provides an environmental event detection method, the method comprising:

[0006] Acquire a reference image sequence corresponding to a target image captured by a camera; the reference image sequence includes n frames of reference images captured by the camera before capturing the target image, where n is an integer greater than 1, and the time difference between the capturing time of the reference image and the capturing time of the target image satisfies a preset time condition;

[0007] Determining first reference data according to an image difference between every two adjacent reference images in the reference image sequence;

[0008] determining second reference data according to an image difference between the target image and an adjacent image corresponding to the target image; the adjacent image is an image adjacent to the target image captured by the camera before capturing the target image;

[0009] Based on the first reference data and the second reference data, it is detected whether a target triggering event occurs within the shooting range of the camera when the camera shoots the target image.

[0010] A second aspect of the present application provides an environmental event detection device, the device comprising:

[0011] A sequence acquisition module, used to acquire a reference image sequence corresponding to a target image captured by a camera; the reference image sequence includes n frames of reference images captured by the camera before capturing the target image, where n is an integer greater than 1, and the time difference between the capturing time of the reference image and the capturing time of the target image satisfies a preset time condition;

[0012] A first determination module, configured to determine first reference data according to an image difference between every two adjacent reference images in the reference image sequence;

[0013] A second determination module is used to determine second reference data according to an image difference between the target image and an adjacent image corresponding to the target image; the adjacent image is an image adjacent to the target image captured by the camera before capturing the target image;

[0014] An event detection module is used to detect whether a target triggering event occurs within the shooting range of the camera when the camera shoots the target image based on the first reference data and the second reference data.

[0015] Optionally, the first determining module is specifically configured to:

[0016] Based on the n reference image frames in the reference image sequence, construct n-1 reference image pairs; each of the reference image pairs includes two reference image frames with adjacent shooting times;

[0017] For each of the reference image pairs, determining image difference data corresponding to the reference image pair according to an image difference between two frames of the reference images included in the reference image pair;

[0018] The average values ​​of the image difference data corresponding to each of the n-1 reference image pairs are calculated as the first reference data.

[0019] Optionally, the first determining module is specifically configured to:

[0020] Performing grayscale processing on the two frames of the reference image respectively to obtain grayscale images corresponding to the two frames of the reference image respectively; determining image difference data between the two frames of the reference image based on the grayscale images corresponding to the two frames of the reference image respectively;

[0021] The second determining module is specifically used for:

[0022] Grayscale processing is performed on the target image and the adjacent image respectively to obtain grayscale images corresponding to the target image and the adjacent image respectively; and the second reference data is determined based on the image difference between the grayscale images corresponding to the target image and the adjacent image respectively.

[0023] Optionally, the first determining module is specifically configured to:

[0024] Determine the overall grayscale value corresponding to each frame of the reference image according to the grayscale value of each pixel in the grayscale image corresponding to each frame of the reference image; calculate the difference between the overall grayscale values ​​corresponding to the two frames of the reference image as the image difference data;

[0025] The second determining module is specifically used for:

[0026] Determine the overall grayscale value corresponding to the target image according to the grayscale value of each pixel in the grayscale image corresponding to the target image; determine the overall grayscale value corresponding to the adjacent image according to the grayscale value of each pixel in the grayscale image corresponding to the adjacent image;

[0027] The difference between the overall grayscale value corresponding to the target image and the overall grayscale value corresponding to the adjacent image is calculated as the second reference data.

[0028] Optionally, the first determining module is specifically configured to:

[0029] Constructing first pixel pairs based on pixel points at the same position in the grayscale images corresponding to the two frames of the reference images; determining the image difference data according to the grayscale difference values ​​corresponding to each of the first pixel pairs;

[0030] The second determining module is specifically used for:

[0031] Constructing second pixel pairs based on the pixel points located at the same position in the grayscale image corresponding to the target image and the grayscale image corresponding to the adjacent image; determining the second reference data according to the grayscale difference values ​​corresponding to each of the second pixel pairs;

[0032] The grayscale difference corresponding to the pixel pair is the difference between the grayscale values ​​corresponding to the two pixel points.

[0033] Optionally, the first determining module is specifically configured to:

[0034] Through the image feature extraction model, feature extraction processing is performed on the two frames of the reference image respectively to obtain image feature data corresponding to the two frames of the reference image respectively; based on the image feature data corresponding to the two frames of the reference image respectively, image difference data between the two frames of the reference image is determined;

[0035] The second determining module is specifically used for:

[0036] The target image and the adjacent image are subjected to feature extraction processing respectively through the image feature extraction model to obtain image feature data corresponding to the target image and the adjacent image respectively; and the second reference data is determined based on the image feature data corresponding to the target image and the adjacent image respectively.

[0037] Optionally, the first determining module is specifically configured to:

[0038] Determine the image difference data between the two frames of the reference image according to the two frames of the reference image by means of an image difference recognition model; the image difference recognition model is used to determine the image difference data used to characterize the difference between the two images inputted;

[0039] The second determining module is specifically used for:

[0040] The second reference data is determined according to the target image and the adjacent image by using the image difference recognition model.

[0041] Optionally, the event detection module is specifically used for:

[0042] When the first reference data is less than the first preset threshold, if the second reference data is greater than or equal to the second preset threshold, it is determined that a target trigger event occurs within the shooting range of the camera when the camera captures the target image; if the second reference data is less than the second preset threshold, it is determined that no target trigger event occurs within the shooting range of the camera when the camera captures the target image.

[0043] Optionally, the event detection module is specifically used for:

[0044] When the first reference data is greater than or equal to a first preset threshold, calculating a reference data difference between the second reference data and the first reference data;

[0045] If the reference data difference is greater than or equal to a third preset threshold, it is determined that a target trigger event occurs within the shooting range of the camera when the camera captures the target image; if the reference data difference is less than the third preset threshold, it is determined that no target trigger event occurs within the shooting range of the camera when the camera captures the target image.

[0046] Optionally, the camera is installed on the target device; the apparatus further includes:

[0047] A state switching module is used to control the target device to enter a standby state if the target trigger event is detected within the shooting range of the camera when the target device is in a sleep state; the power consumption of the target device in the standby state is higher than the power consumption of the target device in the sleep state.

[0048] Optionally, the state switching module is further used for:

[0049] Recording the standby time from the moment when the target device enters the standby state;

[0050] If the standby time reaches a preset time and the target triggering event is not detected within the shooting range of the camera within the standby time, the target device is controlled to enter the sleep state.

[0051] A third aspect of the present application provides a computer device, the device comprising a processor and a memory:

[0052] The memory is used to store computer programs;

[0053] The processor is used to execute the steps of the environmental event detection method as described in the first aspect according to the computer program.

[0054] A fourth aspect of the present application provides a computer-readable storage medium, which is used to store a computer program, and the computer program is used to execute the steps of the environmental event detection method described in the first aspect above.

[0055] In a fifth aspect, the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of the environmental event detection method described in the first aspect above.

[0056] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0057] The embodiment of the present application provides an environmental event detection method, in which a reference image sequence corresponding to a target image captured by a camera is obtained, the reference image sequence includes n (n is an integer greater than 1) frames of reference images captured by the camera before capturing the target image, and the time difference between the capturing time of each frame of the reference image and the capturing time of the target image satisfies a preset time condition. Then, according to the image difference between each two adjacent reference images in the reference image sequence, a first reference data is determined; the first reference data can reflect the basic dynamic situation within the shooting range of the camera before the camera captures the target image to a certain extent. And according to the image difference between the target image and its corresponding adjacent image, a second reference data is determined, the adjacent image is an image captured by the camera before capturing the target image and adjacent to the target image; the second reference data can reflect the dynamic situation within the shooting range of the camera when the camera captures the target image. Finally, based on the above-mentioned first reference data and second reference data, it is determined whether a target trigger event occurs within the shooting range of the camera when the camera captures the target image. In this way, the basic dynamic conditions that originally exist within the shooting range of the camera are taken into consideration, and combined with the first reference data that can reflect the basic dynamic conditions and the second reference data that can reflect the dynamic conditions within the shooting range when shooting the target image, target trigger event detection is performed. This can effectively avoid false detections caused by basic dynamic conditions (such as environmental interference, lighting changes, etc.), thereby improving the accuracy and robustness of trigger event detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 A schematic diagram of an application scenario of the environmental event detection method provided in an embodiment of the present application;

[0059] Figure 2 A schematic diagram of a process flow of an environmental event detection method provided in an embodiment of the present application;

[0060] Figure 3 A schematic diagram of a target image and its corresponding reference image sequence provided in an embodiment of the present application;

[0061] Figure 4 A schematic diagram of another target image and its corresponding reference image sequence provided in an embodiment of the present application;

[0062] Figure 5 A schematic diagram of a process for determining first reference data provided in an embodiment of the present application;

[0063] Figure 6 A schematic diagram of an application of a working state switching mechanism for a target device provided in an embodiment of the present application;

[0064] Figure 7A schematic diagram of the structure of an environmental event detection device provided in an embodiment of the present application;

[0065] Figure 8 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application;

[0066] Fig. 9 A schematic diagram of the structure of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0067] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0068] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0069] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0070] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operating / interactive systems, mechatronics and other technologies. Artificial intelligence software technologies mainly include computer vision technology, speech processing technology, natural language processing technology, as well as machine learning / deep learning, autonomous driving, smart transportation and other major directions.

[0071] Computer vision technology (CV) is a science that studies how to make machines "see". To put it more specifically, it refers to machine vision such as using cameras and computers to replace human eyes to identify and measure targets, and further perform graphic processing to make computer processing into images that are more suitable for human eye observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and mapping, automatic driving, smart transportation and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0072] The solution provided in the embodiments of the present application relates to computer vision technology in artificial intelligence technology, which is specifically described by the following embodiments:

[0073] The environmental event detection method provided in the embodiment of the present application can be executed by a computer device with image processing capabilities, which can be a terminal device or a server. Among them, the terminal device includes but is not limited to a mobile phone, a computer, an intelligent voice interaction device, a smart home appliance, a vehicle terminal, an aircraft, etc. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server.

[0074] It should be noted that the information, data (including but not limited to image data, etc.) and signals involved in the embodiments of the present application are authorized by the relevant objects or fully authorized by all parties, and the collection, use and processing of relevant data comply with the relevant laws, regulations and standards of relevant countries and regions.

[0075] To facilitate understanding of the environmental event detection method provided in the embodiment of the present application, the following uses an example in which the execution subject of the environmental event detection method is a terminal device equipped with a camera as an example to exemplify the application scenario of the environmental event detection method.

[0076] See also Figure 1, Figure 1 Schematic diagram of application scenarios of the environmental event detection method provided in the embodiment of the present application. Figure 1 As shown, the application scenario includes a terminal device 110, Figure 1 In the example, the terminal device 110 is an environmental monitor. The terminal device 110 can collect environmental images through the camera installed thereon, and adopt the environmental event detection method provided in the embodiment of the present application to detect whether a target triggering event occurs within the shooting range of the camera based on the environmental image collected by the camera.

[0077] In actual applications, the camera installed on the terminal device 110 can continuously capture images and transmit the captured images to the processor of the terminal device 110 .

[0078] When it is necessary to detect whether a target trigger event occurs within the shooting range of the camera when the camera is shooting a target image, the processor of the terminal device 110 can obtain a reference image sequence corresponding to the target image, and the reference image sequence includes n frames of reference images taken by the camera before shooting the target image, where n is an integer greater than 1, and the time difference between the shooting time of each frame of the reference image in the reference image sequence and the shooting time of the target image meets the preset time condition, for example, the time difference between the shooting time of each frame of the reference image and the shooting time of the target image does not exceed 3s.

[0079] Furthermore, the processor of the terminal device 110 can determine the first reference data based on the image difference between each two adjacent reference images in the reference image sequence; the first reference data can reflect to a certain extent the basic dynamic situation within the shooting range of the camera before the camera captures the target image. In addition, the processor of the terminal device 110 can also determine the second reference data based on the image difference between the target image and its corresponding adjacent image, the adjacent image being an image adjacent to the target image captured by the camera before the target image is captured; the second reference data can reflect the dynamic situation within the shooting range of the camera when the camera captures the target image.

[0080] Finally, the processor of the terminal device 110 can determine whether a target trigger event occurs within the shooting range of the camera when the camera captures the target image based on the above-mentioned first reference data and second reference data. In this way, the basic dynamic conditions originally existing within the camera shooting range are taken into consideration, and the target trigger event is detected by combining the first reference data that can reflect the basic dynamic conditions and the second reference data that can reflect the dynamic conditions within the shooting range when the target image is captured. This can effectively avoid false detections caused by basic dynamic conditions (such as environmental interference, lighting changes, etc.), thereby improving the accuracy and robustness of trigger event detection.

[0081] Optionally, if the processor of the terminal device 110 detects the occurrence of a target triggering event, it may correspondingly control the terminal device 110 to perform related operations, such as executing an event alarm operation, a device operating state switching operation, and the like.

[0082] It should be understood that Figure 1 The application scenarios shown are only examples. In actual applications, the environmental event detection method provided by the embodiment of the present application can also be applied to other scenarios; for example, the image captured by the camera can be transmitted to the server, and the server executes the environmental event detection method to detect whether a target triggering event occurs within the shooting range of the camera. The application scenarios of the environmental event detection method provided by the embodiment of the present application are not limited in any way.

[0083] The environmental event detection method provided by the present application is introduced in detail below through a method embodiment.

[0084] See also Figure 2 , Figure 2 The following is a flow chart of the environmental event detection method provided in the embodiment of the present application. For the convenience of description, the following embodiment is still introduced by taking the execution subject of the environmental event detection method as a terminal device as an example. Figure 2 As shown, the environmental event detection method includes the following steps:

[0085] Step 201: Obtain a reference image sequence corresponding to a target image captured by a camera; the reference image sequence includes n frames of reference images captured by the camera before capturing the target image, where n is an integer greater than 1, and the time difference between the capturing time of the reference image and the capturing time of the target image satisfies a preset time condition.

[0086] In an embodiment of the present application, when a terminal device needs to detect whether a target trigger event occurs within the shooting range of the camera when the camera captures a target image, the terminal device needs to obtain the target image captured by the camera and the reference image sequence corresponding to the target image.

[0087] Exemplarily, when there is a direct data transmission channel between the camera and the terminal device, the terminal device can directly obtain each frame of images transmitted by the camera, and determine the target image and the reference image sequence corresponding to the target image accordingly. Alternatively, the camera can also transmit each frame of images collected to a data storage area, and the terminal device obtains each frame of images collected by the camera from the data storage area, and determines the target image and the reference image sequence corresponding to the target image accordingly. The embodiments of the present application do not impose any limitation on the manner in which the terminal device obtains the target image and its corresponding reference image sequence.

[0088] It should be noted that the target image in the embodiment of the present application can be any frame image captured by the camera. For example, when the embodiment of the present application is applied in a scene of real-time environmental monitoring, the target image can be the image currently captured by the camera; when the embodiment of the present application is applied in a scene of environmental monitoring based on historical image data, the target image can be a frame image captured historically by the camera. The embodiment of the present application does not impose any limitation on the acquisition time of the target image.

[0089] It should be noted that the reference image sequence corresponding to the target image is composed of n reference images taken by the camera before the target image is taken, where n is an integer greater than 1, and the n reference images in the reference image sequence can be arranged in order from front to back according to the shooting time. The reference image here refers to an image whose time difference between the corresponding shooting time and the shooting time of the target image satisfies a preset time condition; the preset time condition can indicate an upper threshold of the time difference, that is, the time difference between the shooting time of the reference image and the shooting time of the target image cannot exceed the upper threshold of the time difference; the preset time can also indicate an upper threshold of the time difference and a lower threshold of the time difference at the same time, that is, the time difference between the shooting time of the reference image and the shooting time of the target image needs to be between the lower threshold of the time difference and the upper threshold of the time difference.

[0090] For example, Figure 3 A schematic diagram of a target image and its corresponding reference image sequence provided in an embodiment of the present application, such as Figure 3 As shown, I t is the target image. When the preset time condition only indicates the upper limit threshold of the time difference, the image I in the image sequence collected by the camera t-(k+1) 、Image I t 、Image I t-(k-1) ,……,Image I t-3 、Image I t-2 、Image I t-1 Together they form the target image I t The corresponding reference image sequence (n is equal to k+1 in this case), the time difference between the shooting time of each reference image in the reference image sequence and the shooting time of the target image does not exceed the time difference upper limit threshold indicated by the preset time condition. t-(k+2) , the time difference between its corresponding shooting time and the shooting time of the target image exceeds the time difference upper limit threshold indicated by the preset time condition, and therefore, the reference image sequence corresponding to the target image cannot be added.

[0091] For example, Figure 4 A schematic diagram of another target image and its corresponding reference image sequence provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, I tis the target image, and when the preset time condition indicates both the time difference upper limit threshold and the time difference lower limit threshold, the image I in the image sequence collected by the camera t-(k+1) 、Image I t 、Image I t-(k-1) ,……,Image I t-3 、Image I t-2 Together they form the target image I t The corresponding reference image sequence (n is equal to k in this case), the time difference between the shooting time of each reference image in the reference image sequence and the shooting time of the target image does not exceed the upper limit threshold of the time difference indicated by the preset time condition, and is not less than the lower limit threshold of the time difference indicated by the preset time condition. t-(k+2) , the time difference between its corresponding shooting time and the shooting time of the target image exceeds the time difference upper limit threshold indicated by the preset time condition, therefore, the reference image sequence corresponding to the target image cannot be added; for image I in the image sequence t-1 , the time difference between its corresponding shooting time and the shooting time of the target image is less than the time difference lower limit threshold indicated by the preset time condition, therefore, the reference image sequence corresponding to the target image cannot be added.

[0092] It should be understood that the above-mentioned time difference upper limit threshold and time difference lower limit threshold can be set according to actual needs. For example, the time difference upper limit threshold can be a time difference greater than 3s, and the time difference lower limit threshold can be a time difference less than 0.5s. The embodiment of the present application does not impose any limitation on this. In addition, the number of frames n of the reference image included in the reference image sequence can be set according to the time difference upper limit threshold and the camera frame rate (such as 5 frames per second). For example, n can be greater than or equal to 15, and the embodiment of the present application does not impose any limitation on this.

[0093] It should be understood that Figure 3 and Figure 4 Taking the reference images in the reference image sequence as frames of images continuously captured by a camera as an example, in practical applications, the reference images included in the reference image sequence may not be images continuously captured by the camera. For example, a specific number of image frames may be spaced between every two adjacent reference images in the reference image sequence. For example, the reference image sequence includes reference image I t-(k+1) , reference image I t-(k-1) , ..., reference image I t-3 , reference image I t-1 , there is one frame of image between every two adjacent reference images; the embodiment of the present application does not impose any limitation on the relationship between reference images in the reference image sequence.

[0094] Step 202: Determine first reference data according to the image difference between every two adjacent reference images in the reference image sequence.

[0095] Since the reference image sequence includes n frames of reference images arranged in order from front to back according to the shooting time, the terminal device can determine the first reference data based on the image difference between every two adjacent reference images. The first reference data can be used as a reference data for target trigger event detection, and the first reference data is used to characterize the basic dynamic situation within the shooting range of the camera before the camera shoots the target image; the basic dynamic situation here can be understood as the dynamic change of the background environment within the shooting range of the camera. Generally, the basic dynamic situation can be caused by the lighting in the shooting environment, interference factors (such as wind, movement of people in the distance, etc.), etc.

[0096] In a possible implementation, the terminal device can Figure 5 The process shown in FIG. 1 determines the first reference data. Figure 5 As shown, the process of determining the first reference data includes the following steps:

[0097] Step 2021: construct n-1 reference image pairs based on the n reference image frames in the reference image sequence; each reference image pair includes two reference image frames that are captured adjacently in time.

[0098] In specific implementation, the terminal device may use every two adjacent reference image frames in the reference image sequence to form a reference image pair, thereby using n reference image frames to form n-1 reference image pairs.

[0099] by Figure 3 As an example of the reference image sequence shown in FIG. t-(k+1) and reference image I t Form a reference image pair, using the reference image I t- and reference image I t-(k-1) Form a reference image pair, ..., using the reference image I t-3 and reference image I t-2 Form a reference image pair, using the reference image I t-2 and reference image I t-1 form a reference image pair; thus, k reference image pairs will be obtained. Figure 4 As an example of the reference image sequence shown in FIG. t-(k+1) and reference image I t Form a reference image pair, using the reference image I t- and reference image I t-(k-1) Form a reference image pair, ..., using the reference image It-3 and reference image I t-2 Form a reference image pair; thus, k-1 reference image pairs will be obtained.

[0100] Step 2022: For each of the reference image pairs, determine image difference data corresponding to the reference image pair according to an image difference between two frames of the reference images included in the reference image pair.

[0101] For each reference image pair constructed through step 2021, the terminal device can use an inter-frame difference algorithm to calculate image difference data used to characterize the image difference between two reference image frames included in the reference image pair as the image difference data corresponding to the reference image pair.

[0102] As an example, the terminal device can calculate the image difference data based on the grayscale images corresponding to the two frames of reference images, or it can determine the image difference data through a pre-trained neural network model. The embodiments of the present application do not impose any limitation on the method of determining the image difference data.

[0103] Step 2023: Calculate the mean of the image difference data corresponding to each of the n-1 reference image pairs as the first reference data.

[0104] Through step 2022, the terminal device will calculate the image difference data corresponding to each of the n-1 reference image pairs. Further, the terminal device can calculate the mean of the image difference data corresponding to each of the n-1 reference image pairs as the first reference data for characterizing the basic dynamic conditions of the shooting environment.

[0105] by Figure 3 Taking the reference image sequence shown in FIG. 1 as an example, the terminal device can calculate the first reference data by the following formula (1):

[0106]

[0107] in, is the first reference data, Δ1 is the reference image I t-2 and reference image I t-1 The image difference data corresponding to the reference image pair composed of t-3 and reference image I t-2 The image difference data corresponding to the reference image pair composed of t-(k+1) and reference image I t-k The image difference data corresponding to the reference image pair composed.

[0108] by Figure 4Taking the reference image sequence shown in FIG. 1 as an example, the terminal device can calculate the first reference data by the following formula (2):

[0109]

[0110] in, is the first reference data, Δ1′ is the reference image I t-3 and reference image I t-2 The image difference data corresponding to the reference image pair composed of t-4 and reference image I t-3 The image difference data corresponding to the reference image pair composed of t-(k+1) and reference image I t-k The image difference data corresponding to the reference image pair composed.

[0111] In this way, through the above method, the mean of the image difference data corresponding to each reference image pair is used as the first reference data. The first reference data that can more reliably reflect the basic dynamic conditions of the shooting environment can be calculated with low computational complexity and less computing resources consumed, that is, it takes into account both saving computing resources and ensuring the reliability of the first reference data.

[0112] It should be understood that Figure 5 The process of determining the first reference data shown is only an example. In actual applications, in addition to determining the first reference data based on the mean of the image difference data corresponding to each reference image pair, the terminal device can also use other methods to determine the first reference data. For example, the terminal device can configure weight parameters for each reference image pair according to the time difference between the shooting time of the reference image in each reference image pair and the shooting time of the target image. For example, the closer the shooting time of the reference image in the reference image pair is to the shooting time of the target image, the larger the weight parameter configured for the reference image pair; then, based on the weight parameters corresponding to each reference image pair, the image difference data corresponding to each reference image pair is weighted averaged to obtain the first reference data. The embodiment of the present application does not impose any limitation on the method of determining the first reference data.

[0113] Step 203: Determine second reference data according to an image difference between the target image and an adjacent image corresponding to the target image; the adjacent image is an image adjacent to the target image captured by the camera before capturing the target image.

[0114] In addition, the terminal device can also obtain an adjacent image corresponding to the target image, where the adjacent image corresponding to the target image is an image captured by the camera before the target image is captured and is adjacent to the target image in the shooting order. Figure 3 and Figure 4 As an example, the image sequence shown in FIG. t-1 The target image I t The corresponding adjacent images.

[0115] Furthermore, the terminal device can use an inter-frame difference algorithm to calculate the second reference data that can characterize the image difference based on the image difference between the target image and the adjacent image. The second reference data can be used as another reference data for target trigger event detection, and the second reference data is used to characterize the dynamic situation within the shooting range of the camera when the camera shoots the target image; the dynamic situation here can be caused by changes in the background environment of the shooting range (such as changes in lighting in the shooting environment, interference factors, etc.), or caused by changes in the foreground environment of the shooting range (such as caused by the movement of nearby people in the shooting environment).

[0116] As an example, the terminal device can calculate the second reference data based on the grayscale images corresponding to the target image and the adjacent image, or it can determine the second reference data through a pre-trained neural network model. The embodiment of the present application does not impose any limitation on the method of determining the second reference data.

[0117] It should be noted that, in the embodiment of the present application, in order to be able to comprehensively refer to the first reference data and the second reference data for detecting target trigger events, it is usually necessary to ensure that the algorithms used to measure the inter-frame differences when determining the first reference data and the second reference data are the same, that is, to ensure that the algorithm used to determine the image difference data corresponding to the reference image pair (the difference between every two adjacent reference images in the reference image sequence) is the same as the algorithm used to determine the second reference data, thereby making the first reference data and the second reference data inter-frame difference data of the same dimension, which facilitates combining the two to detect target trigger events.

[0118] It should be understood that in actual applications, the terminal device may execute step 202 first and then step 203, or execute step 203 first and then step 202, or execute step 202 and step 203 at the same time. The embodiment of the present application does not impose any limitation on the execution order of step 202 and step 203.

[0119] Step 204: Based on the first reference data and the second reference data, detecting whether a target triggering event occurs within the shooting range of the camera when the camera shoots the target image.

[0120] After the terminal device determines the first reference data and the second reference data, it can comprehensively consider the first reference data and the second reference data to detect whether a target trigger event occurs within the shooting range of the camera when the camera captures the target image. Since the first reference data used to reflect the basic dynamic situation within the camera shooting range is also taken into consideration, false detection caused by basic dynamic situations (such as environmental interference, light changes, etc.) can be effectively avoided.

[0121] It should be noted that the target trigger event in the embodiment of the present application refers to an environmental event that triggers the terminal device to perform a specific operation. For example, the target trigger event can be an environmental event that triggers the terminal device to perform an event alarm operation, and the event alarm operation is used to warn the camera of the presence of a target object (such as a person or an animal) within the shooting range; for another example, the target trigger event can be an environmental event that triggers the terminal device to perform an operation state switching operation, and the operation state switching operation is used to switch the working state of the terminal device from one state to another. The embodiment of the present application does not make any limitation on the target trigger event.

[0122] In a possible implementation, the terminal device may detect the target triggering event in the following manner:

[0123] When the first reference data is less than the first preset threshold, if the second reference data is greater than or equal to the second preset threshold, it is determined that a target trigger event occurs within the shooting range of the camera when the camera captures the target image; if the second reference data is less than the second preset threshold, it is determined that no target trigger event occurs within the shooting range of the camera when the camera captures the target image.

[0124] Specifically, the first reference data If the second reference data Δtarget is less than the first preset threshold value r1, it means that there is no large dynamic change in the shooting range of the camera before the camera captures the target image, that is, the background environment changes slightly in the shooting range of the camera. At this time, if it is detected that the second reference data Δtarget is greater than or equal to the second preset threshold value r2, it means that a large dynamic change has occurred in the shooting range of the camera when the camera captures the target image. When the background environment originally changes slightly, a large-scale dynamic change suddenly occurs, which can be explained that a target trigger event has occurred in the shooting range of the camera when the camera captures the target image. On the contrary, if it is detected that the second reference data Δtarget is less than the second preset threshold value r2, it means that there is still no large dynamic change in the shooting range of the camera when the camera captures the target image. Therefore, it can be determined that no target trigger event has occurred in the shooting range of the camera when the camera captures the target image.

[0125] In this way, by detecting target trigger events in the above manner, when the first reference data indicates that there are no large dynamic changes in the shooting range of the camera, it is possible to accurately measure whether large dynamic changes occur within the shooting range of the camera when the camera captures the target image based on the second reference data, thereby accurately detecting whether a target trigger event occurs within the shooting range of the camera when the camera captures the target image.

[0126] In a possible implementation, the terminal device may detect the target triggering event in the following manner:

[0127] When the first reference data is greater than or equal to the first preset threshold, a reference data difference between the second reference data and the first reference data is calculated. If the reference data difference is greater than or equal to a third preset threshold, it is determined that a target trigger event occurs within the shooting range of the camera when the camera captures the target image; if the reference data difference is less than the third preset threshold, it is determined that no target trigger event occurs within the shooting range of the camera when the camera captures the target image.

[0128] Specifically, the first reference data is greater than or equal to the first preset threshold r1, indicating that there was a large dynamic change in the shooting range of the camera before the camera captured the target image, that is, the background environment in the shooting range of the camera changed greatly. At this time, the terminal device can calculate the second reference data Δtarget and the first reference data The difference between As a reference data difference; the reference data difference can represent the difference between the dynamic situation within the shooting range when the camera shoots the target image and the original basic dynamic situation within the shooting range.

[0129] If the reference data difference is greater than or equal to the third preset threshold value r3, it means that the difference between the dynamic situation in the shooting range when the camera shoots the target image and the original basic dynamic situation in the shooting range is large, that is, when the camera shoots the target image, the shooting environment has undergone a large dynamic change compared to before, which can further illustrate that a target trigger event has occurred in the shooting range of the camera when the camera shoots the target image. If the reference data difference is less than the third preset threshold value r3, it means that the difference between the dynamic situation in the shooting range when the camera shoots the target image and the original basic dynamic situation in the shooting range is small, that is, when the camera shoots the target image, the shooting environment has not undergone a large dynamic change compared to before, which can further illustrate that a target trigger event has not occurred in the shooting range of the camera when the camera shoots the target image.

[0130] In this way, by detecting target trigger events in the above manner, when the first reference data represents that there are originally large dynamic changes within the shooting range of the camera, based on the reference data difference between the second reference data and the first reference data, it can be accurately measured whether the dynamic situation in the shooting environment when the camera captures the target image has undergone a large change compared to the previous dynamic situation, thereby accurately detecting whether a target trigger event has occurred within the shooting range of the camera when the camera captures the target image.

[0131] It should be understood that the above-mentioned first preset threshold r1, second preset threshold r2 and third preset threshold r3 can be set according to actual needs. The first preset threshold r1, second preset threshold r2 and third preset threshold r3 can be equal or different, and the embodiment of the present application does not impose any limitation on this.

[0132] The environmental event detection method provided in the embodiment of the present application takes into account the basic dynamic conditions that originally exist within the shooting range of the camera, and detects target trigger events in combination with first reference data that can reflect the basic dynamic conditions and second reference data that can reflect the dynamic conditions within the shooting range when shooting the target image. This can effectively avoid false detections caused by basic dynamic conditions (such as environmental interference, lighting changes, etc.), and improve the accuracy and robustness of trigger event detection.

[0133] In one possible implementation, the terminal device can determine the image difference between two adjacent reference images, that is, determine the image difference data corresponding to the reference image pair mentioned above, by performing grayscale processing on the two reference images respectively to obtain grayscale images corresponding to the two reference images, and determine the image difference data between the two reference images based on the grayscale images corresponding to the two reference images.

[0134] Similarly, the terminal device can determine the second reference data in the following manner: perform grayscale processing on the target image and the adjacent image respectively to obtain the grayscale images corresponding to the target image and the adjacent image respectively; determine the second reference data based on the image difference between the grayscale images corresponding to the target image and the adjacent image respectively.

[0135] Specifically, for two adjacent reference images in each reference image pair, the terminal device can perform grayscale processing on the two reference images respectively; that is, based on the red, green, and blue (RGB) three-channel values ​​corresponding to each pixel in a reference image, the grayscale value corresponding to the pixel is converted, and then based on the grayscale values ​​corresponding to each pixel in the reference image, the grayscale image corresponding to the reference image is obtained. Furthermore, the terminal device can determine the image difference data between the two reference images based on the difference between the grayscale images corresponding to the two reference images, that is, the image difference data corresponding to the reference image pair.

[0136] Similarly, when the terminal device determines the second reference data, it can perform grayscale processing on the target image and its corresponding adjacent image respectively to obtain a grayscale image corresponding to the target image and a grayscale image corresponding to the adjacent image; it should be understood that the implementation method of grayscale processing on the target image and the adjacent image is similar to the implementation method of grayscale processing on the reference image, which will not be described in detail here. Furthermore, the terminal device can determine the second reference data based on the difference between the grayscale images corresponding to the target image and the adjacent image.

[0137] In this way, determining image difference data based on grayscale images can ensure that the determined image difference data more accurately reflects the difference between two adjacent reference image frames, as well as the difference between the target image and the adjacent images, thereby reducing the consideration of interference factors in the shooting environment to a certain extent, and is conducive to improving the detection accuracy of subsequent trigger events.

[0138] As an example, the terminal device can specifically determine the image difference data between the two frames of reference images based on the grayscale images corresponding to the two frames of reference images in the following manner: determine the overall grayscale value corresponding to each frame of reference image according to the grayscale value of each pixel in the grayscale image corresponding to each frame of reference image; calculate the difference between the overall grayscale values ​​corresponding to the two frames of reference images as the image difference data between the two frames of reference images.

[0139] Similarly, the terminal device can determine the second reference data based on the grayscale images corresponding to the target image and the adjacent image in the following manner: determine the overall grayscale value corresponding to the target image according to the grayscale value of each pixel in the grayscale image corresponding to the target image; determine the overall grayscale value corresponding to the adjacent image according to the grayscale value of each pixel in the grayscale image corresponding to the adjacent image; calculate the difference between the overall grayscale value corresponding to the target image and the overall grayscale value corresponding to the adjacent image as the second reference data.

[0140] Specifically, the terminal device can determine the overall grayscale value of the grayscale image according to the grayscale values ​​of each pixel in the grayscale image for the grayscale image corresponding to each frame of the reference image; for example, the terminal device can calculate the mean of the grayscale values ​​of each pixel in the grayscale image as the overall grayscale value of the grayscale image; for another example, the terminal device can configure corresponding weight parameters for each pixel according to the position of each pixel in the grayscale image, and then perform weighted average processing on the grayscale values ​​of each pixel in the grayscale image to obtain the overall grayscale value of the grayscale image; of course, in practical applications, other methods can also be used to determine the overall grayscale value of the grayscale image according to the grayscale values ​​of each pixel in the grayscale image, and the embodiments of the present application do not impose any restrictions on this. Then, the terminal device can use the overall grayscale value of the grayscale image as the overall grayscale value corresponding to the reference image corresponding to the grayscale image. Then, the difference between the overall grayscale values ​​corresponding to the two adjacent reference images is calculated as the image difference data between the two reference images.

[0141] Similarly, the terminal device can use an algorithm similar to the one for calculating the overall grayscale value corresponding to the reference image for the target image and the adjacent image, and calculate the overall grayscale value corresponding to the target image according to the grayscale value of each pixel in the grayscale image corresponding to the target image, and calculate the overall grayscale value corresponding to the adjacent image according to the grayscale value of each pixel in the grayscale image corresponding to the adjacent image. Then, the difference between the overall grayscale value corresponding to the target image and the overall grayscale value corresponding to the adjacent image is calculated as the second reference data.

[0142] In this way, through the above method, the overall grayscale value of the grayscale image corresponding to the image is determined, and based on this, the image difference between two adjacent reference frames and the image difference between the target image and the adjacent image (i.e., the second reference data) are determined. This allows image difference data that can more accurately reflect the image difference to be determined while consuming fewer computing resources, which is beneficial to improving the detection efficiency and accuracy of trigger events.

[0143] As another example, the terminal device can determine the image difference data between the two frames of reference images based on the grayscale images corresponding to each of the two frames of reference images in the following manner: construct first pixel pairs based on pixel points located at the same position in the grayscale images corresponding to each of the two frames of reference images; determine the image difference data between the two frames of reference images based on the grayscale difference values ​​corresponding to each of the first pixel pairs.

[0144] Similarly, the terminal device can determine the second reference data based on the grayscale images corresponding to the target image and the adjacent image in the following manner: construct a second pixel pair based on the pixels located at the same position in the grayscale image corresponding to the target image and the grayscale image corresponding to the adjacent image; determine the second reference data according to the grayscale difference corresponding to each second pixel pair. The grayscale difference corresponding to the above pixel pair is the difference between the grayscale values ​​corresponding to the two pixels.

[0145] Specifically, the terminal device can use the pixels in the grayscale images corresponding to the two reference images to construct multiple first pixel pairs, each of which includes two pixels in the grayscale images corresponding to the two reference images, and the positions of the two pixels in the grayscale images to which they belong are the same. Then, the terminal device can calculate the difference in the grayscale values ​​of the two pixels in each first pixel pair as the grayscale difference corresponding to the first pixel pair. Furthermore, the terminal device can calculate the average of the grayscale difference corresponding to each first pixel pair as the image difference data between the two reference images; or, the terminal device can also configure a weight parameter for the first pixel pair according to the position of the pixel in each first pixel pair in the grayscale image, and then perform weighted averaging on the grayscale difference corresponding to each first pixel pair to obtain the image difference data between the two reference images; the embodiment of the present application does not make any limitation on the method of determining the image difference data based on the grayscale difference corresponding to the pixel pair.

[0146] Similarly, the terminal device can use the pixel points in the grayscale images corresponding to the target image and the adjacent image to construct multiple second pixel pairs, each of which includes two pixel points in the grayscale images corresponding to the target image and the adjacent image, and the two pixel points are in the same position in the grayscale images to which they belong. Then, the terminal device can calculate the difference in grayscale values ​​of the two pixels in each second pixel pair as the grayscale difference corresponding to the second pixel pair. Furthermore, the terminal device can use the above-mentioned algorithm for determining image difference data based on the grayscale difference corresponding to each first pixel pair, and determine the image difference data between the target image and the adjacent image based on the grayscale difference corresponding to each second pixel pair, that is, determine the second reference data.

[0147] In this way, through the above method, based on the grayscale difference between the pixels that belong to the two frames of grayscale images and are located at the same position, the image difference between the two adjacent reference images and the image difference between the target image and the adjacent image (i.e., the second reference data) are determined. This can ensure that the determined image difference data more accurately reflects the difference between the two frames of images. In addition, it can also save computing resources to a certain extent and improve computing efficiency.

[0148] In one possible implementation, the terminal device can determine the image difference between two adjacent reference images, that is, determine the image difference data corresponding to the reference image pair mentioned above, in the following manner: through an image feature extraction model, perform feature extraction processing on the two reference images respectively to obtain image feature data corresponding to each of the two reference images; based on the image feature data corresponding to each of the two reference images, determine the image difference data between the two reference images.

[0149] Similarly, the terminal device can determine the second reference data in the following manner: through the above-mentioned image feature extraction model, perform feature extraction processing on the target image and the adjacent image respectively, and obtain the image feature data corresponding to the target image and the adjacent image respectively; based on the image feature data corresponding to the target image and the adjacent image respectively, determine the second reference data.

[0150] The embodiment of the present application proposes an image feature extraction model for extracting image features from an input image. The image feature extraction model can specifically be a convolutional neural network (CNN) model, such as a computer vision group (Visual Geometry Group, VGG) network model, etc. The embodiment of the present application does not impose any limitation on the structure of the image feature extraction model.

[0151] In specific implementation, the terminal device can input a frame of reference image into the image feature extraction model, and the image feature extraction model can output image feature data corresponding to the reference image by performing image feature extraction processing on the reference image, and the image feature data can reflect the image characteristics of the reference image in a deeper level. After the terminal device obtains the image feature data corresponding to each of the two adjacent frames of reference images, it can determine the image difference data between the two frames of reference images based on the image feature data corresponding to each of the two frames of reference images; for example, the terminal device can calculate the feature data difference of the image feature data corresponding to each of the two frames of reference images, and then use a fully connected network to process the feature data difference to obtain the image difference data between the two frames of reference images.

[0152] Similarly, the terminal device can use the above-mentioned image feature extraction model to perform image feature extraction processing on the target image and the adjacent image respectively, and obtain the image feature data corresponding to the target image and the adjacent image respectively. Then, the same algorithm as that for calculating the image difference data between two frames of reference images is used to calculate the image difference data between the target image and the adjacent image, that is, the second reference data, based on the image feature data corresponding to the target image and the adjacent image respectively.

[0153] In this way, through the above-mentioned method, the image feature extraction model is used to determine the image feature data, and then the image difference data is determined based on the difference between the image feature data. This can ensure that the determined image difference data more reliably reflects the difference between the deep-level image features of the two reference images, as well as the difference between the deep-level image features of the target image and the adjacent images. Accordingly, the detection of trigger events based on the image difference data determined in this way can improve the accuracy of the detection results.

[0154] In one possible implementation, the terminal device can determine the image difference between two adjacent reference images, that is, determine the image difference data corresponding to the reference image pair mentioned above, in the following manner: determine the image difference data between the two reference images based on the two reference images through an image difference recognition model; the image difference recognition model is used to determine, for two input images, image difference data used to characterize the difference between the two images.

[0155] Similarly, the terminal device may determine the second reference data in the following manner: by using the above-mentioned image difference recognition model, the second reference data is determined according to the target image and the adjacent images.

[0156] The embodiment of the present application innovatively provides an image difference recognition model, which is a neural network model that can recognize the difference between two input images and output image difference data used to characterize the difference. Exemplarily, the image difference recognition model may include two parallel CNN networks, a fully connected network, and a Softmax layer; wherein the two CNN networks are used to perform feature extraction processing on the two input images respectively, obtain the image feature vectors corresponding to the two images, and then calculate the difference between the image feature vectors corresponding to the two images, use the fully connected network to process the feature vector difference, and finally determine the image difference data through the Softmax layer according to the output result of the fully connected network; it should be understood that the model structure of the above-mentioned image difference recognition model is only an example, and the embodiment of the present application does not impose any limitation on the model structure of the image difference recognition model.

[0157] In specific implementation, the terminal device may input two adjacent reference image frames into the above-mentioned image difference recognition model, and the image difference recognition model may process the two input reference image frames and accordingly output image difference data between the two reference image frames. Similarly, the terminal device may input the target image and the adjacent image into the above-mentioned image difference recognition model, and the image difference recognition model may process the input target image and the adjacent image and accordingly output image difference data between the target image and the adjacent image, i.e., the second reference data.

[0158] In this way, by directly using the image difference recognition model to determine the image difference data in the above manner, on the one hand, it can ensure that the determined image difference data accurately reflects the difference between the two reference images and the difference between the target image and the adjacent images; on the other hand, it can also simplify the determination process of the image difference data and improve the determination efficiency.

[0159] It should be understood that the method of determining image difference data introduced above is only an example. In actual applications, the terminal device may also adopt other methods to determine the image difference data between two adjacent frames of reference images, and the image difference data between the target image and the adjacent image (i.e., the second reference data). The embodiments of the present application do not impose any limitations on this.

[0160] In a possible application scenario, the camera for capturing the target image may be installed on a target device. If a target trigger event is detected within the capturing range of the camera while the target device is in a dormant state, the target device is controlled to enter a standby state; the power consumption of the target device in the standby state is higher than that of the target device in the dormant state.

[0161] In actual applications, many devices can include two working states, namely, sleep state and standby state. In the sleep state, the device can run with low power consumption. For example, the camera of the device can capture images at a low frame rate (such as only 5 frames per second), and other components of the device (such as display screen, indicator light, sensor, etc.) are all in a sleep state. When the device detects a target trigger event within the shooting range of the camera based on the image captured by the camera, for example, when a user approaches within the shooting range of the camera, the device can switch from the sleep state to the standby state. In the standby state, the camera of the device can capture images at a high frame rate, the brightness of the display screen of the device can be increased, the indicator light of the device lights up, and the sensor enters the working state.

[0162] In an embodiment of the present application, the environmental event detection method described above can be used to detect whether a target trigger event occurs within the shooting range of the camera based on the target image captured in real time by the camera of the target device and the reference image sequence corresponding to the target image, for example, to detect whether a specific object (such as a person, animal, etc.) approaches within the shooting range of the camera. If the target device is in a dormant state, the environmental event detection method provided by the embodiment of the present application detects that a target trigger event occurs within the shooting range of the camera, and the target device is immediately controlled to switch to a standby state, so as to more quickly respond to related events after the target trigger event, such as face payment, more accurate environmental monitoring, etc.

[0163] In this way, based on the environmental event detection method provided in the embodiment of the present application, it is detected whether a target trigger event occurs within the shooting range of the camera of the target device, and it is determined whether to switch the working state of the target device based on the detection result. Compared with making the target device run at high power consumption all the time, the power consumption of the target device can be reduced to a certain extent, saving the energy consumed by the target device. In addition, once a target trigger event is detected within the shooting range of the camera of the target device, the target device is controlled to enter the standby state, which can also ensure the timeliness of the response of the target device to related events.

[0164] In addition, from the moment the target device enters the standby state, the target device can record the standby time; if the standby time reaches a preset time and no target trigger event is detected within the shooting range of the camera during the standby time, the target device is controlled to enter the sleep state.

[0165] Considering that the target device will generate more energy consumption when it is in standby state all the time, in order to avoid energy waste, in the embodiment of the present application, the standby time can be recorded from the moment when the target device enters the standby state. At the same time, the environmental event detection method provided by the embodiment of the present application is used in the process of the target device being in the standby state to continuously detect whether a target trigger event occurs within the shooting range of the camera of the target device. If the target trigger event is not detected during the period when the standby time reaches the preset time, the target device can be controlled to switch back to the dormant state; on the contrary, if the target trigger event is detected within the shooting range of the camera of the target device before the standby time reaches the preset time, the standby time can be cleared, and the standby time can be re-recorded when it is determined that the target trigger event ends, and the detection process of the target trigger event and the relationship between the detection standby time and the preset time are continuously performed.

[0166] Thus, by controlling the target device to switch from standby state to sleep state based on the delay mechanism, unnecessary energy consumption of the target device can be reduced, while the reliability of the target device's operation can be ensured.

[0167] For example, Figure 6 Schematic diagram of an application scenario of the working state switching mechanism of the target device provided in the embodiment of the present application. Figure 6As shown, taking the target device as a device that performs related tasks (such as door unlocking, payment, etc.) based on the user's palm print as an example, the target device includes a camera, a light ring, a display screen and multiple sensors. The target device has two working states, namely, a dormant state and a standby state; when the target device is in the dormant state, the camera on the target device maintains a low frame rate operation, multiple sensor parts (such as the proximity sensor) are in a low power consumption operation state, the light ring is in a dark state, and the display screen is in a low brightness state; when the target device is in the standby state, the camera on the target device operates at a high frame rate, multiple sensors are in a high power consumption operation state, such as the proximity sensor is in a high frame rate operation state, the light ring is in a bright state, and the display screen is in a high brightness state.

[0168] When the target device is in a sleep state, if a target triggering event is detected within the shooting range of the camera through the environmental event detection method provided in the embodiment of the present application, the target device will switch from the sleep state to the standby state, that is, realizing state route 1; if a target triggering event is not detected within the shooting range of the camera through the environmental event detection method provided in the embodiment of the present application, the target device maintains the sleep state, that is, realizing state route 3.

[0169] When the target device is in standby mode, if the environmental event detection method provided in the embodiment of the present application does not detect a target trigger event (such as a palm approaching) within the shooting range of the camera during the standby time reaches a preset time, the target device will switch from standby mode to sleep mode, thereby realizing state route 2; if the environmental event detection method provided in the embodiment of the present application detects a target trigger event within the shooting range of the camera during the standby time reaches a preset time, the target device maintains standby mode, thereby realizing state route 4.

[0170] With respect to the environmental event detection method described above, the present application also provides a corresponding environmental event detection device so that the above environmental event detection method can be applied and implemented in practice.

[0171] See also Figure 7 , Figure 7 It is the same as above Figure 2 The schematic diagram of the structure of the environmental event detection device 700 corresponding to the environmental event detection method shown in FIG. Figure 7 As shown, the environmental event detection device 700 includes:

[0172] The sequence acquisition module 701 is used to acquire a reference image sequence corresponding to a target image captured by a camera; the reference image sequence includes n frames of reference images captured by the camera before capturing the target image, where n is an integer greater than 1, and the time difference between the capturing time of the reference image and the capturing time of the target image satisfies a preset time condition;

[0173] A first determining module 702, configured to determine first reference data according to an image difference between every two adjacent reference images in the reference image sequence;

[0174] A second determination module 703 is used to determine second reference data according to an image difference between the target image and an adjacent image corresponding to the target image; the adjacent image is an image adjacent to the target image captured by the camera before capturing the target image;

[0175] The event detection module 704 is used to detect whether a target triggering event occurs within the shooting range of the camera when the camera shoots the target image based on the first reference data and the second reference data.

[0176] Optionally, the first determining module 702 is specifically configured to:

[0177] Based on the n reference image frames in the reference image sequence, construct n-1 reference image pairs; each of the reference image pairs includes two reference image frames with adjacent shooting times;

[0178] For each of the reference image pairs, determining image difference data corresponding to the reference image pair according to an image difference between two frames of the reference images included in the reference image pair;

[0179] The average values ​​of the image difference data corresponding to each of the n-1 reference image pairs are calculated as the first reference data.

[0180] Optionally, the first determining module 702 is specifically configured to:

[0181] Performing grayscale processing on the two frames of the reference image respectively to obtain grayscale images corresponding to the two frames of the reference image respectively; determining image difference data between the two frames of the reference image based on the grayscale images corresponding to the two frames of the reference image respectively;

[0182] The second determining module 703 is specifically used for:

[0183] Grayscale processing is performed on the target image and the adjacent image respectively to obtain grayscale images corresponding to the target image and the adjacent image respectively; and the second reference data is determined based on the image difference between the grayscale images corresponding to the target image and the adjacent image respectively.

[0184] Optionally, the first determining module 702 is specifically configured to:

[0185] Determine the overall grayscale value corresponding to each frame of the reference image according to the grayscale value of each pixel in the grayscale image corresponding to each frame of the reference image; calculate the difference between the overall grayscale values ​​corresponding to the two frames of the reference image as the image difference data;

[0186] The second determining module 703 is specifically used for:

[0187] Determine the overall grayscale value corresponding to the target image according to the grayscale value of each pixel in the grayscale image corresponding to the target image; determine the overall grayscale value corresponding to the adjacent image according to the grayscale value of each pixel in the grayscale image corresponding to the adjacent image;

[0188] The difference between the overall grayscale value corresponding to the target image and the overall grayscale value corresponding to the adjacent image is calculated as the second reference data.

[0189] Optionally, the first determining module 702 is specifically configured to:

[0190] Constructing first pixel pairs based on pixel points at the same position in the grayscale images corresponding to the two frames of the reference images; determining the image difference data according to the grayscale difference values ​​corresponding to each of the first pixel pairs;

[0191] The second determining module 703 is specifically used for:

[0192] Constructing second pixel pairs based on the pixel points located at the same position in the grayscale image corresponding to the target image and the grayscale image corresponding to the adjacent image; determining the second reference data according to the grayscale difference values ​​corresponding to each of the second pixel pairs;

[0193] The grayscale difference corresponding to the pixel pair is the difference between the grayscale values ​​corresponding to the two pixel points.

[0194] Optionally, the first determining module 702 is specifically configured to:

[0195] Through the image feature extraction model, feature extraction processing is performed on the two frames of the reference image respectively to obtain image feature data corresponding to the two frames of the reference image respectively; based on the image feature data corresponding to the two frames of the reference image respectively, image difference data between the two frames of the reference image is determined;

[0196] The second determining module 703 is specifically used for:

[0197] The target image and the adjacent image are subjected to feature extraction processing respectively through the image feature extraction model to obtain image feature data corresponding to the target image and the adjacent image respectively; and the second reference data is determined based on the image feature data corresponding to the target image and the adjacent image respectively.

[0198] Optionally, the first determining module 702 is specifically configured to:

[0199] Determine the image difference data between the two frames of the reference image according to the two frames of the reference image by means of an image difference recognition model; the image difference recognition model is used to determine the image difference data used to characterize the difference between the two images inputted;

[0200] The second determining module 703 is specifically used for:

[0201] The second reference data is determined according to the target image and the adjacent image by using the image difference recognition model.

[0202] Optionally, the event detection module 704 is specifically used for:

[0203] When the first reference data is less than the first preset threshold, if the second reference data is greater than or equal to the second preset threshold, it is determined that a target trigger event occurs within the shooting range of the camera when the camera captures the target image; if the second reference data is less than the second preset threshold, it is determined that no target trigger event occurs within the shooting range of the camera when the camera captures the target image.

[0204] Optionally, the event detection module 704 is specifically used for:

[0205] When the first reference data is greater than or equal to a first preset threshold, calculating a reference data difference between the second reference data and the first reference data;

[0206] If the reference data difference is greater than or equal to a third preset threshold, it is determined that a target trigger event occurs within the shooting range of the camera when the camera captures the target image; if the reference data difference is less than the third preset threshold, it is determined that no target trigger event occurs within the shooting range of the camera when the camera captures the target image.

[0207] Optionally, the camera is installed on the target device; the apparatus further includes:

[0208] A state switching module is used to control the target device to enter a standby state if the target trigger event is detected within the shooting range of the camera when the target device is in a sleep state; the power consumption of the target device in the standby state is higher than the power consumption of the target device in the sleep state.

[0209] Optionally, the state switching module is further used for:

[0210] Recording the standby time from the moment when the target device enters the standby state;

[0211] If the standby time reaches a preset time and the target triggering event is not detected within the shooting range of the camera within the standby time, the target device is controlled to enter the sleep state.

[0212] The environmental event detection device provided in the embodiment of the present application takes into account the basic dynamic conditions originally existing in the shooting range of the camera, and detects target trigger events in combination with first reference data that can reflect the basic dynamic conditions and second reference data that can reflect the dynamic conditions within the shooting range when shooting the target image. This can effectively avoid false detections caused by basic dynamic conditions (such as environmental interference, lighting changes, etc.), thereby improving the accuracy and robustness of trigger event detection.

[0213] The embodiment of the present application also provides a computer device for detecting environmental events, which may specifically be a terminal device or a server. The terminal device and server provided in the embodiment of the present application will be introduced below from the perspective of hardware instantiation.

[0214] See also Figure 8 , Figure 8 Schematic diagram of the structure of the terminal device provided in the embodiment of the present application. Figure 8For the sake of convenience, only the parts related to the embodiments of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiments of the present application. The terminal can be any terminal device including a mobile phone, a tablet computer, a personal digital assistant (PDA), a point of sales (POS), a car computer, etc., taking a smart phone as an example:

[0215] Figure 8 FIG. 1 is a block diagram showing a partial structure of a smart phone related to a terminal provided in an embodiment of the present application. Figure 8 The smartphone includes: a radio frequency (RF) circuit 810, a memory 820, an input unit 830 (including a touch panel 831 and other input devices 832), a display unit 840 (including a display panel 841), a sensor 850, an audio circuit 860 (which can be connected to a speaker 861 and a microphone 862), a wireless fidelity (WiFi) module 870, a processor 880, and a power supply 890. Those skilled in the art will appreciate that Figure 8 The structure of the smartphone shown in the figure does not constitute a limitation of the smartphone, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0216] The memory 820 can be used to store software programs and modules. The processor 880 executes various functional applications and data processing of the smartphone by running the software programs and modules stored in the memory 820. The memory 820 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the smartphone (such as audio data, a phone book, etc.), etc. In addition, the memory 820 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0217] The processor 880 is the control center of the smartphone, which uses various interfaces and lines to connect various parts of the entire smartphone, and executes various functions of the smartphone and processes data by running or executing software programs and / or modules stored in the memory 820, and calling data stored in the memory 820. Optionally, the processor 880 may include one or more processing units; preferably, the processor 880 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 880.

[0218] In an embodiment of the present application, the processor 880 included in the terminal is also used to execute the steps of any implementation method of the environmental event detection method provided in the embodiment of the present application.

[0219] See also Fig. 9 , Fig. 9 A schematic diagram of the structure of a server 900 provided for an embodiment of the present application. The server 900 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 922 (for example, one or more processors) and a memory 932, and one or more storage media 930 (for example, one or more mass storage devices) storing application programs 942 or data 944. Among them, the memory 932 and the storage medium 930 may be temporary storage or permanent storage. The program stored in the storage medium 930 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the server. Furthermore, the central processing unit 922 may be configured to communicate with the storage medium 930 to execute a series of instruction operations in the storage medium 930 on the server 900.

[0220] The server 900 may also include one or more power supplies 926, one or more wired or wireless network interfaces 950, one or more input and output interfaces 958, and / or one or more operating systems, such as Windows Server 2000. TM , Mac OS X TM , Unix TM ,Linux TM , FreeBSD TM etc.

[0221] The steps performed by the server in the above embodiment can be based on the Fig. 9 The server structure shown.

[0222] Among them, CPU 922 can also be used to execute the steps of any implementation method of the environmental event detection method provided in the embodiments of the present application.

[0223] An embodiment of the present application also provides a computer-readable storage medium for storing a computer program, which is used to execute any implementation of an environmental event detection method described in the aforementioned embodiments.

[0224] The embodiment of the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes any one of the implementation methods of the environmental event detection method described in the above embodiments.

[0225] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0226] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0227] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0228] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0229] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store computer programs.

[0230] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0231] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting environmental events, characterized in that: The method comprises: Acquire a reference image sequence corresponding to a target image captured by a camera; the reference image sequence includes n frames of reference images captured by the camera before capturing the target image, where n is an integer greater than 1, and the time difference between the capturing time of the reference image and the capturing time of the target image satisfies a preset time condition; Determining first reference data according to an image difference between every two adjacent reference images in the reference image sequence; determining second reference data according to an image difference between the target image and an adjacent image corresponding to the target image; the adjacent image is an image adjacent to the target image captured by the camera before capturing the target image; Based on the first reference data and the second reference data, it is detected whether a target triggering event occurs within the shooting range of the camera when the camera shoots the target image.

2. The method according to claim 1, characterized in that The determining of the first reference data according to the image difference between every two adjacent reference images in the reference image sequence comprises: Based on the n reference image frames in the reference image sequence, construct n-1 reference image pairs; each of the reference image pairs includes two reference image frames with adjacent shooting times; For each of the reference image pairs, determining image difference data corresponding to the reference image pair according to an image difference between two frames of the reference images included in the reference image pair; The average values ​​of the image difference data corresponding to each of the n-1 reference image pairs are calculated as the first reference data.

3. The method according to claim 1 or 2, characterized in that: The image difference between the two adjacent reference images is determined in the following manner: Performing grayscale processing on the two frames of the reference image respectively to obtain grayscale images corresponding to the two frames of the reference image respectively; determining image difference data between the two frames of the reference image based on the grayscale images corresponding to the two frames of the reference image respectively; The determining of the second reference data according to the image difference between the target image and an adjacent image corresponding to the target image comprises: Performing grayscale processing on the target image and the adjacent image respectively to obtain grayscale images corresponding to the target image and the adjacent image respectively; The second reference data is determined based on an image difference between the target image and the grayscale images corresponding to each of the adjacent images.

4. The method according to claim 3, characterized in that The determining of image difference data between the two frames of the reference images based on the grayscale images corresponding to the two frames of the reference images respectively includes: Determine the overall grayscale value corresponding to each frame of the reference image according to the grayscale value of each pixel in the grayscale image corresponding to each frame of the reference image; calculate the difference between the overall grayscale values ​​corresponding to the two frames of the reference image as the image difference data; The determining the second reference data based on the image difference between the target image and the grayscale images corresponding to the adjacent images respectively includes: Determine the overall grayscale value corresponding to the target image according to the grayscale value of each pixel in the grayscale image corresponding to the target image; determine the overall grayscale value corresponding to the adjacent image according to the grayscale value of each pixel in the grayscale image corresponding to the adjacent image; The difference between the overall grayscale value corresponding to the target image and the overall grayscale value corresponding to the adjacent image is calculated as the second reference data.

5. The method according to claim 3, characterized in that: The determining of image difference data between the two frames of the reference images based on the grayscale images corresponding to the two frames of the reference images respectively includes: Constructing first pixel pairs based on pixel points at the same position in the grayscale images corresponding to the two frames of the reference images; determining the image difference data according to the grayscale difference values ​​corresponding to each of the first pixel pairs; The determining the second reference data based on the image difference between the target image and the grayscale images corresponding to the adjacent images respectively includes: Constructing second pixel pairs based on the pixel points located at the same position in the grayscale image corresponding to the target image and the grayscale image corresponding to the adjacent image; determining the second reference data according to the grayscale difference values ​​corresponding to each of the second pixel pairs; The grayscale difference corresponding to the pixel pair is the difference between the grayscale values ​​corresponding to the two pixel points.

6. The method according to claim 1 or 2, characterized in that: The image difference between the two adjacent reference images is determined in the following manner: Through the image feature extraction model, feature extraction processing is performed on the two frames of the reference image respectively to obtain image feature data corresponding to the two frames of the reference image respectively; based on the image feature data corresponding to the two frames of the reference image respectively, image difference data between the two frames of the reference image is determined; The determining of the second reference data according to the image difference between the target image and an adjacent image corresponding to the target image comprises: The target image and the adjacent image are subjected to feature extraction processing respectively through the image feature extraction model to obtain image feature data corresponding to the target image and the adjacent image respectively; and the second reference data is determined based on the image feature data corresponding to the target image and the adjacent image respectively.

7. The method according to claim 1 or 2, characterized in that: The image difference between the two adjacent reference images is determined in the following manner: Determine the image difference data between the two frames of the reference image according to the two frames of the reference image by means of an image difference recognition model; the image difference recognition model is used to determine the image difference data used to characterize the difference between the two images inputted; The determining of the second reference data according to the image difference between the target image and an adjacent image corresponding to the target image comprises: The second reference data is determined according to the target image and the adjacent image by using the image difference recognition model.

8. The method according to claim 1 or 2, characterized in that: The detecting, based on the first reference data and the second reference data, whether a target triggering event occurs within the shooting range of the camera when the camera shoots the target image comprises: When the first reference data is less than the first preset threshold, if the second reference data is greater than or equal to the second preset threshold, it is determined that a target trigger event occurs within the shooting range of the camera when the camera captures the target image; if the second reference data is less than the second preset threshold, it is determined that no target trigger event occurs within the shooting range of the camera when the camera captures the target image.

9. The method according to claim 1 or 2, characterized in that: The detecting, based on the first reference data and the second reference data, whether a target triggering event occurs within the shooting range of the camera when the camera shoots the target image comprises: When the first reference data is greater than or equal to a first preset threshold, calculating a reference data difference between the second reference data and the first reference data; If the reference data difference is greater than or equal to a third preset threshold, it is determined that a target trigger event occurs within the shooting range of the camera when the camera captures the target image; if the reference data difference is less than the third preset threshold, it is determined that no target trigger event occurs within the shooting range of the camera when the camera captures the target image.

10. The method according to claim 1, characterized in that The camera is installed on the target device; the method further includes: If the target triggering event is detected to occur within the shooting range of the camera when the target device is in a sleep state, the target device is controlled to enter a standby state; the power consumption of the target device in the standby state is higher than the power consumption of the target device in the sleep state.

11. The method according to claim 10, characterized in that The method further comprises: Recording the standby time from the moment when the target device enters the standby state; If the standby time reaches a preset time and the target triggering event is not detected within the shooting range of the camera within the standby time, the target device is controlled to enter the sleep state.

12. An environmental event detection device, characterized in that: The device comprises: A sequence acquisition module, used to acquire a reference image sequence corresponding to a target image captured by a camera; the reference image sequence includes n frames of reference images captured by the camera before capturing the target image, where n is an integer greater than 1, and the time difference between the capturing time of the reference image and the capturing time of the target image satisfies a preset time condition; A first determination module, configured to determine first reference data according to an image difference between every two adjacent reference images in the reference image sequence; A second determination module is used to determine second reference data according to an image difference between the target image and an adjacent image corresponding to the target image; the adjacent image is an image adjacent to the target image captured by the camera before capturing the target image; An event detection module is used to detect whether a target triggering event occurs within the shooting range of the camera when the camera shoots the target image based on the first reference data and the second reference data.

13. A computer device, characterized in that: The device includes a processor and a memory; The memory is used to store computer programs; The processor is configured to execute the environmental event detection method according to any one of claims 1 to 11 according to the computer program.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the environmental event detection method according to any one of claims 1 to 11.

15. A computer program product, comprising a computer program or instructions, characterized in that: When the computer program or the instruction is executed by a processor, the environmental event detection method according to any one of claims 1 to 11 is implemented.