Video acquisition method, device, equipment and readable medium based on indoor environment

Abstract

The embodiments of the present disclosure disclose a video acquisition method, device, equipment and readable medium based on an indoor environment. A specific implementation of the method includes: controlling the camera to acquire video data corresponding to the target area; performing background modeling on the target area to generate a regional background model; performing the following determination steps: performing motion target detection processing on the first two regional video frames in the regional video frame sequence according to the regional background model; determining the target type of the moving target; performing behavior analysis processing on the moving target according to the target type and each regional video frame in the regional video frame sequence except the first two regional video frames to generate an analysis result; controlling the intelligent receiving device to perform an alarm operation; controlling the camera to acquire video data corresponding to the target area to obtain a regional video frame sequence. This implementation avoids the waste of storage resources.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a video acquisition method, device, equipment, and readable medium based on an indoor environment. Background Art

[0002] With the popularity of home surveillance, surveillance in the home space plays a variety of roles in the home environment. At present, when performing abnormal monitoring on the home surveillance area, the commonly used method is: real-time video acquisition of the surveillance area, and storing the acquired video in a storage device, and determining whether there is an abnormal situation through the stored video.

[0003] However, it is found in practice that when the above method is used to collect videos of the monitoring area, the following technical problems often occur: the abnormal situation in the monitoring area exists for a short time, and when the video is collected and stored in real time, a large amount of useless video is collected, resulting in a waste of storage resources.

[0004] In the process of adopting technical solutions to solve the above technical problem 1, the following problems often occur: Technical problem 2: The videos collected by home monitoring often include user privacy information, which may be intercepted during video transmission, resulting in leakage of user privacy information and low information security. For the above technical problem 2, the conventional solution is generally to encrypt and transmit all videos. However, the above conventional solution still has the following problems: when all videos are encrypted and transmitted, it takes a long time to decrypt and analyze the encrypted videos, resulting in the inability to detect abnormal situations in the monitoring area in a timely manner, and low user security.

[0005] In the process of adopting technical solutions to solve the above technical problem 1, the following problems often occur: Technical problem 3: Existing home monitoring can only determine whether a stranger has entered, but cannot determine the abnormal behavior of family members, resulting in low home security. For the above technical problem 3, the conventional solution is generally to determine the abnormal behavior of family members through a classification model. However, the above conventional solution still has the following problem: the classification model cannot identify abnormal behavior in long and unsegmented videos, resulting in low home security.

[0006] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0007] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0008] Some embodiments of the present disclosure propose a video acquisition method, device, electronic device, and computer-readable medium based on an indoor environment to solve one or more of the technical problems mentioned in the above background technology section.

[0009] In a first aspect, some embodiments of the present disclosure provide a video acquisition method based on an indoor environment, the method comprising: in response to the current time being a preset acquisition time, controlling the above-mentioned camera to acquire video data corresponding to a target area to obtain a regional video frame sequence, wherein the above-mentioned target area is an indoor area; based on the above-mentioned regional video frame sequence, performing background modeling on the above-mentioned target area to generate a regional background model; based on the regional video frame sequence, performing the following determination steps: based on the above-mentioned regional background model, performing motion target detection processing on the first two regional video frames in the regional video frame sequence; in response to detecting a motion target, determining the target type of the above-mentioned motion target; based on the above-mentioned target type and each regional video frame in the regional video frame sequence except the first two regional video frames, performing behavior analysis processing on the above-mentioned motion target to generate an analysis result; based on the above-mentioned analysis result, controlling the above-mentioned intelligent receiving device to perform an alarm operation; in response to no motion target being detected and the regional video frame sequence except the above-mentioned first two regional video frames is empty, controlling the above-mentioned camera to acquire video data corresponding to the target area to obtain a regional video frame sequence.

[0010] In a second aspect, some embodiments of the present disclosure provide a video acquisition device based on an indoor environment, the device comprising: a control unit, configured to control the above-mentioned camera to acquire video data corresponding to a target area in response to a current time being a preset acquisition time, and obtain a regional video frame sequence, wherein the above-mentioned target area is an indoor area; a background modeling unit, configured to perform background modeling on the above-mentioned target area according to the above-mentioned regional video frame sequence to generate a regional background model; an execution unit, configured to perform the following determination steps based on the regional video frame sequence: performing motion target detection processing on the first two regional video frames in the regional video frame sequence according to the above-mentioned regional background model; determining the target type of the above-mentioned motion target in response to detecting a motion target; performing behavior analysis processing on the above-mentioned motion target according to the above-mentioned target type and each regional video frame in the regional video frame sequence except the first two regional video frames to generate an analysis result; controlling the above-mentioned intelligent receiving device to perform an alarm operation according to the above-mentioned analysis result; in response to no motion target being detected and the regional video frame sequence except the above-mentioned first two regional video frames being empty, controlling the above-mentioned camera to acquire video data corresponding to the target area, and obtain a regional video frame sequence.

[0011] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0012] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0013] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through the video acquisition method based on the indoor environment of some embodiments of the present disclosure, the waste of storage resources is avoided. Specifically, the reason for the waste of storage resources is that the time when the abnormal situation exists in the monitoring area is short, and when the video is collected and stored in real time, a large amount of useless video is collected, resulting in a waste of storage resources. Based on this, the video acquisition method based on the indoor environment of some embodiments of the present disclosure, first, in response to the current time being the preset acquisition time, controls the above-mentioned camera to collect the video data corresponding to the target area, and obtains the regional video frame sequence. Thus, indoor video data can be obtained. Secondly, according to the above-mentioned regional video frame sequence, the background modeling of the above-mentioned target area is performed to generate a regional background model. Thus, an indoor static model can be generated. Then, based on the regional video frame sequence, the following determination steps are performed: First, according to the above-mentioned regional background model, the first two regional video frames in the regional video frame sequence are subjected to moving target detection processing. Thus, it can be determined whether there is a moving target in the target area. Second, in response to the detection of a moving target, the target type of the moving target is determined; according to the target type and each regional video frame in the regional video frame sequence except the first two regional video frames, the moving target is subjected to behavior analysis processing to generate an analysis result. Thus, by determining the type of the moving target, a behavior analysis can be performed to determine whether it is an abnormal behavior. Third, according to the above analysis results, the above intelligent receiving device is controlled to perform an alarm operation. Thus, an alarm can be issued for an abnormal situation. Fourth, in response to the fact that no moving target is detected and the regional video frame sequence except the first two regional video frames is empty, the above camera is controlled to collect video data corresponding to the target area to obtain a regional video frame sequence. Thus, the video data can be re-collected and the abnormal situation can be determined. Therefore, the video of the indoor area is collected only at the preset collection time, and the collected video can be deleted when an abnormal situation occurs, thereby avoiding the storage of too many useless videos, thereby avoiding the waste of storage resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0015] Figure 1 is a flow chart of some embodiments of the video acquisition method based on indoor environment according to the present disclosure;

[0016] Figure 2 is a schematic structural diagram of some embodiments of a video acquisition device based on an indoor environment according to the present disclosure;

[0017] Figure 3 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0018] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0019] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0020] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0021] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0022] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0023] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0024] Figure 1 The process 100 of some embodiments of the video acquisition method based on indoor environment according to the present disclosure is shown. The video acquisition method based on indoor environment includes the following steps:

[0025] Step 101, in response to the current time being a preset acquisition time, controlling the camera to acquire video data corresponding to a target area to obtain a regional video frame sequence.

[0026] In some embodiments, the execution subject (e.g., a server) of the video acquisition method based on an indoor environment can control the camera to acquire video data corresponding to the target area in response to the current time being a preset acquisition time, and obtain a regional video frame sequence. Among them, the video acquisition method based on an indoor environment can be applied to a camera module. The camera module may include a camera and an intelligent receiving device. The camera is installed in an indoor environment where the target area can be acquired. The intelligent receiving device is used to receive the video acquired by the camera and analyze the video. The preset acquisition time may be a pre-set time for acquiring the video of the target area. For example, the preset acquisition time may be the first second of every minute. The target area may be an indoor area.

[0027] As an example, the execution entity may control the camera to collect half a minute of video data corresponding to the target area at the first second of every minute.

[0028] Step 102: Perform background modeling on the target region according to the regional video frame sequence to generate a regional background model.

[0029] In some embodiments, the execution subject may perform background modeling on the target region according to the regional video frame sequence to generate a regional background model. In practice, the target region may be modeled using a Gaussian mixture background modeling algorithm to generate a regional background model.

[0030] Considering the problem that the conventional solution mentioned above encrypts all videos for transmission, facing the above technical problem 2: the videos collected by home monitoring often include user privacy information, which may be intercepted during video transmission, resulting in leakage of user privacy information and low information security. Combining the advantages, we can decide to adopt the following solution.

[0031] Optionally, after step 102, the following steps are further included:

[0032] In the first step, a difference process is performed on each regional video frame in the regional video frame sequence to generate a regional binary image, thereby obtaining a regional binary image sequence.

[0033] In some embodiments, the execution subject may perform differential processing on each regional video frame in the regional video frame sequence to generate a regional binary image and obtain a regional binary image sequence. The differential processing may be performing differential processing on each regional video frame in the regional video frame sequence using a Gaussian mixture background difference algorithm.

[0034] In the second step, for each regional binary image in the regional binary image sequence, a pixel ratio value corresponding to the regional binary image is determined according to the regional background model.

[0035] In some embodiments, the execution subject may determine, for each regional binary image in the regional binary image sequence, a pixel ratio value corresponding to the regional binary image according to the regional background model. In practice, for each regional binary image in the regional binary image sequence, a ratio of the number of foreground pixels in the regional binary image to the number of pixels included in the regional binary image is determined as the pixel ratio value.

[0036] In the third step, according to the pixel ratio values ​​corresponding to the regional binary image sequence, the regional binary images included in the regional binary image sequence are classified to generate a first binary image sequence and a second binary image sequence.

[0037] In some embodiments, the execution subject may classify the regional binary images included in the regional binary image sequence according to the pixel proportions corresponding to the regional binary image sequence to generate a first binary image sequence and a second binary image sequence. In practice, first, a preset proportion threshold may be determined. Second, for each pixel proportion value among the pixel proportion values ​​corresponding to the regional binary image sequence, in response to the pixel proportion value being greater than the preset proportion threshold, the regional binary image corresponding to the pixel proportion value is determined as the first binary image. In response to the pixel proportion value being less than or equal to the preset proportion threshold, the regional binary image corresponding to the pixel proportion value is determined as the second binary image.

[0038] Step 4: For each first binary image in the first binary image sequence, perform the following encryption steps:

[0039] The first encryption step is to perform discrete wavelet transform processing on the first binary image to generate an image high-frequency information matrix and an image low-frequency information matrix.

[0040] In some embodiments, the execution subject may perform discrete wavelet transform processing on the first binary image to generate an image high-frequency information matrix and an image low-frequency information matrix, wherein the length and width of the image low-frequency information matrix are respectively half of the length and width of the first binary image.

[0041] The second encryption step is to reconstruct the image low-frequency information matrix to generate reconstructed low-frequency information.

[0042] In some embodiments, the execution subject may reconstruct the image low-frequency information matrix to generate reconstructed low-frequency information, wherein the reconstructed low-frequency information includes three channels.

[0043] In the third encryption step, the low-frequency information corresponding to the images corresponding to the three channels are combined into an information matrix.

[0044] In some embodiments, the execution subject may combine the low-frequency information corresponding to the images corresponding to the three channels into an information matrix. In practice, four images may be taken as a group, and three channels may be twelve images to form a combination matrix. Here, in response to the number of remaining images being less than four, a zero matrix of the same size is generated for completion processing.

[0045] The fourth encryption step is to perform branch encryption processing on the above information matrix to generate an encryption matrix.

[0046] In some embodiments, the execution subject may perform branch encryption processing on the information matrix to generate an encryption matrix. In practice, the information matrix may be input into a pre-trained branch encryption network to generate an encryption matrix. The branch encryption network may be a Feistel network.

[0047] The fifth step is to transmit each generated encryption matrix.

[0048] In some embodiments, the execution entity may transmit each generated encryption matrix.

[0049] The first step to the fifth step mentioned above, as an inventive point of an embodiment of the present disclosure, solves the second technical problem mentioned in the background technology, "the video collected by home monitoring often includes user privacy information, which may be intercepted during the video transmission process, resulting in the leakage of user privacy information and low information security." The reasons for the leakage of user privacy information and low information security are as follows: the video collected by home monitoring often includes user privacy information, which may be intercepted during the video transmission process, resulting in the leakage of user privacy information and low information security. If the above factors are solved, the effect of avoiding the leakage of user privacy information and improving information security can be achieved. In order to achieve this effect, the present disclosure first performs differential processing on each regional video frame in the above regional video frame sequence to generate a regional binary image and obtain a regional binary image sequence. Thereby, motion detection can be performed to determine the moving target. Second, for each regional binary image in the above regional binary image sequence, according to the above regional background model, the pixel ratio corresponding to the above regional binary image is determined. Thereby, the ratio of the foreground moving target to the entire binary image can be determined. Third, according to the pixel ratio values ​​corresponding to the above-mentioned regional binary image sequence, the regional binary images included in the regional binary image sequence are classified to generate a first binary image sequence and a second binary image sequence. Thus, it is possible to determine whether the image is a sensitive image by the ratio value, and to classify sensitive images and non-sensitive images. Fourth, for each first binary image in the above-mentioned first binary image sequence, the following encryption steps are performed: First, the above-mentioned first binary image is subjected to discrete wavelet transform processing to generate an image high-frequency information matrix and an image low-frequency information matrix. Thus, the high-frequency feature information and low-frequency feature information of the image can be distinguished. Secondly, the image low-frequency information matrix is ​​reconstructed to generate reconstructed low-frequency information. Thus, the low-frequency information can be reconstructed and the first encryption can be performed. Then, the low-frequency information corresponding to the images corresponding to the above-mentioned three channels is combined into an information matrix. Thus, an information matrix can be generated through three channels. Finally, the above-mentioned information matrix is ​​subjected to branch encryption processing to generate an encryption matrix. Thus, the second encryption can be performed through branch encryption to obtain an encryption matrix. Thus, the low-frequency information of the sensitive image can be encrypted twice, thereby avoiding the leakage of user privacy information and improving information security. Fifth, the generated encryption matrices are transmitted. Thus, the transmission of the encrypted video is completed. Avoiding the leakage of user privacy information and improving information security.

[0050] Step 103, based on the regional video frame sequence, perform the following determination steps:

[0051] Step 1031 , performing moving target detection processing on the first two regional video frames in the regional video frame sequence according to the regional background model.

[0052] In some embodiments, the execution entity may perform moving target detection processing on the first two regional video frames in the regional video frame sequence according to the regional background model.

[0053] Step 1032, in response to detecting a moving target, determining a target type of the moving target.

[0054] In some embodiments, the execution subject may determine the target type of the moving target in response to detecting the moving target, wherein the target type may be the biological type of the moving target.

[0055] Step 1033 , performing behavior analysis processing on the moving target according to the target type and each regional video frame in the regional video frame sequence except the first two regional video frames, to generate an analysis result.

[0056] In some embodiments, the execution subject may perform behavior analysis processing on the moving target according to the target type and each regional video frame in the regional video frame sequence except the first two regional video frames to generate an analysis result.

[0057] In practice, the above-mentioned motion targets can be analyzed and processed through the following steps:

[0058] In the first step, in response to the target type being the first target type, performing key frame extraction processing on the video frames of each region to generate a key frame group. The first target type represents an animal. For example, the first target type may be a cat or a dog.

[0059] In the second step, according to the key frame group, the motion trajectory detection process is performed on the motion target to generate motion trajectory information. In practice, the motion trajectory information of the motion target can be determined by determining the position of the motion target in each key frame in the key frame group. The motion trajectory information can include multiple motion trajectory coordinates.

[0060] The third step is to generate a motion trajectory range according to the motion trajectory information. In practice, the range of the motion trajectory of the moving target within the background model can be determined by using the coordinates of each motion trajectory included in the motion trajectory information.

[0061] Step 4: For each key frame in the above key frame group, perform the following processing steps:

[0062] In the first processing step, the moving target displayed in the key frame is subjected to contour detection processing to generate a contour-annotated image. In practice, the moving target can be subjected to contour detection processing to generate a contour-annotated image by using a target contour detection method based on OpenCV. Here, the moving target is annotated in the contour-annotated image by a minimum tangent rectangle.

[0063] The second processing step is to perform cropping processing on the above-mentioned contour-annotated image to generate a cropped annotated image.

[0064] The third processing step is to perform behavior analysis on the cropped and annotated image to generate a behavior analysis result. In practice, the cropped and annotated image can be input into a pre-trained animal behavior analysis model to generate a behavior analysis result. The behavior analysis model can be a classification model that takes the cropped and annotated image as input and takes the behavior analysis result as output.

[0065] The fifth step is to combine the generated behavior analysis results and the above-mentioned motion trajectory range into an analysis result.

[0066] Optionally, after the fifth step, the following steps are further included:

[0067] Step 6: In response to the target type being the second target type, each of the regional video frames except the first two regional video frames in the regional video frame sequence is determined as a target video frame sequence. The second target type may represent a person.

[0068] In the seventh step, each target video frame in the target video frame sequence is subjected to key frame extraction processing to generate a target key frame sequence. In practice, each target video frame in the target video frame sequence can be subjected to key frame extraction processing by an OpenCV-based video key frame extraction algorithm to generate a target key frame sequence.

[0069] In the eighth step, for each target key frame in the target key frame sequence, the target key frame is input into a pre-trained abnormal behavior detection model to obtain a behavior detection result.

[0070] In practice, the above abnormal behavior detection model can be a pre-trained predefined model. Among them, the above predefined model includes a nine-layer network structure. The first layer, the input layer, is used to input each target key frame included in the target key frame sequence. The second layer, the third layer and the fourth layer respectively include a pseudo three-dimensional convolution residual network and three pooling layers. The three pseudo three-dimensional convolution residual networks are different. The above three different pseudo three-dimensional convolution residual networks all include two 1×1×1 convolution layers, a 1×3×3 two-dimensional spatial convolution and a 3×1×1 one-dimensional temporal convolution, and the three different pseudo three-dimensional convolution residual networks include different structures of convolution layers. The first pseudo three-dimensional convolution residual network stacks four convolutions. The second pseudo three-dimensional convolution residual network connects a 1×3×3 two-dimensional spatial convolution and a 3×1×1 one-dimensional temporal convolution in parallel in different paths. The third pseudo three-dimensional convolution residual network simultaneously establishes a 1×3×3 two-dimensional spatial convolution and a 3×1×1 one-dimensional temporal convolution and a direct impact between the final output. The fifth layer, the sixth layer and the seventh layer respectively include a custom convolution residual network, wherein the fifth layer also includes a pooling layer. The above three custom convolution residual networks are respectively generated by combining the above three different pseudo three-dimensional convolution residual networks with the first preset module and the second preset model. The above first preset module includes an average pooling feature layer and a maximum pooling feature layer, a multilayer perceptron with a hidden layer, an element addition algorithm and an activation function. The above second preset module includes an average pooling feature layer and a maximum pooling feature layer, a standard convolution layer and an activation function. The above first preset module is cascaded after a 1×3×3 two-dimensional spatial convolution. The above second preset module is cascaded after a 3×1×1 one-dimensional temporal convolution. The eighth layer, the global pooling layer, is used to prevent overfitting and reduce the dimension of the input features. The ninth layer, the output layer, includes a softmax layer for outputting a key frame score. Here, in response to the above key frame score being greater than or equal to the preset score, an analysis result characterizing the abnormality is generated.

[0071] The eighth step mentioned above, as an inventive point of an embodiment of the present disclosure, solves the technical problem three mentioned in the background technology, "the existing home monitoring can only determine whether a stranger enters, but cannot determine the abnormal behavior of family members, resulting in low home security". The reasons for the low home security are as follows: the existing home monitoring can only determine whether a stranger enters, but cannot determine the abnormal behavior of family members, resulting in low home security. If the above factors are solved, the effect of improving home security can be achieved. In order to achieve this effect, the present disclosure can effectively extract features in the video through the features of both space and time through a predefined model, improve the accuracy of distinguishing abnormal behaviors, and can focus on important features and suppress unnecessary features through a convolutional attention module to improve the expression ability of the feature map. In order to effectively extract key information in the video frame and enhance the key features of human behavior, the first preset module and the second preset module are used to focus on each channel of the frame in the video and the timing information between two frames, obtain the key frames in the video, improve the model expression ability, and enhance the key spatial features on each frame. Thereby, the accuracy of behavior recognition can be improved, thereby improving home security.

[0072] In step 9, in response to at least one behavior detection result satisfying a first preset condition among the obtained behavior detection results, an analysis result corresponding to the at least one behavior detection result satisfying the first preset condition is generated. The first preset condition may be that the behavior detection result represents abnormal behavior.

[0073] Step 1034, based on the analysis result, control the intelligent receiving device to perform an alarm operation.

[0074] In some embodiments, the execution subject may control the intelligent receiving device to perform an alarm operation according to the analysis result. In practice, in response to the analysis result indicating the presence of an abnormal situation, the intelligent receiving device may be controlled to send an alarm message to an associated terminal. The associated terminal may be a user terminal connected to the execution subject by wireless or wired connection. For example, the terminal may be a mobile phone or a computer.

[0075] Step 1035 , in response to no moving target being detected and the regional video frame sequence excluding the first two regional video frames being empty, the camera is controlled to collect video data corresponding to the target region to obtain a regional video frame sequence.

[0076] In some embodiments, the execution subject may control the camera to collect video data corresponding to the target area in response to the fact that no moving target is detected and the regional video frame sequence excluding the first two regional video frames is empty, to obtain a regional video frame sequence.

[0077] Optionally, in response to no moving target being detected and the regional video frame sequence excluding the first two regional video frames is not empty, the regional video frame sequence excluding the first two regional video frames is taken as the regional video frame sequence, and the above determination step is performed again.

[0078] In some embodiments, the above-mentioned execution entity can respond to the fact that no moving target is detected and the regional video frame sequence excluding the above-mentioned first two regional video frames is not empty, and take the regional video frame sequence excluding the above-mentioned first two regional video frames as the regional video frame sequence, and execute the above-mentioned determination step again.

[0079] Optionally, after step 105, the following steps are further included:

[0080] In the first step, in response to the target type corresponding to the moving target being the first target type and the moving target satisfying a second preset condition, the regional video frame sequence is deleted, wherein the second preset condition is that the moving target does not move again.

[0081] In the second step, in response to the target type corresponding to the moving target being the first target type and the moving target not satisfying the second preset condition, the camera is controlled to perform intermittent target acquisition on the moving target. The intermittent target acquisition may be performed at intervals of a preset duration, and the camera is controlled to perform video acquisition.

[0082] In the third step, in response to the target type corresponding to the above-mentioned sports target being the second target type, determining whether the above-mentioned sports target meets a third preset condition. The above-mentioned third preset condition is that the sports target is a pre-stored target. As an example, the above-mentioned sports target can be a pre-set family member.

[0083] In the fourth step, in response to the above-mentioned moving target not satisfying the above-mentioned third preset condition, abnormal personnel warning information is generated, and the above-mentioned abnormal personnel warning information is sent to the associated terminal for display. Among them, the above-mentioned abnormal personnel warning information can represent the existence of an unstored moving target. The above-mentioned associated terminal can be a user terminal connected to the above-mentioned execution subject by wire or wireless connection. Here, it can also be sent to the software installed in the associated terminal.

[0084] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through the video acquisition method based on the indoor environment of some embodiments of the present disclosure, the waste of storage resources is avoided. Specifically, the reason for the waste of storage resources is that the time when the abnormal situation exists in the monitoring area is short, and when the video is collected and stored in real time, a large amount of useless video is collected, resulting in a waste of storage resources. Based on this, the video acquisition method based on the indoor environment of some embodiments of the present disclosure, first, in response to the current time being the preset acquisition time, controls the above-mentioned camera to collect the video data corresponding to the target area, and obtains the regional video frame sequence. Thus, indoor video data can be obtained. Secondly, according to the above-mentioned regional video frame sequence, the background modeling of the above-mentioned target area is performed to generate a regional background model. Thus, an indoor static model can be generated. Then, based on the regional video frame sequence, the following determination steps are performed: First, according to the above-mentioned regional background model, the first two regional video frames in the regional video frame sequence are subjected to moving target detection processing. Thus, it can be determined whether there is a moving target in the target area. Second, in response to the detection of a moving target, the target type of the moving target is determined; according to the target type and each regional video frame in the regional video frame sequence except the first two regional video frames, the moving target is subjected to behavior analysis processing to generate an analysis result. Thus, by determining the type of the moving target, a behavior analysis can be performed to determine whether it is an abnormal behavior. Third, according to the above analysis results, the above intelligent receiving device is controlled to perform an alarm operation. Thus, an alarm can be issued for an abnormal situation. Fourth, in response to the fact that no moving target is detected and the regional video frame sequence except the first two regional video frames is empty, the above camera is controlled to collect video data corresponding to the target area to obtain a regional video frame sequence. Thus, the video data can be re-collected and the abnormal situation can be determined. Therefore, the video of the indoor area is collected only at the preset collection time, and the collected video can be deleted when an abnormal situation occurs, thereby avoiding the storage of too many useless videos, thereby avoiding the waste of storage resources.

[0085] Further references Figure 2 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a video acquisition device based on an indoor environment. These device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the video acquisition device based on the indoor environment can be specifically applied to various electronic devices.

[0086] like Figure 2As shown, the video acquisition device 200 based on the indoor environment of some embodiments includes: a control unit 201, a background modeling unit 202 and an execution unit 203. Among them, the control unit 201 is configured to control the above-mentioned camera to collect video data corresponding to the target area in response to the current time being the preset collection time, and obtain a regional video frame sequence, wherein the above-mentioned target area is an indoor area; the background modeling unit 202 is configured to perform background modeling on the above-mentioned target area according to the above-mentioned regional video frame sequence to generate a regional background model; the execution unit 203 is configured to perform the following determination steps based on the regional video frame sequence: according to the above-mentioned regional background model, the first two regional video frames in the regional video frame sequence are subjected to motion target detection processing; in response to the detection of the motion target, the target type of the above-mentioned motion target is determined; according to the above-mentioned target type and each regional video frame in the regional video frame sequence except the first two regional video frames, the behavior analysis processing is performed on the above-mentioned motion target to generate an analysis result; according to the above-mentioned analysis result, the above-mentioned intelligent receiving device is controlled to perform an alarm operation; in response to the detection of no motion target and the regional video frame sequence except the first two regional video frames is empty, the above-mentioned camera is controlled to collect video data corresponding to the target area to obtain a regional video frame sequence.

[0087] It can be understood that the units recorded in the video acquisition device 200 based on the indoor environment are similar to the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the video acquisition device 200 based on the indoor environment and the units contained therein, and will not be described in detail here.

[0088] Reference below Figure 3 , which shows a schematic diagram of the structure of an electronic device 300 suitable for implementing some embodiments of the present disclosure. The electronic devices in some embodiments of the present disclosure may include but are not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0089] like Figure 3As shown, the electronic device 300 may include a processing device 301 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0090] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0091] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0092] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0093] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0094] The computer-readable medium may be included in the electronic device; or it may exist separately without being assembled into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: in response to the current time being the preset acquisition time, the camera is controlled to acquire the video data corresponding to the target area to obtain a regional video frame sequence, wherein the target area is an indoor area. According to the regional video frame sequence, the target area is subjected to background modeling to generate a regional background model. Based on the regional video frame sequence, the following determination steps are performed: according to the regional background model, the first two regional video frames in the regional video frame sequence are subjected to moving target detection processing; in response to the detection of a moving target, the target type of the moving target is determined; according to the target type and each regional video frame in the regional video frame sequence except the first two regional video frames, the moving target is subjected to behavior analysis processing to generate an analysis result; according to the analysis result, the intelligent receiving device is controlled to perform an alarm operation; in response to the detection of no moving target and the regional video frame sequence except the first two regional video frames is empty, the camera is controlled to acquire the video data corresponding to the target area to obtain a regional video frame sequence.

[0095] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0096] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0097] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be provided in a processor, for example, may be described as: a processor comprising a control unit, a background modeling unit, and an execution unit. The names of these units do not, in some cases, constitute limitations on the units themselves, for example, the control unit may also be described as "a unit that controls the above-mentioned camera to collect video data corresponding to the target area in response to the current time being a preset acquisition time, and obtains a regional video frame sequence".

[0098] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0099] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A video acquisition method based on an indoor environment, applied to a camera module, wherein: The camera module includes: a camera and an intelligent receiving device, and the video acquisition method based on the indoor environment includes: In response to the current time being the preset acquisition time, controlling the camera to acquire video data corresponding to the target area to obtain a regional video frame sequence, wherein the target area is an indoor area; Performing background modeling on the target region according to the regional video frame sequence to generate a regional background model; Performing differential processing on each regional video frame in the regional video frame sequence to generate a regional binary image, thereby obtaining a regional binary image sequence; For each regional binary image in the regional binary image sequence, determining a pixel ratio value corresponding to the regional binary image according to the regional background model; According to each pixel ratio value corresponding to the regional binary image sequence, each regional binary image included in the regional binary image sequence is classified and processed to generate a first binary image sequence and a second binary image sequence; For each first binary image in the first binary image sequence, the following encryption steps are performed: The first binary image is subjected to discrete wavelet transform processing to generate an image high-frequency information matrix and an image low-frequency information matrix: Reconstructing the image low-frequency information matrix to generate reconstructed low-frequency information; The low-frequency information corresponding to the images corresponding to the three channels is combined into an information matrix; Performing branch encryption processing on the information matrix to generate an encryption matrix; Transmitting each generated encryption matrix; Based on the regional video frame sequence, the following determination steps are performed: According to the regional background model, performing moving target detection processing on the first two regional video frames in the regional video frame sequence; In response to detecting a moving object, determining an object type of the moving object; According to the target type and each regional video frame in the regional video frame sequence except the first two regional video frames, the behavior analysis processing is performed on the moving target to generate an analysis result; According to the analysis result, controlling the intelligent receiving device to perform an alarm operation; In response to no moving target being detected and the regional video frame sequence excluding the first two regional video frames being empty, controlling the camera to collect video data corresponding to the target region to obtain a regional video frame sequence; The performing behavior analysis on the moving target according to the target type and each regional video frame in the regional video frame sequence except the first two regional video frames to generate an analysis result includes: In response to the target type being the first target type, performing key frame extraction processing on each of the regional video frames to generate a key frame group; According to the key frame group, performing motion trajectory detection processing on the moving target to generate motion trajectory information; Generate a motion trajectory range according to the motion trajectory information; For each key frame in the key frame group, the following processing steps are performed: Performing contour detection processing on the moving object displayed in the key frame to generate a contour-annotated image; Performing cropping processing on the contour-annotated image to generate a cropped annotated image; Performing behavior analysis processing on the cropped and annotated image to generate a behavior analysis result; The generated behavior analysis results and the motion trajectory range are combined into an analysis result.

2. The method according to claim 1, wherein: The method further comprises: In response to no moving target being detected and the regional video frame sequence excluding the first two regional video frames is not empty, the regional video frame sequence excluding the first two regional video frames is used as the regional video frame sequence, and the determining step is performed again.

3. The method according to claim 1, wherein: The performing behavior analysis processing on the moving target according to the target type and each regional video frame in the regional video frame sequence except the first two regional video frames to generate an analysis result also includes: In response to the target type being the second target type, determining each regional video frame except the first two regional video frames in the regional video frame sequence as a target video frame sequence; Performing key frame extraction processing on each target video frame in the target video frame sequence to generate a target key frame sequence; For each target key frame in the target key frame sequence, input the target key frame into a pre-trained abnormal behavior detection model to obtain a behavior detection result; In response to at least one behavior detection result satisfying a first preset condition among the obtained behavior detection results, an analysis result corresponding to the at least one behavior detection result satisfying the first preset condition is generated.

4. The method according to claim 1, wherein: The method further comprises: In response to the target type corresponding to the moving target being the first target type and the moving target satisfying a second preset condition, deleting the regional video frame sequence; In response to the target type corresponding to the moving target being the first target type and the moving target not satisfying the second preset condition, controlling the camera to perform intermittent target acquisition on the moving target; In response to the target type corresponding to the moving target being the second target type, determining whether the moving target satisfies a third preset condition; In response to the moving target not satisfying the third preset condition, abnormal person warning information is generated, and the abnormal person warning information is sent to an associated terminal for display.

5. A video acquisition device based on an indoor environment, comprising: A control unit is configured to control the camera to collect video data corresponding to a target area in response to the current time being a preset collection time, so as to obtain a regional video frame sequence, wherein the target area is an indoor area; The background modeling unit is configured to perform background modeling on the target area according to the regional video frame sequence to generate a regional background model; the background modeling unit is further configured to: Performing differential processing on each regional video frame in the regional video frame sequence to generate a regional binary image, thereby obtaining a regional binary image sequence; For each regional binary image in the regional binary image sequence, determining a pixel ratio value corresponding to the regional binary image according to the regional background model; According to each pixel ratio value corresponding to the regional binary image sequence, each regional binary image included in the regional binary image sequence is classified and processed to generate a first binary image sequence and a second binary image sequence; For each first binary image in the first binary image sequence, the following encryption steps are performed: The first binary image is subjected to discrete wavelet transform processing to generate an image high-frequency information matrix and an image low-frequency information matrix: Reconstructing the image low-frequency information matrix to generate reconstructed low-frequency information; The low-frequency information corresponding to the images corresponding to the three channels is combined into an information matrix; Performing branch encryption processing on the information matrix to generate an encryption matrix; Transmitting each generated encryption matrix; The execution unit is configured to perform the following determination steps based on the regional video frame sequence: according to the regional background model, the first two regional video frames in the regional video frame sequence are subjected to moving target detection processing; in response to detecting a moving target, the target type of the moving target is determined; according to the target type and each regional video frame in the regional video frame sequence except the first two regional video frames, the behavior analysis processing of the moving target is performed to generate an analysis result; according to the analysis result, the intelligent receiving device is controlled to perform an alarm operation; in response to no moving target being detected and the regional video frame sequence except the first two regional video frames is empty, the camera is controlled to collect video data corresponding to the target area to obtain a regional video frame sequence; the execution unit is further configured to: In response to the target type being the first target type, performing key frame extraction processing on each of the regional video frames to generate a key frame group; According to the key frame group, performing motion trajectory detection processing on the moving target to generate motion trajectory information; Generate a motion trajectory range according to the motion trajectory information; For each key frame in the key frame group, the following processing steps are performed: Performing contour detection processing on the moving object displayed in the key frame to generate a contour-annotated image; Performing cropping processing on the contour-annotated image to generate a cropped annotated image; Performing behavior analysis processing on the cropped and annotated image to generate a behavior analysis result; The generated behavior analysis results and the motion trajectory range are combined into an analysis result.

6. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 4.

7. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images