Camera and router collaborative working method and system for smart home

Abstract

The present invention relates to the field of electrical communication technology, and discloses a method and system for the coordinated work of a camera and a router for smart homes, the method comprising: establishing a data transmission channel and an instruction issuing channel between multiple cameras and a router; marking non-repetitive videos in fixed videos by a data processing program in the router; performing target detection on flexible videos by a data processing program in the router, and judging whether there is a special focus target in the detection target; calculating the target movement of the special focus target by using the data processing program, setting the next time period of the flexible camera by using the target movement, and comparing whether the video picture corresponding to the next time period of the steering has a video conflict with the non-repetitive video; performing fixed instruction execution on the fixed control instruction in the flexible camera to obtain a fixed instruction execution result; and performing flexible instruction execution on the flexible control instruction in the flexible camera. The present invention enables the router to replace the sending end to issue control instructions to the camera.

Description

Technical Field

[0001] The present invention relates to the field of electrical communication technology, and in particular to a camera and router collaborative working method and system for smart homes. Background Art

[0002] As a type of smart home, the camera can monitor the scene at home. The router, as a data transceiver, can receive the image data collected by the camera and then transfer the image data to the cloud or the local storage device of the mobile phone. In this process, the router mainly plays the role of a data transmission intermediary.

[0003] At present, some solutions try to add data processing and calculation software to the router, so that after receiving the request from the camera or mobile device, the router can process and calculate the request data, and then feed back the calculation results to the camera, cloud and mobile device. However, in the process of the router acting as an intermediary, it is the cloud or mobile device and other sending ends that calculate and process the data, and send control instructions to the router, which then transfers the control instructions to the camera. There is no case of directly controlling the operation of the camera through the router.

[0004] For example, the invention patent with announcement number CN105898225B discloses a real-time camera monitoring method based on Openwrt router, which is divided into two parts. The first part is the remote real-time viewing of video monitoring information, which uses the MJPG-streamer solution to achieve image acquisition, and transmits the video to the browser on the PC through the HTTP protocol to achieve the function of remote monitoring; the second part is the cloud storage of video content, which saves the video obtained from the camera in the form of pictures locally at regular intervals and uploads it to the FTP server specified by the user. The present invention realizes the remote monitoring function and the timed capture function of the camera on the Openwrt router, and can directly upload the pictures to the designated cloud server for storage, with larger storage space, safer information, and lower cost.

[0005] Therefore, a solution technology is urgently needed that allows the router to send control instructions to the camera instead of the sender. Summary of the invention

[0006] In order to solve the above problems, the present invention provides a method and system for the coordinated working of a camera and a router for a smart home, which can enable the router to send control instructions to the camera instead of the sending end.

[0007] In a first aspect, the present invention provides a method for collaborative working of a camera and a router for a smart home, comprising:

[0008] After the plurality of cameras are wirelessly connected to the router, a data transmission channel and a command issuing channel are established between the plurality of cameras and the router, and a fixed steering mode and a flexible steering mode of the plurality of cameras are set, wherein the router includes a data processing program and a data transmission function;

[0009] Obtain a fixed camera in the fixed steering mode among the multiple cameras, transmit the fixed video collected by the fixed camera to the router through the data transmission channel, mark the non-repetitive videos in the fixed video through the data processing program in the router, and transmit the non-repetitive videos and the repetitive videos in the fixed video to a storage device through the data transmission function in the router to complete the collaborative work between the fixed camera and the router, and obtain the collaborative work result between the fixed camera and the router;

[0010] Acquire a flexible camera in the flexible steering mode among the multiple cameras, transmit the flexible video captured by the flexible camera to the router, and transmit the steering range and current steering of the flexible camera to the router respectively, perform target detection on the flexible video through a data processing program in the router to obtain a detected target, transmit the flexible video to the storage device to complete the first collaborative work between the flexible camera and the router, obtain the first collaborative work result between the flexible camera and the router, and judge whether there is a target of special concern among the detected targets;

[0011] When the special attention target exists in the detection target, the target movement of the special attention target is calculated by using the data processing program, and the next time period steering of the flexible camera is set by using the target movement based on the steering range and the current steering, and the non-repetitive video in the storage device is retrieved in the router to compare whether the video picture corresponding to the next time period steering has a video conflict with the non-repetitive video;

[0012] When a video conflict occurs between the video screen corresponding to the next time period and the non-repetitive video, a fixed control instruction is sent from the router to the flexible camera through the instruction sending channel, and the fixed control instruction is executed in the flexible camera to obtain a fixed instruction execution result;

[0013] When the corresponding video screen in the next time period does not conflict with the non-repetitive video, a flexible control instruction is sent from the router to the flexible camera through the instruction sending channel, and the flexible control instruction is flexibly executed in the flexible camera to obtain a flexible instruction execution result, and the fixed instruction execution result and the flexible instruction execution result are used as the second collaborative work result between the flexible camera and the router.

[0014] In a possible implementation manner of the first aspect, establishing a data transmission channel and an instruction issuing channel between the multiple cameras and the router includes:

[0015] Respectively constructing video sending frequencies when the multiple cameras send videos and command sending frequencies when the router sends commands;

[0016] Keeping the video sending frequency and the command sending frequency the same, so as to obtain the same video sending frequency and the same command sending frequency;

[0017] Using the same video transmission frequency to set a first simplex communication channel in the direction from the plurality of cameras to the router;

[0018] determining a video encryption protocol for the first simplex communication channel;

[0019] Using the first simplex communication channel including the video encryption protocol as a data transmission channel;

[0020] Using the same instruction sending frequency to set a second simplex communication channel in the direction from the router to the multiple cameras;

[0021] The second simplex communication channel is used as a command issuing channel between the multiple cameras and the router.

[0022] In a possible implementation manner of the first aspect, marking the non-repetitive videos in the fixed videos by a data processing program in the router includes:

[0023] The data processing program includes:

[0024] Sorting the fixed videos in ascending order of video time to obtain sorted video frames;

[0025] Starting from a starting video frame in the sorted video frames, sequentially calculating video frame similarities between the starting video frame and subsequent video frames of the starting video frame;

[0026] When the video frame similarity is not greater than a preset similarity, stop calculating the video frame similarity between the starting video frame and subsequent video frames of the starting video frame;

[0027] Obtaining a target subsequent video frame whose similarity among the video frames is greater than the preset similarity;

[0028] The gray value distribution distance between the starting video frame and the target subsequent video frame is calculated using the following formula:

[0029] ;

[0030] in, Indicates The gray value distribution distance in the video frame, Indicates the first number between the starting video frame and the target subsequent video frame. In the video frame The size of The gray value is closest to the The distance between gray values ​​of size, Indicates the number of categories of grayscale value size, Indicates The number of gray values ​​of size;

[0031] Selecting the maximum distribution distance from the gray value distribution distances;

[0032] The video frame corresponding to the maximum distribution distance between the starting video frame and the target subsequent video frame is taken as the initial non-repetitive video, and the remaining non-repetitive videos in the sorted video frames are screened in the same manner as the initial non-repetitive video, and the initial non-repetitive video and the remaining non-repetitive videos are taken as the non-repetitive videos.

[0033] In a possible implementation manner of the first aspect, performing target detection on the flexible video by a data processing program in the router to obtain a detection target includes:

[0034] The data processing program includes:

[0035] Detecting the target area in the flexible video using a preset binarization threshold algorithm;

[0036] Determining whether a user of the router sets a special attention target in the router;

[0037] When a user of the router sets a special attention target in the router, calculating a target similarity between the target area and the special attention target;

[0038] When the target similarity is greater than a preset similarity, taking the target area as the detection target;

[0039] When the target similarity is not greater than the preset similarity, determining whether the target of special interest is a dynamic target;

[0040] When the target of special concern is a dynamic target, detecting the dynamic target area in the target area by using a preset inter-frame difference method;

[0041] Detecting a neighborhood target area of ​​the dynamic target area;

[0042] Using the neighborhood target area to identify the area orientation of the dynamic target area;

[0043] Calculating the position-camera distance between the region position and the flexible camera;

[0044] According to the position-camera distance, the true height of the dynamic target area is calculated using the following formula:

[0045] ;

[0046] ;

[0047] in, Indicates the true height, Indicates the target size height of the flexible camera. Indicates the position-camera distance, Indicates the focal length of the flexible camera;

[0048] By comparing the real height with the height of the dynamic target, determining whether the dynamic target area is the dynamic target;

[0049] When the dynamic target area is the dynamic target, taking the dynamic target area as the detection target;

[0050] When the special attention target is not a dynamic target, identifying the target orientation of the target area by using the neighborhood area of ​​the target area;

[0051] By comparing the target orientation with the orientation of the target of special concern, determining whether the target area is the target of special concern;

[0052] When the target area is the special attention target, the target area is used as the detection target.

[0053] In a possible implementation manner of the first aspect, calculating the target movement of the special attention target by using the data processing program includes:

[0054] The data processing program includes:

[0055] Calculate the target center point of the special focus target in each video frame of the flexible video;

[0056] According to the video time sequence of the flexible video from small to large, the coordinate point vector of the target center point between every two consecutive video frames of the flexible video is calculated;

[0057] The target movement of the target of special interest is extracted from the coordinate point vector.

[0058] In a possible implementation of the first aspect, the step of setting the next time period steering of the flexible camera based on the steering range and the current steering and using the target movement includes:

[0059] Calculating a coordinate system mapping relationship between a coordinate system corresponding to the target movement and a coordinate system corresponding to the current turn;

[0060] Mapping the target movement to the coordinate system corresponding to the current turn using the coordinate system mapping relationship to obtain a mapped movement;

[0061] Determining whether the target movement of the flexible camera exceeds the steering range based on the current steering;

[0062] When the target movement of the flexible camera exceeds the steering range based on the current steering, the camera movement that does not exceed the steering range is used as the steering for the next time period;

[0063] When the target movement of the flexible camera based on the current turn does not exceed the turn range, the target movement of the flexible camera based on the current turn will be used as the turn for the next time period.

[0064] In a possible implementation manner of the first aspect, comparing whether a video picture corresponding to the next time period turns to has a video conflict with the non-repetitive video includes:

[0065] Performing similarity matching between the video frame and the non-repetitive video;

[0066] When similarity matching between the video picture and the non-repetitive video is successfully performed, it is determined that a video conflict occurs between the video picture corresponding to the next time period and the non-repetitive video;

[0067] When similarity matching between the video frame and the non-repetitive video fails, it is determined that no video conflict occurs between the video frame corresponding to the next time period and the non-repetitive video.

[0068] In a possible implementation manner of the first aspect, performing fixed instruction execution on the fixed control instruction in the flexible camera to obtain a fixed instruction execution result includes:

[0069] According to the fixing control instruction, after keeping the flexible camera fixed, collect images taken by the flexible camera during the fixing period;

[0070] The images captured during the fixed period are returned to the router so as to determine the execution result of the fixed instruction of the flexible camera through the router.

[0071] In a possible implementation manner of the first aspect, the performing flexible instruction execution on the flexible control instruction in the flexible camera to obtain a flexible instruction execution result includes:

[0072] Performing steering processing on the flexible camera according to the flexible control instruction;

[0073] During the steering process of the flexible camera, the image captured by the flexible camera is temporarily stored in the local memory of the flexible camera to obtain the image stored in the local memory;

[0074] When the steering process of the flexible camera is completed, the image stored in the local memory is returned to the router so that the flexible instruction execution result of the flexible camera can be determined through the router.

[0075] In a second aspect, the present invention provides a camera and router collaborative working system for smart homes, the system comprising:

[0076] A mode setting module is used to establish a data transmission channel and a command issuing channel between the multiple cameras and the router after the multiple cameras are wirelessly connected to the router, and to set a fixed steering mode and a flexible steering mode for the multiple cameras, wherein the router includes a data processing program and a data transmission function;

[0077] A fixed collaboration module, used for acquiring a fixed camera in the fixed steering mode among the multiple cameras, transmitting the fixed video collected by the fixed camera to the router through the data transmission channel, marking non-repetitive videos in the fixed video through the data processing program in the router, and transmitting the non-repetitive videos and the repetitive videos in the fixed video to a storage device through the data transmission function in the router, so as to complete the collaborative work between the fixed camera and the router and obtain the collaborative work result between the fixed camera and the router;

[0078] A first collaborative module is used to obtain a flexible camera in the flexible steering mode among the multiple cameras, transmit the flexible video collected by the flexible camera to the router, and transmit the steering range and current steering of the flexible camera to the router respectively, perform target detection on the flexible video through a data processing program in the router to obtain a detected target, transmit the flexible video to the storage device to complete a first collaborative work between the flexible camera and the router, obtain a first collaborative work result between the flexible camera and the router, and determine whether there is a target of special concern among the detected targets;

[0079] a conflict comparison module, configured to, when the special attention target exists in the detection target, calculate the target movement of the special attention target by using the data processing program, set the next time period steering of the flexible camera by using the target movement based on the steering range and the current steering, and retrieve the non-repetitive video in the storage device in the router to compare whether the video picture corresponding to the next time period steering has a video conflict with the non-repetitive video;

[0080] An instruction execution module is used for sending a fixed control instruction from the router to the flexible camera through the instruction sending channel when a video conflict occurs between the video picture corresponding to the next time period and the non-repetitive video, and performing fixed instruction execution on the fixed control instruction in the flexible camera to obtain a fixed instruction execution result;

[0081] The second collaborative module is used to send flexible control instructions from the router to the flexible camera through the instruction sending channel when the corresponding video screen in the next time period does not cause video conflict with the non-repetitive video, and flexibly execute the flexible control instructions in the flexible camera to obtain a flexible instruction execution result, and use the fixed instruction execution result and the flexible instruction execution result as the second collaborative work result between the flexible camera and the router.

[0082] Compared with the prior art, the technical principle and beneficial effects of this solution are:

[0083] The embodiment of the present invention establishes a data transmission channel and an instruction issuing channel between the multiple cameras and the router to realize one-way transmission of video from the camera to the router and one-way issuing of instructions from the router to the camera, and encrypts the video during video transmission to prevent leakage of video data during transmission. The embodiment of the present invention marks the non-repetitive videos in the fixed video through the data processing program in the router to retain the images containing rich image data. When the image data taken by the flexible camera is subsequently detected, the marked non-repetitive videos containing rich information can be retrieved for data comparison, and there is no need to retrieve data containing a large number of repeated videos, thereby reducing the amount of data retrieval and reducing the data processing pressure of the router. Furthermore, the embodiment of the present invention uses the data processing program in the router to The data processing program performs target detection on the flexible video to obtain a detection target, which is used to identify whether there is a target set by the user that needs to be focused on in front of the camera, so as to judge whether to track the focus target set by the user in the future. Furthermore, the embodiment of the present invention compares whether the video screen corresponding to the next time period turns to whether there is a video conflict with the non-repetitive video, so as to judge whether the video screen corresponding to the next time period turns to is the same as the one shot in the non-repetitive video. If the shooting content is consistent, the camera corresponding to the next time period does not need to turn, so as to save the power consumption of the camera turning. The embodiment of the present invention flexibly executes the flexible control instructions in the flexible camera, so as to enable the router to send control instructions to the camera instead of the sending end. Therefore, the embodiment of the present invention proposes a method and system for the coordinated work of a camera and a router for a smart home, which enables the router to send control instructions to the camera instead of the sending end. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0085] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0086] Figure 1 A flowchart of a method for cooperating between a camera and a router for a smart home provided by an embodiment of the present invention;

[0087] Figure 2 A module diagram of a camera and router collaborative working system for a smart home provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0088] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0089] The embodiment of the present invention provides a method for the coordinated working of a camera and a router for a smart home, and the execution subject of the method for the coordinated working of a camera and a router for a smart home includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiment of the present invention. In other words, the method for the coordinated working of a camera and a router for a smart home can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0090] See also Figure 1 FIG. 1 is a flow chart of a method for cooperating between a camera and a router for a smart home provided by an embodiment of the present invention. Figure 1 The method for cooperating between cameras and routers for smart homes described in the specification includes:

[0091] S1. After wirelessly connecting multiple cameras to a router, establish data transmission channels and command issuing channels between the multiple cameras and the router, and set fixed steering modes and flexible steering modes of the multiple cameras, wherein the router includes a data processing program and a data transmission function.

[0092] In the embodiment of the present invention, the multiple cameras refer to cameras at multiple angles set in a home scene.

[0093] Furthermore, the embodiment of the present invention establishes a data transmission channel and a command issuing channel between the multiple cameras and the router to realize unidirectional transmission of video from the camera to the router and unidirectional issuing of commands from the router to the camera, and encrypts the video during transmission to prevent leakage of video data during the transmission process.

[0094] In one embodiment of the present invention, the establishment of a data transmission channel and an instruction issuing channel between the multiple cameras and the router includes: respectively constructing a video sending frequency when the multiple cameras send videos and an instruction sending frequency when the router sends instructions; keeping the video sending frequency and the instruction sending frequency the same to obtain the same video sending frequency and the same instruction sending frequency; using the same video sending frequency to set a first simplex communication channel in the direction from the multiple cameras to the router; determining a video encryption protocol for the first simplex communication channel; using the first simplex communication channel containing the video encryption protocol as a data transmission channel; using the same instruction sending frequency to set a second simplex communication channel in the direction from the router to the multiple cameras; and using the second simplex communication channel as the instruction issuing channel between the multiple cameras and the router.

[0095] Among them, the video sending frequency and the instruction sending frequency refer to frequencies set during simplex communication that are different from other signals. By setting the sizes of different frequencies and letting the receiving device check the frequency sizes, it is decided which frequencies of data the receiving device should receive. The first simplex communication channel and the second simplex communication channel refer to two simplex communication channels in opposite directions. The video encryption protocol refers to a data encryption protocol jointly negotiated by the receiving end and the sending end. The receiving end has a key for data decryption, and the sending end has a key for data encryption. After the sending end of the first simplex communication channel encrypts the data through the video encryption protocol, the encrypted data is transmitted on the first simplex communication channel.

[0096] Furthermore, in an embodiment of the present invention, the fixed steering mode and the flexible steering mode of the multiple cameras are set by the users of the cameras, that is, when the user requires the camera to rotate freely for shooting, the flexible steering mode is set, and when the user requires the camera to be fixed and not turned, the fixed steering mode of the camera is set.

[0097] It should be noted that the data processing program refers to a code program including various mathematical algorithms and mathematical formulas. These code programs can perform operations and processing on image data. The data processing program includes codes for similarity calculation, numerical value comparison, fixed numerical value calculation, binary threshold algorithm, frame difference method, etc. Furthermore, the data transmission function refers to the data transmission function of the router before it is modified.

[0098] S2. Obtain a fixed camera in the fixed steering mode among the multiple cameras, transmit the fixed video collected by the fixed camera to the router through the data transmission channel, mark the non-repetitive videos in the fixed video through the data processing program in the router, and transmit the non-repetitive videos and the repetitive videos in the fixed video to a storage device through the data transmission function in the router to complete the collaborative work between the fixed camera and the router, and obtain the collaborative work result between the fixed camera and the router.

[0099] The embodiment of the present invention marks the non-repetitive videos in the fixed video through the data processing program in the router to retain the images containing rich image data. When the image data taken by the flexible camera is subsequently detected, the marked non-repetitive videos containing rich information can be retrieved for data comparison, and there is no need to retrieve data containing a large number of repeated videos, thereby reducing the amount of data retrieval and reducing the data processing pressure of the router.

[0100] It should be noted that non-repetitive video refers to a set of video frames in which each video frame is different. Since there is no external force or dynamic objects in the camera monitoring scene, most of the monitored scenes are in a fixed state. Therefore, the monitoring videos collected every moment will generate a large number of videos of repetitive scenes.

[0101] In one embodiment of the present invention, the non-repetitive videos in the fixed videos are marked by the data processing program in the router, including: the data processing program includes: sorting the fixed videos in ascending order of the video moments of the fixed videos to obtain sorted video frames; starting from the starting video frame in the sorted video frames, sequentially calculating the video frame similarity between the starting video frame and the subsequent video frames of the starting video frame; when the video frame similarity is not greater than the preset similarity, stopping the calculation of the video frame similarity between the starting video frame and the subsequent video frames of the starting video frame; obtaining the target subsequent video frame whose video frame similarity is greater than the preset similarity; and calculating the gray value distribution distance between the starting video frame and the target subsequent video frame using the following formula:

[0102] ;

[0103] in, Indicates The gray value distribution distance in the video frame, Indicates the first number between the starting video frame and the target subsequent video frame. In the video frame The size of The gray value is closest to the The distance between gray values ​​of size, Indicates the number of categories of grayscale value size, Indicates The number of gray values ​​of size;

[0104] The maximum distribution distance is selected from the gray value distribution distances; the video frame corresponding to the maximum distribution distance in the starting video frame and the target subsequent video frame is used as the initial non-repetitive video, and the remaining non-repetitive videos in the sorted video frames are screened in the same manner as the initial non-repetitive video, and the initial non-repetitive video and the remaining non-repetitive videos are used as the non-repetitive videos.

[0105] The subsequent video frames refer to all video frames arranged after the starting video frame.

[0106] Exemplarily, the process of sequentially calculating the video frame similarities between the starting video frame and subsequent video frames of the starting video frame includes, for example: calculating the similarity between the first video frame and the second video frame, calculating the similarity between the first video frame and the third video frame, and calculating the similarity between the first video frame and the fourth video frame; the process of obtaining target subsequent video frames whose video frame similarities are greater than the preset similarity includes, for example, taking the second video frame and the third video frame as target subsequent video frames when the similarity between the first video frame and the fourth video frame is not greater than the preset similarity.

[0107] It should be noted that the denser the distribution of grayscale values ​​of the same size in the image, the more these grayscale values ​​are in the same object part. Therefore, the farther the distribution distance of grayscale values ​​of the same size is, the more these grayscale values ​​are in different object parts. It can be seen that the richer the different object parts that appear, the richer the information contained in the image, and the image containing rich information is retained. Furthermore, the process of screening the remaining non-repetitive videos in the sorted video frames in the manner of screening the initial non-repetitive videos is, for example: calculating the similarity between the 4th video frame and the 5th video frame, calculating the similarity between the 4th video frame and the 6th video frame, calculating the similarity between the 4th video frame and the 6th video frame, until an image with a similarity greater than a preset similarity is encountered.

[0108] Furthermore, in an embodiment of the present invention, the storage device refers to a cloud or mobile device, etc., which is used to store data received by the router.

[0109] S3. Obtain a flexible camera in the flexible steering mode from among the multiple cameras, transmit the flexible video captured by the flexible camera to the router, and transmit the steering range and current steering of the flexible camera to the router respectively, perform target detection on the flexible video through the data processing program in the router to obtain the detected target, transmit the flexible video to the storage device to complete the first collaborative work between the flexible camera and the router, obtain the first collaborative work result between the flexible camera and the router, and determine whether there is a target of special concern among the detected targets.

[0110] In the embodiment of the present invention, the steering range refers to the maximum range in which the flexible camera can rotate at various angles. The current steering is, for example, a direction of 30 degrees relative to the x-axis in the coordinate system of the flexible camera.

[0111] Furthermore, the embodiment of the present invention performs target detection on the flexible video through the data processing program in the router to obtain a detected target for identifying whether there is a target set by the user that needs to be focused on in front of the camera, so as to subsequently determine whether to track the target set by the user.

[0112] In one embodiment of the present invention, the target detection is performed on the flexible video by the data processing program in the router to obtain the detected target, including: the data processing program includes: using a preset binary threshold algorithm to detect the target area in the flexible video; judging whether the user of the router sets a special attention target in the router; when the user of the router sets the special attention target in the router, calculating the target similarity between the target area and the special attention target; when the target similarity is greater than the preset similarity, taking the target area as the detection target; when the target similarity is not greater than the preset similarity, judging whether the special attention target is a dynamic target; when the special attention target is a dynamic target, using a preset inter-frame difference method to detect the dynamic target area in the target area; detecting the neighborhood target area of ​​the dynamic target area; using the neighborhood target area to identify the regional orientation of the dynamic target area; calculating the orientation-camera distance between the regional orientation and the flexible camera; according to the orientation-camera distance, calculating the true height of the dynamic target area using the following formula:

[0113] ;

[0114] ;

[0115] in, Indicates the true height, Indicates the target size height of the flexible camera. Indicates the position-camera distance, Indicates the focal length of the flexible camera;

[0116] By comparing the true height with the height of the dynamic target, it is determined whether the dynamic target area is the dynamic target; when the dynamic target area is the dynamic target, the dynamic target area is used as the detection target; when the special focus target is not a dynamic target, the target orientation of the target area is identified using the neighborhood area of ​​the target area; by comparing the target orientation with the orientation of the special focus target, it is determined whether the target area is the special focus target; when the target area is the special focus target, the target area is used as the detection target.

[0117] Among them, the binarization threshold algorithm refers to an algorithm that uses a preset threshold to distinguish which areas are objects after the image is binarized. The target area refers to an image containing a complete object or creature, such as an image area of ​​a dog. The special focus target refers to the image data of the target that the user needs to track and monitor with the camera, such as a baby in a furniture scene. The preset similarity is similar to the preset similarity set when the video frame similarity is not greater than the preset similarity, and can be set to a value such as 0.90. The neighborhood target area refers to the images of other objects around the dynamic target area.

[0118] Optionally, the process of using the neighborhood target area to identify the regional orientation of the dynamic target area means that when the distribution of other surrounding objects is known, the position of the dynamic target area can be determined. For example, if the surrounding objects are images such as a television and a sofa, then the regional orientation is the position between the television and the sofa.

[0119] It should be noted that when the user of the router does not set a special target of interest in the router, the target area is used as the detection target, and when no special target of interest is set in the router, it means that there is no need to control the camera to track the target, that is, keep the camera angle fixed.

[0120] S4. When the special attention target exists in the detection targets, the target movement of the special attention target is calculated by using the data processing program, and the next time period steering of the flexible camera is set by using the target movement based on the steering range and the current steering. The non-repetitive video in the storage device is retrieved in the router to compare whether the video picture corresponding to the next time period steering has a video conflict with the non-repetitive video.

[0121] In one embodiment of the present invention, the use of the data processing program to calculate the target movement of the special attention target includes: the data processing program includes: calculating the target center point of the special attention target in each video frame of the flexible video; calculating the coordinate point vector of the target center point between every two consecutive video frames of the flexible video in order of video moments of the flexible video from small to large; extracting the target movement of the special attention target from the coordinate point vector.

[0122] The coordinate point vector includes distance and direction information between the coordinate points.

[0123] In one embodiment of the present invention, the next time period steering of the flexible camera is set based on the steering range and the current steering and using the target movement, including: calculating the coordinate system mapping relationship between the coordinate system corresponding to the target movement and the coordinate system corresponding to the current steering; mapping the target movement to the coordinate system corresponding to the current steering using the coordinate system mapping relationship to obtain a mapped movement; judging whether the target movement of the flexible camera based on the current steering exceeds the steering range; when the target movement of the flexible camera based on the current steering exceeds the steering range, taking the camera movement that does not exceed the steering range as the next time period steering; when the target movement of the flexible camera based on the current steering does not exceed the steering range, taking the target movement of the flexible camera based on the current steering as the next time period steering.

[0124] Optionally, the process of calculating the coordinate system mapping relationship between the coordinate system corresponding to the target movement and the coordinate system corresponding to the current turn is, for example: dividing the coordinate system matrix corresponding to the target movement by the coordinate system matrix corresponding to the current turn.

[0125] Optionally, before comparing whether the video picture corresponding to the next time period turns to or not has a video conflict with the non-repetitive video, the process also includes: retrieving panoramic image data of the flexible camera from a storage device; and querying the video picture corresponding to the next time period turns to from the panoramic image data.

[0126] Furthermore, the embodiment of the present invention compares whether the video picture corresponding to the next time period is the same as the one shot in the non-repetitive video by comparing whether there is a video conflict with the non-repetitive video. If the shooting content is consistent, the camera corresponding to the next time period does not need to be turned, thereby saving the power consumption of the camera turning.

[0127] In one embodiment of the present invention, the comparing whether the video screen corresponding to the next time period turns to has a video conflict with the non-repetitive video includes: performing similarity matching on the video screen and the non-repetitive video; when the similarity matching on the video screen and the non-repetitive video is successful, determining that the video screen corresponding to the next time period turns to has a video conflict with the non-repetitive video; when the similarity matching on the video screen and the non-repetitive video fails, determining that the video screen corresponding to the next time period turns to has no video conflict with the non-repetitive video.

[0128] S5. When a video conflict occurs between the video screen corresponding to the next time period and the non-repetitive video, a fixed control instruction is sent from the router to the flexible camera through the instruction sending channel, and the fixed control instruction is executed in the flexible camera to obtain a fixed instruction execution result.

[0129] In one embodiment of the present invention, the fixed control instruction is executed in the flexible camera to obtain a fixed instruction execution result, including: according to the fixed control instruction, after keeping the flexible camera fixed, collecting images taken by the flexible camera during the fixed period; returning the images taken during the fixed period to the router, so as to determine the fixed instruction execution result of the flexible camera through the router.

[0130] S6. When the corresponding video screen in the next time period does not conflict with the non-repetitive video, a flexible control instruction is sent from the router to the flexible camera through the instruction sending channel, and the flexible control instruction is flexibly executed in the flexible camera to obtain a flexible instruction execution result, and the fixed instruction execution result and the flexible instruction execution result are used as the second collaborative work result between the flexible camera and the router.

[0131] The embodiment of the present invention performs flexible instruction execution on the flexible control instruction in the flexible camera, so as to enable the router to send the control instruction to the camera instead of the sending end.

[0132] In one embodiment of the present invention, the flexible control instruction is flexibly executed in the flexible camera to obtain a flexible instruction execution result, including: steering the flexible camera according to the flexible control instruction; during the steering process of the flexible camera, temporarily storing the image captured by the flexible camera in the local memory of the flexible camera to obtain the image stored in the local memory; when the steering process of the flexible camera is completed, returning the image stored in the local memory to the router to determine the flexible instruction execution result of the flexible camera through the router.

[0133] It can be seen that the embodiment of the present invention establishes a data transmission channel and an instruction issuing channel between the multiple cameras and the router to realize the one-way transmission of video from the camera to the router and the one-way issuing of instructions from the router to the camera, and encrypts the video during video transmission to prevent leakage of video data during transmission. The embodiment of the present invention marks the non-repetitive videos in the fixed video through the data processing program in the router to retain the images containing rich image data. When the image data taken by the flexible camera is subsequently detected, the marked non-repetitive videos containing rich information can be retrieved for data comparison, and there is no need to retrieve data containing a large number of repeated videos, thereby reducing the amount of data retrieval and reducing the data processing pressure of the router. Furthermore, the embodiment of the present invention uses the router The data processing program in the device performs target detection on the flexible video to obtain a detection target, which is used to identify whether there is a target set by the user that needs to be focused on in front of the camera, so as to judge whether to track the focus target set by the user in the future. Furthermore, the embodiment of the present invention compares whether the video screen corresponding to the next time period turns to whether there is a video conflict with the non-repetitive video, so as to judge whether the video screen corresponding to the next time period turns to is the same as the one shot in the non-repetitive video. If the shooting content is consistent, the camera corresponding to the next time period does not need to turn, so that the power consumption of the camera turning can be saved. The embodiment of the present invention flexibly executes the flexible control instructions in the flexible camera, so as to enable the router to send control instructions to the camera instead of the sending end. Therefore, the embodiment of the present invention proposes a method and system for the coordinated work of a camera and a router for a smart home, which enables the router to send control instructions to the camera instead of the sending end.

[0134] like Figure 2 As shown, it is a functional module diagram of the camera and router collaborative working system for smart home of the present invention.

[0135] The camera and router collaborative working system 200 for smart home of the present invention can be installed in an electronic device. According to the functions to be implemented, the camera and router collaborative working system for smart home can include a mode setting module 201, a fixed collaborative module 202, a first collaborative module 203, a conflict comparison module 204, an instruction execution module 205 and a second collaborative module 206. The module of the present invention can also be called a unit, which refers to a series of computer program segments that can be executed by a processor of an electronic device and can complete fixed functions, which are stored in the memory of the electronic device.

[0136] In the embodiment of the present invention, the functions of each module / unit are as follows:

[0137] The mode setting module 201 is used to establish a data transmission channel and a command issuing channel between the multiple cameras and the router after the multiple cameras are wirelessly connected to the router, and set the fixed steering mode and the flexible steering mode of the multiple cameras, wherein the router includes a data processing program and a data transmission function;

[0138] The fixed collaboration module 202 is used to obtain a fixed camera in the fixed steering mode among the multiple cameras, transmit the fixed video collected by the fixed camera to the router through the data transmission channel, mark the non-repetitive video in the fixed video through the data processing program in the router, and transmit the non-repetitive video and the repetitive video in the fixed video to a storage device through the data transmission function in the router to complete the collaborative work between the fixed camera and the router, and obtain the collaborative work result between the fixed camera and the router;

[0139] The first collaborative module 203 is used to obtain a flexible camera in the flexible steering mode among the multiple cameras, transmit the flexible video collected by the flexible camera to the router, and transmit the steering range and current steering of the flexible camera to the router respectively, perform target detection on the flexible video through the data processing program in the router to obtain the detected target, transmit the flexible video to the storage device to complete the first collaborative work between the flexible camera and the router, obtain the first collaborative work result between the flexible camera and the router, and determine whether there is a special focus target among the detected targets;

[0140] The conflict comparison module 204 is used to calculate the target movement of the target of special concern by using the data processing program when the target of special concern exists in the detection target, set the next time period steering of the flexible camera by using the target movement based on the steering range and the current steering, and retrieve the non-repetitive video in the storage device in the router to compare whether the video picture corresponding to the next time period steering has a video conflict with the non-repetitive video;

[0141] The instruction execution module 205 is used to send a fixed control instruction from the router to the flexible camera through the instruction sending channel when a video conflict occurs between the video picture corresponding to the next time period and the non-repetitive video, and to execute the fixed control instruction in the flexible camera to obtain a fixed instruction execution result;

[0142] The second collaborative module 206 is used to send flexible control instructions from the router to the flexible camera through the instruction sending channel when the corresponding video screen in the next time period does not cause video conflict with the non-repetitive video, and flexibly execute the flexible control instructions in the flexible camera to obtain a flexible instruction execution result, and use the fixed instruction execution result and the flexible instruction execution result as the second collaborative work result between the flexible camera and the router.

[0143] In detail, the modules in the smart home camera and router collaborative working system 200 described in the embodiment of the present invention are used in the same manner as described above. Figure 1 The technical means used are the same as the method of collaborative working between cameras and routers for smart homes described in , and can produce the same technical effects, so they will not be repeated here.

[0144] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0145] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0146] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0147] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0148] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A method for cooperating between a camera and a router for smart home, characterized in that: The method comprises: After the plurality of cameras are wirelessly connected to the router, a data transmission channel and a command issuing channel are established between the plurality of cameras and the router, and a fixed steering mode and a flexible steering mode of the plurality of cameras are set, wherein the router includes a data processing program and a data transmission function; Obtain a fixed camera in the fixed steering mode among the multiple cameras, transmit the fixed video collected by the fixed camera to the router through the data transmission channel, mark the non-repetitive videos in the fixed video through the data processing program in the router, and transmit the non-repetitive videos and the repetitive videos in the fixed video to a storage device through the data transmission function in the router to complete the collaborative work between the fixed camera and the router, and obtain the collaborative work result between the fixed camera and the router; Acquire a flexible camera in the flexible steering mode among the multiple cameras, transmit the flexible video captured by the flexible camera to the router, and transmit the steering range and current steering of the flexible camera to the router respectively, perform target detection on the flexible video through a data processing program in the router to obtain a detected target, transmit the flexible video to the storage device to complete the first collaborative work between the flexible camera and the router, obtain the first collaborative work result between the flexible camera and the router, and judge whether there is a target of special concern among the detected targets; When the special attention target exists in the detection target, the target movement of the special attention target is calculated by using the data processing program, and the next time period steering of the flexible camera is set by using the target movement based on the steering range and the current steering, and the non-repetitive video in the storage device is retrieved in the router to compare whether the video picture corresponding to the next time period steering has a video conflict with the non-repetitive video; When a video conflict occurs between the video screen corresponding to the next time period and the non-repetitive video, a fixed control instruction is sent from the router to the flexible camera through the instruction sending channel, and the fixed control instruction is executed in the flexible camera to obtain a fixed instruction execution result; When the corresponding video screen in the next time period does not conflict with the non-repetitive video, a flexible control instruction is sent from the router to the flexible camera through the instruction sending channel, and the flexible control instruction is flexibly executed in the flexible camera to obtain a flexible instruction execution result, and the fixed instruction execution result and the flexible instruction execution result are used as the second collaborative work result between the flexible camera and the router.

2. The method according to claim 1, characterized in that The establishing of a data transmission channel and a command issuing channel between the plurality of cameras and the router comprises: Respectively constructing video sending frequencies when the multiple cameras send videos and command sending frequencies when the router sends commands; Keeping the video sending frequency and the command sending frequency the same, so as to obtain the same video sending frequency and the same command sending frequency; Using the same video transmission frequency to set a first simplex communication channel in the direction from the plurality of cameras to the router; determining a video encryption protocol for the first simplex communication channel; Using the first simplex communication channel including the video encryption protocol as a data transmission channel; Using the same instruction sending frequency to set a second simplex communication channel in the direction from the router to the multiple cameras; The second simplex communication channel is used as a command issuing channel between the multiple cameras and the router.

3. The method according to claim 1, characterized in that The step of marking the non-repetitive videos in the fixed videos by a data processing program in the router includes: The data processing program includes: Sorting the fixed videos in ascending order of video time to obtain sorted video frames; Starting from a starting video frame in the sorted video frames, sequentially calculating video frame similarities between the starting video frame and subsequent video frames of the starting video frame; When the video frame similarity is not greater than a preset similarity, stop calculating the video frame similarity between the starting video frame and subsequent video frames of the starting video frame; Obtaining a target subsequent video frame whose similarity among the video frames is greater than the preset similarity; The gray value distribution distance between the starting video frame and the target subsequent video frame is calculated using the following formula: ； in, Indicates The gray value distribution distance in the video frame, Indicates the first number between the starting video frame and the target subsequent video frame. In the video frame The size of The gray value is closest to the The distance between gray values ​​of size, Indicates the number of categories of grayscale value size, Indicates The number of gray values ​​of size; Selecting the maximum distribution distance from the gray value distribution distances; The video frame corresponding to the maximum distribution distance between the starting video frame and the target subsequent video frame is taken as the initial non-repetitive video, and the remaining non-repetitive videos in the sorted video frames are screened in the same manner as the initial non-repetitive video, and the initial non-repetitive video and the remaining non-repetitive videos are taken as the non-repetitive videos.

4. The method according to claim 1, characterized in that: The step of performing target detection on the flexible video through a data processing program in the router to obtain a detection target includes: The data processing program includes: Detecting the target area in the flexible video using a preset binarization threshold algorithm; Determining whether a user of the router sets a special attention target in the router; When a user of the router sets a special attention target in the router, calculating a target similarity between the target area and the special attention target; When the target similarity is greater than a preset similarity, taking the target area as the detection target; When the target similarity is not greater than the preset similarity, determining whether the target of special interest is a dynamic target; When the target of special concern is a dynamic target, detecting the dynamic target area in the target area by using a preset inter-frame difference method; Detecting a neighborhood target area of ​​the dynamic target area; Using the neighborhood target area to identify the area orientation of the dynamic target area; Calculating the position-camera distance between the region position and the flexible camera; According to the position-camera distance, the true height of the dynamic target area is calculated using the following formula: ； ； in, Indicates the true height, Indicates the target size height of the flexible camera. Indicates the position-camera distance, Indicates the focal length of the flexible camera; By comparing the real height with the height of the dynamic target, determining whether the dynamic target area is the dynamic target; When the dynamic target area is the dynamic target, taking the dynamic target area as the detection target; When the special attention target is not a dynamic target, identifying the target orientation of the target area by using the neighborhood area of ​​the target area; By comparing the target orientation with the orientation of the target of special concern, determining whether the target area is the target of special concern; When the target area is the special attention target, the target area is used as the detection target.

5. The method according to claim 1, characterized in that The step of calculating the target movement of the target of special interest by using the data processing program includes: The data processing program includes: Calculate the target center point of the special focus target in each video frame of the flexible video; According to the video time sequence of the flexible video from small to large, the coordinate point vector of the target center point between every two consecutive video frames of the flexible video is calculated; The target movement of the target of special interest is extracted from the coordinate point vector.

6. The method according to claim 1, characterized in that The step of setting the next time period steering of the flexible camera based on the steering range and the current steering and using the target movement includes: Calculating a coordinate system mapping relationship between a coordinate system corresponding to the target movement and a coordinate system corresponding to the current turn; Mapping the target movement to the coordinate system corresponding to the current turn using the coordinate system mapping relationship to obtain a mapped movement; Determining whether the target movement of the flexible camera exceeds the steering range based on the current steering; When the target movement of the flexible camera exceeds the steering range based on the current steering, the camera movement that does not exceed the steering range is used as the steering for the next time period; When the target movement of the flexible camera based on the current turn does not exceed the turn range, the target movement of the flexible camera based on the current turn will be used as the turn for the next time period.

7. The method according to claim 1, characterized in that The comparing whether the video picture corresponding to the next time period turns to has a video conflict with the non-repetitive video includes: Performing similarity matching between the video frame and the non-repetitive video; When similarity matching between the video picture and the non-repetitive video is successfully performed, it is determined that a video conflict occurs between the video picture corresponding to the next time period and the non-repetitive video; When similarity matching between the video frame and the non-repetitive video fails, it is determined that no video conflict occurs between the video frame corresponding to the next time period and the non-repetitive video.

8. The method according to claim 1, characterized in that The step of executing the fixed control instruction in the flexible camera to obtain a fixed instruction execution result includes: According to the fixing control instruction, after keeping the flexible camera fixed, collect images taken by the flexible camera during the fixing period; The images captured during the fixed period are returned to the router so as to determine the execution result of the fixed instruction of the flexible camera through the router.

9. The method according to claim 1, characterized in that: The step of performing flexible instruction execution on the flexible control instruction in the flexible camera to obtain a flexible instruction execution result includes: Performing steering processing on the flexible camera according to the flexible control instruction; During the steering process of the flexible camera, the image captured by the flexible camera is temporarily stored in the local memory of the flexible camera to obtain the image stored in the local memory; When the steering process of the flexible camera is completed, the image stored in the local memory is returned to the router so that the flexible instruction execution result of the flexible camera can be determined through the router.

10. A camera and router collaborative working system for smart home, characterized in that: The system comprises: A mode setting module is used to establish a data transmission channel and a command issuing channel between the multiple cameras and the router after the multiple cameras are wirelessly connected to the router, and to set a fixed steering mode and a flexible steering mode for the multiple cameras, wherein the router includes a data processing program and a data transmission function; A fixed collaboration module, used for acquiring a fixed camera in the fixed steering mode among the multiple cameras, transmitting the fixed video collected by the fixed camera to the router through the data transmission channel, marking non-repetitive videos in the fixed video through the data processing program in the router, and transmitting the non-repetitive videos and the repetitive videos in the fixed video to a storage device through the data transmission function in the router, so as to complete the collaborative work between the fixed camera and the router and obtain the collaborative work result between the fixed camera and the router; A first collaborative module is used to obtain a flexible camera in the flexible steering mode among the multiple cameras, transmit the flexible video collected by the flexible camera to the router, and transmit the steering range and current steering of the flexible camera to the router respectively, perform target detection on the flexible video through a data processing program in the router to obtain a detected target, transmit the flexible video to the storage device to complete a first collaborative work between the flexible camera and the router, obtain a first collaborative work result between the flexible camera and the router, and determine whether there is a target of special concern among the detected targets; a conflict comparison module, configured to, when the special attention target exists in the detection target, calculate the target movement of the special attention target by using the data processing program, set the next time period steering of the flexible camera by using the target movement based on the steering range and the current steering, and retrieve the non-repetitive video in the storage device in the router to compare whether the video picture corresponding to the next time period steering has a video conflict with the non-repetitive video; An instruction execution module is used for sending a fixed control instruction from the router to the flexible camera through the instruction sending channel when a video conflict occurs between the video picture corresponding to the next time period and the non-repetitive video, and performing fixed instruction execution on the fixed control instruction in the flexible camera to obtain a fixed instruction execution result; The second collaborative module is used to send flexible control instructions from the router to the flexible camera through the instruction sending channel when the corresponding video screen in the next time period does not cause video conflict with the non-repetitive video, and flexibly execute the flexible control instructions in the flexible camera to obtain a flexible instruction execution result, and use the fixed instruction execution result and the flexible instruction execution result as the second collaborative work result between the flexible camera and the router.

Images