Method, system, device and storage medium for monitoring working status of camera equipment

Abstract

The embodiments of this specification provide a method and system, device, and storage medium for monitoring the working status of a camera device, which are executed by a camera device monitoring unit. The method includes obtaining the data flow of the camera device; judging whether there is a suspected camera device based on changes in the data flow, where the suspected camera device is a camera device that may have a fault; and in response to the existence of a suspected camera device, performing a fault check on the suspected camera device.

Description

Technical Field

[0001] The present invention relates to the field of monitoring equipment, and in particular to a method and system, device, and storage medium for monitoring the working status of a camera device. Background Art

[0002] The use of video equipment has become popular. In some occasions, a large number of video equipment are often deployed, such as on roads, in airports, shopping malls and other locations. However, the working status of the video equipment may be abnormal or malfunction, and manual on-site inspection is time-consuming and labor-intensive. Especially when the number of devices deployed is large and the coverage area is large, the problem of equipment maintenance will become a problem, and it is a great trouble for manual maintenance.

[0003] Therefore, there is a need to provide a method and system for monitoring the working status of camera equipment, which can help to promptly discover camera equipment that may fail, reduce the consumption of manpower, material resources and time caused by manual inspection, and improve the efficiency of equipment maintenance. Summary of the invention

[0004] One of the embodiments of the present specification provides a method for monitoring the working status of a camera device, which is executed by a camera device monitoring unit. The method for monitoring the working status of a camera device includes: obtaining the data flow of the camera device; judging whether there is a suspected camera device based on changes in the data flow, wherein the suspected camera device is a camera device that may have a fault; and in response to the existence of the suspected camera device, performing a fault check on the suspected camera device.

[0005] One of the embodiments of the present specification provides a system for monitoring the working status of a camera device, the system comprising: an acquisition module for acquiring data flow of the camera device; a judgment module for judging whether there is a suspected camera device based on changes in the data flow, the suspected camera device being a camera device that may have a fault; and an inspection module for performing a fault inspection on the suspected camera device.

[0006] One of the embodiments of the present specification provides a device for monitoring the working status of a camera device, including a processor, wherein the processor is used to execute the method for monitoring the working status of a camera device.

[0007] One of the embodiments of the present specification provides a computer-readable storage medium, wherein the storage medium stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the above-mentioned method for monitoring the working status of a camera device. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] This specification will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0009] Figure 1 is a schematic diagram of an application scenario of a system for monitoring the working status of a camera device according to some embodiments of this specification;

[0010] Figure 2 is an exemplary schematic diagram of a system for monitoring the working status of a camera device according to some embodiments of this specification;

[0011] Figure 3 is an exemplary flow chart of a method for monitoring the working status of a camera device according to some embodiments of this specification;

[0012] Figure 4 is an exemplary schematic diagram of a prediction model according to some embodiments of this specification;

[0013] Figure 5 is an exemplary schematic diagram of a camera device fault check according to some embodiments of the present specification;

[0014] Figure 6 It is an exemplary schematic diagram of the model structure of the discriminant model shown in some embodiments of this specification. DETAILED DESCRIPTION

[0015] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of this specification. For ordinary technicians in this field, this specification can also be applied to other similar scenarios based on these drawings without creative work. Unless it is obvious from the language environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.

[0016] It should be understood that the "system", "device", "unit" and / or "module" used herein are a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0017] As shown in this specification and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "comprise" and "include" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0018] Flowcharts are used in this specification to illustrate the operations performed by the system according to the embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed precisely in order. Instead, the steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or one or more operations may be removed from these processes.

[0019] Figure 1 It is a schematic diagram of an application scenario of a monitoring camera device working status system according to some embodiments of this specification.

[0020] like Figure 1 As shown, the application scenario 100 of the system for monitoring the working status of a camera device may include a camera device 110, a server 120, a flow of people 130, and a road 140. In some embodiments, the components in the application scenario 100 may be connected and / or communicate with each other via a network (e.g., a wireless connection, a wired connection, or a combination thereof).

[0021] In some embodiments, the camera device 110 can be used to collect data and / or information. For example, the camera device 110 can be used to collect images, videos, etc. The camera device 110 can be a camera device such as a data camera, a camera head, etc. In some embodiments, the camera device 110 can include multiple camera devices, for example, a camera device 111, a camera device 112, a camera device 113, and a camera device 114. In some embodiments, the camera device 110 can be deployed on the side of a road 140. In some embodiments, the camera device 110 can send the collected data and / or information to the server 120. For example, the image and video data collected by the camera device 110. In some embodiments, the camera device 110 can send the collected data and / or information to the server 120 in various ways, for example, a wired network or a wireless network.

[0022] In some embodiments, the server 120 is used to process information and / or data related to the application scenario 100. For example, images, video data, etc. collected by the camera device 110. In some embodiments, the server 120 may include one or more processing engines (for example, a single-chip processing engine or a multi-chip processing engine). As an example only, the server 120 may include a central processing unit (CPU). The server 120 can process data, information and / or processing results obtained from other devices or system components, and execute program instructions based on these data, information and / or processing results to perform one or more functions described in this specification. For example, a suspect device is determined based on the data traffic of multiple camera devices.

[0023] In some embodiments, the flow of people 130 can be used to represent the data of the flow of people on the road 140. The flow of people 130 includes data such as the number of pedestrians on the road 140 and the direction of walking. In some embodiments, the camera device 110 can obtain image data based on the flow of people 130, and then determine the corresponding data flow. For example, based on the flow of people 130 at multiple adjacent time points, the data flow changes of the corresponding camera devices at multiple time points are obtained. In some embodiments, the flow of people 130 can also be extended to the flow data of other movable objects. For example, cars, movable robots, etc.

[0024] In some embodiments, the road 140 can be used to deploy the camera device 110. The road 140 can include various types of roads such as one-way streets, two-way streets, bifurcated roads, and cross roads; the road 140 can be a street, or a road in a shopping mall, a park, or a community. In some embodiments, multiple camera devices such as the camera device 111, the camera device 112, and the camera device 113 that are close to each other can be deployed on the road 140 to collect video data related to the flow of people 130 at multiple time points.

[0025] In some embodiments, the server 120 may process the images and / or video data related to the flow of people 130 acquired by the camera device 110 deployed on the side of the road 140 to determine one or more camera devices among the camera devices 110 that may have a fault. For example, the camera devices 111, 112, 113, and 114 deployed on the side of the road 140 acquire video data of multiple time points within a period of time based on the flow of people 130 on the road 140, and send the data to the server 120. The server 120 may determine the suspected camera devices among the camera devices 111, 112, 113, and 114 based on the data flow differences. For example, if the data flow changes of the camera devices 111, 112, and 114 are within the normal range, while the data flow changes of the camera device 113 are not within the normal range, then the camera device 113 is a camera device that may have a fault. For more information on determining suspected camera devices, please refer to other parts of this manual, such as Figure 3 wait.

[0026] It should be noted that the application scenarios are provided only for the purpose of illustration and are not intended to limit the scope of this specification. For those of ordinary skill in the art, various modifications or variations can be made according to the description of this specification. For example, the application scenarios may also include a database. For another example, the application scenarios may be implemented on other devices to achieve similar or different functions. However, the changes and modifications will not deviate from the scope of this specification.

[0027] Figure 2 It is an exemplary schematic diagram of a monitoring camera device working status system according to some embodiments of this specification.

[0028] like Figure 2 As shown, the system 200 for monitoring the working state of a camera device may include an acquisition module 210, a determination module 220, and an inspection module 230. In some embodiments, the system 200 for monitoring the working state of a camera device may be implemented by a camera device monitoring unit.

[0029] The acquisition module 210 is used to acquire the data flow of the camera device.

[0030] For more details on obtaining the data flow of the camera device, see Figure 3 and related descriptions.

[0031] The judgment module 220 is used to judge whether there is a suspicious camera device based on the change of data flow, and the suspicious camera device is a camera device that may have a fault.

[0032] In some embodiments, the judgment module 220 can also be used to determine the predicted flow rate of each camera device through a prediction model based on the data flow rate of multiple camera devices and the position relationship between the multiple camera devices, and then determine whether the camera device is a suspected camera device based on the difference between the predicted flow rate and the data flow rate of the camera device. For more details about the prediction model, see Figure 4 and its related description.

[0033] The inspection module 230 is used to perform a fault inspection on the suspected camera device. For more details on the fault inspection, see Figure 5 and related descriptions.

[0034] It should be understood that Figure 2 The illustrated system and its modules may be implemented in various ways.

[0035] It should be noted that the above description of the system and its modules is only for convenience of description and does not limit the present specification to the scope of the embodiments. It is understandable that, after understanding the principle of the system, those skilled in the art may arbitrarily combine the modules or form a subsystem to connect with other modules without deviating from the principle. In some embodiments, Figure 2 The acquisition module 210, the judgment module 220, and the inspection module 230 disclosed in the specification can be different modules in a system, or a module can realize the functions of two or more modules. For example, each module can share a storage device, and each component can also have its own storage device. Such variations are all within the scope of protection of this specification.

[0036] Figure 3 300 is an exemplary flow chart of a method for monitoring the working status of a camera device according to some embodiments of this specification. In some embodiments, process 300 may be executed by a camera device monitoring unit. The camera device monitoring unit may be implemented by a computing device having a processing chip, such as server 120. Figure 3 As shown, process 300 includes the following steps:

[0037] Step 310: Obtain the data flow of the camera device.

[0038] The camera device may be a collection terminal, which is used to collect image data and transmit the image data to the camera device monitoring unit via a network.

[0039] Data flow may refer to the data flow generated when the image data collected by the camera device is transmitted. For example, 120K, 1M, etc. It is understandable that the larger the amount of image data collected by the camera device, the larger the corresponding data flow. The image data may be multiple original or compressed images or videos collected by the camera device, or one or more frames of images in the collected video.

[0040] In some embodiments, the camera monitoring unit can obtain real-time data flow based on the image data of the human flow collected by the monitoring camera. For example, the camera monitoring unit can determine the real-time data flow by obtaining the data volume of the video code stream transmitted by the video data collected by the camera.

[0041] Exemplarily, the camera equipment on the road collects the video of the flow of people, and generates a video code stream after compressing the video data of the flow of people through various video encoding and decoding technologies (such as AVS3, H.264, etc.). The video code stream includes multiple image frames (such as I frames, B frames, P frames). When the flow of people on the road is greater, the content change in the image (such as the change in the number of human bodies, the change in human movements, etc.) is greater, and the corresponding video code stream has a larger data volume (such as a large number of I frames). The video code stream is transmitted to a terminal device such as a display screen through a wired or wireless network, and the data is restored and displayed after decoding. The camera equipment monitoring unit can obtain the real-time data flow at multiple time points based on the data volume of the video code stream received by the terminal device at multiple time points. In some embodiments, the terminal device can display the real-time data flow, such as the data volume flow at the current moment is 200K.

[0042] Step 320: Based on the change in data flow, determine whether there is a suspected camera device, where a suspected camera device is a camera device that may have a fault.

[0043] The change of data flow may refer to the change information of the data flow of the camera device at multiple time points. The change information of data flow may include the direction (or type) of the change of data flow, for example, positive (increase), negative (decrease), etc. The change information of data flow may also include the change amount of data flow, for example, how much it has increased or decreased. Exemplarily, the data flow of the camera device at 5 sequential moments T1, T2, T3, T4, and T5 are 100K, 102K, 106K, 1K, and 2K respectively, then the change of data flow at the above 5 moments is 2K, 4K, -105K, and 2K. Among them, the data flow rate dropped sharply from moment T3 to moment T4, that is, the change of data flow rate is a decrease, and the amount of change is 105K.

[0044] In some embodiments, the camera device monitoring unit may perform modeling or use various data analysis algorithms, such as regression analysis, discriminant analysis, etc., to analyze and process the data flow of the camera device at multiple sequential time points to obtain change information of the data flow.

[0045] Suspected devices refer to cameras that may have faults, such as those with faulty network cards, poor cable contact, dusty or damaged lenses, or loose optical fiber connections.

[0046] In some embodiments, a suspicious device may be determined based on a change in data traffic. For example, a suspicious device may be a device with abnormal data traffic changes.

[0047] In some embodiments, the camera device that needs to be judged as abnormal can be used as the target device, and the camera device monitoring unit can determine whether the target device is a suspect device based on whether the change in the data flow of the target device exceeds a preset threshold. For example, if the data flow of the target device at two adjacent moments is 205K and 100K respectively, and the difference is greater than the preset difference threshold of 100K, then the target device is a suspect device, where the preset difference threshold can be set according to actual conditions. It should be noted that the number of times the data flow changes of the target device at multiple groups of adjacent time points within a period of time exceed the preset difference threshold can also be counted. When the number exceeds the preset number threshold (such as 10 times), the target device is determined to be a suspect device.

[0048] For another example, a ratio interval of [80%, 120%] may be preset. When the ratio of the actual data flow of the target device to the predicted data flow is within the preset interval, the target device is judged to be normal, otherwise it is judged to be a suspicious camera device. In some embodiments, the predicted data flow may be determined based on the historical flow data of the target device, for example, the average of the historical flow of the target device in multiple historical time periods is used as the predicted data flow. In some embodiments, the predicted data flow may also be determined based on a prediction model. For details, please refer to other parts of this specification, such as Figure 4 content, etc.

[0049] In some embodiments, the camera device monitoring unit can determine the predicted flow of the target device through a prediction model based on the data flow of multiple camera devices and the positional relationship between the multiple camera devices, and further determine whether the target device is a suspected camera device based on the difference between the predicted flow of the target device and the current data flow.

[0050] In some embodiments, the prediction model may be a trained machine learning model. The prediction model may be implemented by a variety of neural network models, such as any one or combination of a recurrent neural network model, a convolutional neural network, or other custom model structures.

[0051] The prediction model can output the predicted traffic of the target device based on the input data traffic and location information of multiple camera devices with certain association, etc. The target device can be any one of the multiple camera devices with certain association.

[0052] Exemplarily, the camera devices with a certain association may be multiple camera devices deployed in sequence on the same road, and their location information may be location coordinate information; the data flow may be the data flow at multiple time points within a preset period of time (such as a certain day or week in the past).

[0053] The prediction model can output the predicted traffic of the target device. For example, the predicted traffic can be the data traffic at one (or more) time points after the above multiple time points. It is understandable that the actual data traffic at one (or more) time points after the above multiple time points and the predicted traffic output by the prediction model may differ greatly or slightly. The camera device monitoring unit can determine whether the target device is a suspect device based on whether the difference between the actual data traffic and the predicted traffic exceeds a preset threshold.

[0054] In some embodiments, the camera monitoring unit can obtain a prediction model based on multiple training samples and label training. The training samples include data flow data at multiple historical time points of multiple associated camera devices on the same road. The label of the training sample can be the actual data flow of the corresponding target device at a certain time point after multiple historical time points, and the label of the training sample can be manually marked. The camera monitoring unit can establish a loss function based on the label of the training sample and the output of the prediction model, and update the parameters of the model. And iteratively update the parameters of the prediction model based on the loss function until the preset conditions are met and the training is completed to obtain a trained prediction model. Among them, the preset conditions can be that the loss function is less than a threshold, converges, or the training cycle reaches a threshold.

[0055] For more details on determining the predicted traffic volume of camera devices using the prediction model, see Figure 4 and its related description.

[0056] Step 330: In response to the existence of a suspected camera device, a fault check is performed on the suspected camera device.

[0057] Fault check can refer to a check to confirm whether a suspected device actually has a fault. For example, relevant personnel can perform network checks, power-on checks, etc. on the suspected device to confirm whether the suspected device is offline or damaged.

[0058] In some embodiments, the camera monitoring unit can obtain the suspicion index of the suspected camera. When the suspicion index meets the preset conditions, the inspection robot performs action changes or brightness changes in front of the suspected camera to check whether the corresponding data flow changes are within the normal range. Figure 5 and its related description.

[0059] Some embodiments of the present specification determine whether a camera device may have a fault by changing the data flow rate, which can achieve rapid positioning and response to camera devices with abnormal data flow rates, thereby improving the real-time and timeliness of the judgment on suspicious camera devices.

[0060] Figure 4 is an exemplary schematic diagram of a prediction model according to some embodiments of the present specification.

[0061] In some embodiments, the prediction model is a graph neural network (GNN) model. Figure 4 As shown, the prediction model 400 can obtain the predicted traffic based on the processing of the traffic graph; the traffic graph uses the camera devices as nodes and the roads between the camera devices as edges, and the specific description is shown below.

[0062] Traffic prediction refers to predicting the possible traffic of each camera device based on the data traffic of multiple cameras and the positional relationship between each camera device in the multiple cameras. Multiple cameras can be cameras installed at different locations within a certain range, such as all cameras in a park.

[0063] In some embodiments, the input of the prediction model 400 may be a flow map composed of multiple camera devices, and the output may be the predicted flow of each camera device.

[0064] A flow graph is a data structure consisting of nodes and edges / paths, which may include multiple nodes and multiple edges / paths connecting multiple nodes.

[0065] In some embodiments, the flow graph can be constructed based on cameras and roads connecting different cameras. The nodes of the flow graph are different cameras, and the edges are roads connecting different cameras, that is, edges are constructed between adjacent cameras that are directly connected by roads.

[0066] In some embodiments, the attributes of the node may include data flow and location information of the camera device, etc.

[0067] In a flow graph, edges have directions and can point from one node to another. The number of edges pointed to by each node is the in-degree of the node. The number of edges pointing out of the node is the out-degree of the node. For example, Figure 4In the example, there are 2 edges pointing to node 1, so the in-degree of node 1 is 2; there is 1 edge pointing from node 1 to other nodes, so the out-degree of node 1 is 1.

[0068] In some embodiments, the attributes of a node may also include a fault index. The fault index is an index that expresses the possibility that the camera device corresponding to the node actually fails. In some embodiments, the fault index may be determined by the ratio of the number of times the camera device actually fails to the number of times it is determined to be a suspect device. For example, camera device A is determined to be a suspect device 5 times, and upon inspection, the number of times it actually fails is 4. The fault index of the node corresponding to camera device A is 0.8, that is, the higher the fault index, the higher the possibility that the camera device corresponding to the node actually fails.

[0069] In some embodiments, the fault index can be updated based on the results of the fault check. Figure 5 The relevant part of .

[0070] In some embodiments, introducing a fault index into the node attributes can take into account the possibility of actual failure of the camera device, thereby improving the accuracy of the prediction model 400.

[0071] In some embodiments, the edges of the traffic graph are directed edges, and the edge attributes of the edges include weight values; and the edges emitted by nodes with out-degree greater than 1 are first-class edges, and the edge attributes of the first-class edges are determined based on the location of the nodes pointed to by the first-class edges.

[0072] Directed edges are edges that connect different nodes in the flow graph. They are directional edges that can be directed from one node to another. For example, Figure 4 In the example, the edge from node 2 to node 3 is a directed edge. In some embodiments, the directed edge can be a road connecting different cameras. For example, an edge is constructed between two cameras on a one-way road, with the camera that the pedestrian passes by first pointing to the camera that the pedestrian passes by later.

[0073] The first type of edge is an edge that is emitted by a node with an out-degree greater than 1. For example, Figure 4 In the example, the out-degree of node 2 is 2, and the two edges from node 2 are first-class edges.

[0074] In some embodiments, the attributes of a directed edge include the distance of the camera device and the weight value of the edge. The distance of the camera device refers to the shortest walking distance between two cameras, which can be determined by the road connecting the two. The weight value of the edge can represent the probability of a pedestrian walking from the node where the edge originates to the node where the edge points when the pedestrian is at the location of the camera device corresponding to the node where the edge originates. For example, Figure 4In the example, there are two edges from node 2 connecting node 1 and node 3 respectively. If the pedestrian is at the location of the camera corresponding to node 2, the probability of him walking towards the camera corresponding to node 1 and node 3 is 0.5 respectively. Therefore, the weight value of the edge from node 2 to node 1 can be 0.5, and the weight value of the edge from node 2 to node 3 can be 0.5.

[0075] In some embodiments, the weight value of the first type of edge can also be set based on the corresponding relationship of the roads. For example, the two roads starting from camera device 3 (corresponding to node 3) point to camera device 1 (corresponding to node 1) and camera device n (corresponding to node n) respectively. The road between camera device 3 and camera device 1 is a one-way street, and the road between camera device 3 and camera device n is also a one-way street. Therefore, the weight value of the edge from node 3 to node 3 can be 0.5; the weight value of the edge from node 2 to node 1 can be 0.5.

[0076] In some embodiments, the weight value of the first type of edge can also be determined based on the type of location where the camera device corresponding to the node to which the edge points is located. Among them, the greater the flow of people at the location where the camera device is located, the greater the edge weight. For example, if the road after the camera device 3 (corresponding to node 3) is divided into two forks, one pointing to the camera device 1 (corresponding to node 1), and the other pointing to the camera device n (corresponding to node n), wherein the installation location of the camera device 1 is the entrance of the shopping mall, and the installation location of the camera device n is the entrance of the public toilet. Since the flow of people at the entrance of the shopping mall is higher than the flow of people at the entrance of the public toilet, the weight value of the edge from node 3 to node 1 can be set to 0.8, and the weight value of the edge from node 3 to node n can be set to 0.2.

[0077] In some embodiments, the prediction model 400 may be trained based on a large number of training samples with labels. Specifically, the training samples with labels are input into the initial prediction model 400, and the parameters of the initial prediction model 400 are updated through training. The initial prediction model 400 may be a prediction model without setting parameters.

[0078] In some embodiments, the training sample may include a flow graph of historical camera devices and roads connected to the camera devices. In some embodiments, the training sample may be acquired based on historical data, for example, the server 120 may acquire historical data flow information and location information of the camera device, and graph information of roads connected to the camera device.

[0079] In some embodiments, the training tags may include historical camera traffic information. In some embodiments, the training tags may be acquired based on historical camera data.

[0080] In some embodiments, training can be performed based on training samples by various methods. For example, training can be performed based on a gradient descent method. During training, the training samples can be input into the initial prediction model 400 to determine the model output, the model output and the training label can be input into a loss function, and the initial prediction model 400 can be iterated based on the loss function until the training is completed. The trained initial prediction model 400 is used as the prediction model 400. The completion of training can include the number of iterations exceeding a threshold, the deviation value of the model output converging, etc.

[0081] In some embodiments, by constructing a traffic graph using data flow information and location information between multiple cameras as nodes and roads connecting the cameras as edges, it is possible to better reflect the relationship between different cameras and their mutual influence while reflecting the characteristics of the characteristic data of the camera devices themselves, thereby improving the accuracy of the prediction model 400 in predicting data flow.

[0082] Figure 5 FIG. 5 is an exemplary flow chart of a method for checking a fault of a camera device according to some embodiments of the present specification. In some embodiments, process 500 may be executed by server 120. Figure 5 As shown, process 500 includes the following steps:

[0083] Step 510: Obtain the suspicion index of the suspected camera device.

[0084] The suspicion index is data indicating the possibility of a suspected device failing, wherein the greater the suspicion index, the greater the possibility of a suspected device failing. The suspicion index may be represented by a numerical value, for example, the suspicion index may be 0.1.

[0085] In some embodiments, the suspicion index can be determined based on a variety of methods. For example, the suspicion index can be determined based on historical failure data of camera devices of various models or different installation locations. In some embodiments, the failure index of the suspected camera device can be used as the suspicion index. For a description of the failure index, see Figure 4 .

[0086] In some embodiments, the suspicion index can also be determined based on the difference between the ratio of the second component to the first component and a preset reference value of the suspect device. For details, see Figure 6 and its related description.

[0087] In some embodiments, the server 120 may obtain the suspicion index of the suspected camera device through the discriminant model. For more details about the discriminant model, see Figure 6 and its related description.

[0088] Step 520, in response to the suspicion index satisfying the preset condition, based on the movement change or brightness change of the inspection robot in front of the suspected camera device, check whether the corresponding data flow change is within the normal range.

[0089] The preset condition means that the suspicion index is greater than the preset suspicion index. The preset suspicion index can be determined based on the historical working data of different camera devices. The more times the failure occurs, the smaller the preset suspicion index. For example, the preset suspicion index of camera device A is 0.1, and the actual suspicion index of camera device A is 0.4. It can be considered that the suspicion index of camera device A meets the preset condition.

[0090] Motion changes refer to changes in the motion of an object captured by a camera, such as swinging arms, moving, squatting, standing up, etc.

[0091] In some embodiments, the motion change includes a performance by the inspection robot of a motion amplitude that changes in the time domain. The inspection robot can obtain data flow change information of the inspection robot image captured by the camera device by performing a performance based on the motion amplitude that changes in the time domain within the shooting range of the suspected device. For example, the inspection robot can gradually increase the amplitude of the motion performance (e.g., from swinging the arm to moving back and forth) within the shooting screen of the suspected device according to the time change. Among them, the inspection robot can be a robot used to patrol and inspect the fault conditions of the camera device.

[0092] In some embodiments, it is possible to determine whether the camera device is faulty based on the data flow change information of the inspection robot image captured by the camera device. For example, it is possible to determine whether the camera device is faulty based on the flow change information being less than a threshold. The threshold can be preset manually according to different situations.

[0093] In some embodiments, the accuracy of fault identification can be improved by judging whether the camera device is faulty through the changes in the movements of the inspection robot.

[0094] The brightness change refers to the brightness change information of the picture taken by the camera device. In some embodiments, the inspection robot can change the brightness value of the picture taken by the camera device by irradiating the camera device with strong light, continuously turning on or off the lighting device of the camera device shooting area, etc.

[0095] In some embodiments, the brightness value of the image captured by the camera device when illuminated by different light intensities can be preset, and the server can determine that the camera device has a fault based on the difference between the actual brightness value of the image captured by the camera device and the preset brightness value being greater than the preset brightness threshold. The preset brightness threshold can be manually preset according to different ambient brightness conditions. For example, the preset brightness value corresponding to a certain ambient brightness is 30, the preset brightness threshold is 3, and the actual brightness value of the image captured by camera device A is 20, then the server can determine that camera device A has a fault.

[0096] In some embodiments, a change threshold of the brightness change of the image corresponding to the camera device when the ambient brightness changes continuously can be preset, and when the actual brightness change of the camera device is less than the change threshold, the processor can determine that the camera device has a fault. For example, the change threshold corresponding to a certain brightness change can be preset as 10, and under the brightness change, the actual brightness change of the image captured by the camera device is 5, then the processor can determine that the camera device has a fault.

[0097] In some embodiments, a variation range of the data flow may be preset, wherein the preset variation range of the data flow may be determined based on historical data.

[0098] In some embodiments, when there is a change in the motion of the image captured by the camera device, the data flow rate will also change accordingly. For example, when there is a certain motion change in the camera device, the preset flow rate change threshold is 0.7, and the data flow rate change determined by the actual image captured by the camera device is 0.2, which is less than the preset flow rate change threshold. It can be considered that the change in data flow rate does not meet the normal range, and the suspected device may be faulty, and further troubleshooting of fault information is required.

[0099] In some embodiments, when there is a brightness change in the image captured by the camera device, the data flow rate will also change accordingly. For example, when there is a certain brightness change in the camera device, the preset flow rate change threshold is 0.4, and the data flow rate change of the image actually captured by the camera device is 0.1, which is less than the preset flow rate change threshold. It can be considered that the change in data flow rate does not meet the normal range, and the suspected device may be faulty, and further troubleshooting of fault information is required.

[0100] In some embodiments, the inspection robot uses both motion changes and brightness changes to perform fault inspections on suspected equipment, which can improve the accuracy of fault inspections on suspected equipment.

[0101] It should be noted that the description of the above-mentioned process of fault inspection is only for example and explanation, and does not limit the scope of application of this specification. For those skilled in the art, various modifications and changes can be made to the above-mentioned process under the guidance of this specification. However, these modifications and changes are still within the scope of this specification.

[0102] Figure 6 It is an exemplary schematic diagram of a discriminant model according to some embodiments of the present specification.

[0103] In some embodiments, the fault check further includes identifying the images taken by the suspected camera device through a discriminant model to predict the suspicion index. The discriminant model may refer to a model for determining the suspicion index of the camera device fault.

[0104] like Figure 6 As shown, the discriminant model 600 includes an image recognition layer 610, a sequence layer 620, and a judgment layer 630. In some embodiments, the image recognition layer 610 may be a convolutional neural network (CNN), the sequence layer may be a recurrent neural network (RNN), and the judgment layer may be a neural network (NN).

[0105] In some embodiments, the input of the image recognition layer 610 is a plurality of frames of human images taken by a camera device, and the output is the key point information of the human body. For example, the input of the image recognition layer 610 can be a plurality of frames of human images taken within a period of time (e.g., 3 seconds) in a shopping mall, and the output can be the key point information of each person in each frame of the image. For a detailed description of the key point information of the human body, please refer to the relevant part below.

[0106] In some embodiments, the input of the sequence layer 620 is the human body key point information output by the image recognition layer, and the output is a human body key point information sequence. For a detailed description of the human body key point information sequence, please refer to the relevant part below.

[0107] In some embodiments, the input of the judgment layer 630 is the human body key point sequence output by the sequence layer, and the output is the character state. For example, the character state can be still, moving, etc. For a detailed description of the character state judgment, please refer to the relevant part below.

[0108] In some embodiments, the discriminant model can be determined by training an initial discriminant model based on historical data. The initial discriminant model can include an initial image recognition layer, an initial sequence layer, and an initial judgment layer. The discriminant model can be obtained by jointly training the initial image recognition layer, the initial sequence layer, and the initial judgment layer. Among them, the initial image recognition layer, the initial sequence layer, and the initial judgment layer can all be models without setting parameters.

[0109] The historical data may include training samples and training labels. The training samples may include multiple groups of historical multi-frame character images. The training labels may be status information of the characters in the historical multi-frame character images. The training samples and labels may be obtained by collecting images taken by different camera devices and manually annotating the character status, and then inputting the model as sample data for training.

[0110] In some embodiments, the output of the image recognition layer can be the input of the sequence layer, and the output of the sequence layer can be used as the input of the judgment layer. The joint training process may include: inputting multiple groups of historical multi-frame character images into the initial image recognition layer; inputting the human key point information output by the initial image recognition layer into the initial sequence layer, inputting the human key point sequence output by the initial sequence layer into the initial judgment layer to determine the output of the initial discriminant model, inputting the character state information and training labels output by the initial discriminant model into the loss function, and iterating the initial discriminant model based on the loss function value until the training is completed. The trained initial discriminant model is used as the discriminant model. Among them, the completion of training may include the number of iterations exceeding a threshold, the deviation value of the model output converging, etc.

[0111] In some embodiments, using a machine learning model to predict the suspicion index can achieve a higher recognition accuracy rate and a higher recognition efficiency than human judgment.

[0112] In some embodiments, the discriminant model can obtain key point information based on human body recognition, determine the first component based on the key point information sequence of each person, determine the second component based on the statistical value of the stationary person, and determine the suspicion index based on the first component and the second component.

[0113] Key points of the human body refer to the points on the human body that can determine the posture of the human body, such as the head, left and right elbow joints, left and right hands, left and right knee joints, left and right feet, etc.

[0114] The key point information of the human body refers to the position of the key point of the human body in the image of the person taken by the camera and the corresponding time. For example, at time T1, the left hand of the person in the image is at position A in the picture, and the key point information of the left hand of the person is position A at time T1.

[0115] The human key point information sequence refers to a set of sequences of key point information based on time. For example, the head of a person in the image is at position A1 at time T1, at position A2 at time T2, at position A3 at time T3, etc. The human key point sequence of the head of the person is (A1, A2, A3). Among them, T1, T2, T3, can be time points with a certain interval of the same time (such as 0.1 seconds).

[0116] In some embodiments, the judgment layer 630 can judge the state information of the character based on the human key point sequence. The greater the character's action, the greater the change amplitude of each position in the human key point sequence. In some embodiments, the change amplitude of each position in the human key point sequence can be determined based on the pixel point, and the greater the pixel point of the human key point movement, the greater the change amplitude of each position in the human key point sequence.

[0117] In some embodiments, a person whose human key point position change amplitude is less than a certain threshold can be considered as a stationary person, wherein the threshold can be set manually. For example, the human key point position change amplitude threshold can be preset as 10 pixels, and when the human key point position change amplitude exceeds the amplitude threshold, it is in motion. For example, it can be judged that person A whose key point position change amplitude is 8 pixels at time T1 and T2 is in a stationary state, and person B whose human key point position change amplitude is 70 pixels is in a motion state.

[0118] The first component refers to the number information of all persons in the image captured by the camera device, including the number information of all moving and stationary persons. For example, the image recognition layer 610 recognizes that there are 6 persons in an image, and the first component of the image is 6.

[0119] In some embodiments, the first component may be determined by the server based on the output of the image recognition layer 610. In some embodiments, the image recognition layer may determine the first component in the image based on the recognized human key point information. For example, if the image recognition layer recognizes that there are 10 head key point information in the image, the server may determine that the first component in the image is 10.

[0120] In some embodiments, the second component refers to the number of stationary persons in the image captured by the camera device. For the determination of stationary persons, please refer to the relevant description of the human body key point sequence above. For example, if there are 3 stationary persons in an image, the second component of the image is 3.

[0121] In some embodiments, the second component may be determined by the server based on the character status information output by the judgment layer 630. For example, if the judgment layer 630 judges that there are three people in a stationary state in the image, the server may judge that the second component of the image is 3.

[0122] In some embodiments, the suspicion index can be determined based on the difference between the ratio of the second component to the first component and the preset reference value of the camera device. The preset reference value refers to the preset ratio of the second component to the first component, and the preset value can be artificially set based on the flow of people at the location of the camera device. For example, the camera device R located in a shopping mall has a preset ratio of the second component to the first component of 0.05, and the ratio of the second component to the first component obtained based on the discriminant model is 0.3, and the suspicion index of the camera device R is 0.25.

[0123] In some embodiments, based on the human body recognition key point information, the ratio of the second component to the first component is calculated to determine the suspicion index, which can automatically determine the possibility of failure of the camera equipment, save labor costs, and improve the fault inspection efficiency of the camera equipment.

[0124] Some embodiments of the present specification also provide a monitoring camera device working status device, comprising at least one processor and at least one memory, wherein the at least one memory is used to store computer instructions; the at least one processor is used to execute at least part of the computer instructions to implement the above-mentioned monitoring camera device working status method.

[0125] Some embodiments of the present specification also disclose a computer-readable storage medium, which stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer runs the above-mentioned method for monitoring the working status of a camera device.

[0126] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only for example and does not constitute a limitation of this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements and corrections to this specification. Such modifications, improvements and corrections are suggested in this specification, so such modifications, improvements and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.

[0127] At the same time, this specification uses specific words to describe the embodiments of this specification. For example, "one embodiment", "an embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more in different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures or characteristics in one or more embodiments of this specification can be appropriately combined.

[0128] In addition, unless explicitly stated in the claims, the order of the processing elements and sequences described in this specification, the use of alphanumeric characters, or the use of other names are not intended to limit the order of the processes and methods of this specification. Although the above disclosure discusses some invention embodiments that are currently considered useful through various examples, it should be understood that such details are only for illustrative purposes, and the attached claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the essence and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing server or mobile device.

[0129] Similarly, it should be noted that in order to simplify the description disclosed in this specification and thus help understand one or more embodiments of the invention, in the above description of the embodiments of this specification, multiple features are sometimes combined into one embodiment, figure or description thereof. However, this disclosure method does not mean that the features required by the subject matter of this specification are more than the features mentioned in the claims. In fact, the features of the embodiments are less than all the features of the single embodiment disclosed above.

[0130] In some embodiments, numbers describing the number of components and attributes are used. It should be understood that such numbers used in the description of the embodiments are modified by the modifiers "about", "approximately" or "substantially" in some examples. Unless otherwise specified, "about", "approximately" or "substantially" indicate that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may change according to the required features of individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and adopt the general method of retaining digits. Although the numerical domains and parameters used to confirm the breadth of their range in some embodiments of this specification are approximate values, in specific embodiments, the setting of such numerical values ​​is as accurate as possible within the feasible range.

[0131] Each patent, patent application, patent application publication, and other materials, such as articles, books, specifications, publications, documents, etc., cited in this specification are hereby incorporated by reference in their entirety. Except for application history documents that are inconsistent with or conflicting with the contents of this specification, documents that limit the broadest scope of the claims of this specification (currently or later attached to this specification) are also excluded. It should be noted that if the descriptions, definitions, and / or use of terms in the materials attached to this specification are inconsistent or conflicting with the contents described in this specification, the descriptions, definitions, and / or use of terms in this specification shall prevail.

[0132] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, as an example and not a limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims

1. A method for monitoring the working status of a camera device, performed by a camera device monitoring unit, characterized in that: include: Get the data flow of the camera device; Based on the change in the data flow, determining whether there is a suspected camera device, wherein the suspected camera device is a camera device that may have a fault; In response to the existence of the suspected camera device, performing a fault check on the suspected camera device, including: Obtaining the suspicion index of the suspected camera device through a discriminant model; the discriminant model includes performing human body recognition based on an image and obtaining key point information, determining a first component based on a key point information sequence of each person, determining a second component based on a statistical value of a stationary person, and determining the suspicion index based on the first component and the second component; In response to the suspicion index satisfying a preset condition, based on the inspection robot performing a motion change or a brightness change in front of the suspected camera device, checking whether the corresponding change in the data flow rate is within a normal range; In response to the change in the data flow not being within a normal range, the suspected camera device is faulty.

2. The method for monitoring the working status of a camera device according to claim 1, characterized in that: The determining whether there is a suspicious camera device based on the change in the data flow rate includes: Based on the data flow of the plurality of camera devices and the positional relationship between the plurality of camera devices, a predicted flow of each camera device is determined by a prediction model; the prediction model is a machine learning model; Based on the difference between the predicted flow rate of the camera device and the data flow rate, it is determined whether the camera device is the suspected camera device.

3. The method for monitoring the working status of a camera device according to claim 2, characterized in that: The prediction model is a graph neural network model, and the prediction model obtains the predicted traffic based on processing a traffic graph; the traffic graph uses the camera devices as nodes and the roads between the camera devices as edges.

4. The method for monitoring the working status of a camera device according to claim 3, characterized in that: The edge of the flow graph is a directed edge, and the edge attribute of the edge includes a weight value; The edge emitted by the node with an out-degree greater than 1 is a first-type edge, and the edge attribute of the first-type edge is determined based on the position of the node pointed to by the first-type edge.

5. The method for monitoring the working status of a camera device according to claim 1, characterized in that: The motion change includes a performance in which the inspection robot performs motion amplitude changes in the time domain.

6. A system for monitoring the working status of a camera device, characterized in that: include: An acquisition module, used to acquire data flow of a camera device; A judgment module, used for judging whether there is a suspected camera device based on the change of the data flow, wherein the suspected camera device is a camera device that may have a fault; The inspection module is used to perform a fault inspection on the suspected camera device, including: Obtaining the suspicion index of the suspected camera device through a discriminant model; the discriminant model includes performing human body recognition based on an image and obtaining key point information, determining a first component based on a key point information sequence of each person, determining a second component based on a statistical value of a stationary person, and determining the suspicion index based on the first component and the second component; In response to the suspicion index satisfying a preset condition, based on the inspection robot performing a motion change or a brightness change in front of the suspected camera device, checking whether the corresponding change in the data flow rate is within a normal range; In response to the change in the data flow not being within a normal range, the suspected camera device is faulty.

7. The system for monitoring the working status of a camera device according to claim 6, characterized in that: The judging module is also used for: Based on the data flow of the plurality of camera devices and the positional relationship between the plurality of camera devices, a predicted flow of each camera device is determined by a prediction model; the prediction model is a machine learning model; Based on the difference between the predicted flow rate of the camera device and the data flow rate, it is determined whether the camera device is the suspected camera device.

8. A device for monitoring the working status of a camera device, characterized in that: The device comprises: at least one storage medium storing computer instructions; At least one processor executes the computer instructions to implement the monitoring camera equipment working status method according to any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that: The storage medium stores computer instructions. When a computer reads the computer instructions, the computer executes the method for monitoring the working status of a camera device according to any one of claims 1 to 5.

Images