A production detection method, system, electronic device and storage medium

Abstract

The application discloses a production detection method and device, electronic equipment and a storage medium. Video data is collected by a camera facing a production scene related to production safety, and the video data is transmitted to an edge computing node. When the content of the video data is detected to change according to the similarity between frames at the camera, a channel established between the camera and the edge computing node is woken up. The edge computing node responds to the woken-up channel, detects the production state of the above-mentioned production scene in the received video data, and performs an alarm operation when the detected production state is abnormal, to remind the safety condition of the current production scene to be processed, thereby improving the safety of the production process.

Description

Technical Field

[0001] This invention relates to the field of safety production testing technology, and in particular to a production testing method, system, electronic device, and storage medium. Background Technology

[0002] In the power and coal mining sectors, safety in production is often a primary concern. To ensure safety, video recognition equipment is commonly used for monitoring the production process. This involves deploying cameras in production workshops or mines and performing intelligent image analysis on the video data collected by these cameras to determine the safety of the production process.

[0003] However, currently, there are two main methods for achieving safe production through video recognition equipment: 1. Analyzing video data uploaded from various cameras via a cloud platform video monitoring and analysis server; 2. Analyzing video data transmitted from cameras via edge computing nodes deployed in monitoring rooms in factories or mines. Both methods have their drawbacks. For example, when using a cloud platform for video monitoring and analysis, the wide monitoring range, large number of connected cameras, and diverse types of abnormal scenarios that do not conform to safe production requirements place higher demands on the cloud platform's computing power and latency. Furthermore, because video data consumes a lot of bandwidth, the cloud platform typically cannot handle a large volume of simultaneous video data access, creating blind spots in safety monitoring and hindering the achievement of safe production. Moreover, the cloud server nodes in the cloud platform are usually far from the camera nodes collecting video data, resulting in significant delays in the video data acquisition-analysis-retransmission process. If dangerous scenes such as open flames or smoke appear in the video data, timely response is difficult, increasing the potential dangers in the production process.

[0004] When edge computing nodes are used to analyze video data to ensure safe production, the video data often contains a large proportion of still images (images that meet safety requirements) and a small proportion of valid images (images that do not meet safety requirements). This results in a large amount of invalid data being analyzed by the edge computing nodes, leading to a waste of computing power. Furthermore, when multiple cameras are connected, edge computing nodes often use a round-robin approach to analyze the video data collected by each camera. When the number of connected cameras is too large, the round-robin efficiency of the edge computing nodes decreases, making it difficult to process valid video data in a timely manner. As the monitoring range and the number of connected cameras increase, the processing time of the edge computing nodes for video data will further increase, and the processing accuracy will further decrease, making it difficult to meet the needs of monitoring safe production. Summary of the Invention

[0005] This invention provides a production inspection method, system, electronic device, and storage medium to solve the problem of large video data processing volume and long processing time during production inspection, which leads to untimely response to abnormal production states.

[0006] According to one aspect of the present invention, a production inspection method is provided, comprising:

[0007] The camera is positioned to collect video data from the production scene involving production safety, and transmits the video data to the edge computing node.

[0008] The camera detects whether the content has changed in the video data based on the similarity between frames.

[0009] If so, then the channel is activated;

[0010] If not, the channel remains in an unwakeable state;

[0011] The edge computing node responds to the awakened channel and detects the production status of the production scene in the video data;

[0012] When the production status is abnormal, the edge computing node performs an alarm operation.

[0013] According to another aspect of the present invention, a production inspection system is provided, the system comprising an edge computing node and a plurality of cameras arranged in the same production scene, wherein a channel is established between the edge computing node and each of the cameras;

[0014] The camera includes:

[0015] The video data acquisition module is used to acquire video data for the production scenario involving production safety, and transmit the video data to the edge computing node;

[0016] The content detection module is used to detect whether the content in the video data has changed based on the similarity between frames. If it has, the channel wake-up module is called; otherwise, the channel maintenance module is called.

[0017] A channel wake-up module is used to wake up the module.

[0018] A channel maintenance module is used to maintain the channel in an unwakeable state.

[0019] The edge computing nodes include:

[0020] A production status detection module is used to respond to the awakened channel and detect the production status of the production scene in the video data;

[0021] The alarm module is used to perform an alarm operation when the production status is abnormal.

[0022] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0023] At least one processor; and

[0024] A memory communicatively connected to the at least one processor; wherein,

[0025] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the production testing method according to any embodiment of the present invention.

[0026] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the production inspection method according to any embodiment of the present invention.

[0027] The technical solution provided by this invention utilizes a cascaded combination of multiple cameras and edge computing nodes deployed in the same production scene. The cameras collect video data from the production scene, which involves production safety, and transmit the video data to the edge computing nodes. When a change in the video data content is detected at the camera based on frame similarity, the channel established between the camera and the edge computing node is activated. The edge computing node responds to the activated channel, detects the production status of the aforementioned production scene in the received video data, and executes an alarm operation when an abnormal production status is detected, thereby prompting action to address the safety situation of the current production scene and improving the safety of the production process.

[0028] The technical solution provided by this invention, compared to methods that analyze video data uploaded from various cameras via a cloud platform video surveillance analysis server, detects changes in the content of video data at the camera and wakes up the channel between the camera that has collected changed video data and the edge computing node. This allows the edge computing node to detect the production status of the production scene through the video data from the woken-up channel, reducing the amount of video data processed at the edge computing node, improving the efficiency of safety production monitoring and the response efficiency to abnormal production states, reducing response latency, and reducing the waste of computing power caused by identifying unchanged video data. When the number of cameras increases, this application's channel wake-up processing method, compared to the method of processing video data collected by cameras indiscriminately during edge computing node round-robin, reduces the processing latency of video data with changed content, improves processing accuracy, and thus meets the requirements for effective safety production monitoring.

[0029] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a flowchart of a production testing method provided according to Embodiment 1 of the present invention;

[0032] Figure 2 This is a schematic diagram of a production testing system according to Embodiment 3 of the present invention;

[0033] Figure 3 This is a schematic diagram of the structure of an electronic device that implements the production testing method of this invention. Detailed Implementation

[0034] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0035] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0036] Example 1

[0037] Figure 1This invention provides a flowchart of a production inspection method according to Embodiment 1. This embodiment is applicable to situations where abnormal production turntables are identified during production and alarm operations are executed. This method can be performed by a production inspection system, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes:

[0038] The S110 camera collects video data from production scenarios involving production safety and transmits the video data to edge computing nodes.

[0039] In this embodiment, the production scenario can be any of the various safety-related production scenarios involved in power generation or coal mining. For the same production scenario, a cascaded combination of edge computing nodes and multiple cameras can be set up. By collecting video data from different angles and fields of view from multiple cameras, the monitoring coverage of the production scenario can be completed, improving the safety monitoring coverage rate. After collecting video data, multiple cameras can transmit the video data to the edge computing node through the channel established between the camera and the edge computing node.

[0040] S120: The camera detects whether the content in the video data has changed based on the similarity between frames. If so, proceed to step S130; otherwise, proceed to step S140.

[0041] In this embodiment, the similarity between video data frames captured by the camera can be used to detect whether the content of the video data has changed. Typically, during camera monitoring, video data content can be categorized into four types based on whether it has changed: 1) no change; 2) changes due to camera shake caused by external forces such as wind or bird strikes; 3) changes in ambient light due to sunrise or sunset; and 4) actual changes in content. In this embodiment, the detected change in video data content refers to an actual change. When the content of the video data has actually changed, the similarity between video data frames will decrease. Therefore, this embodiment can detect whether the video data content has actually changed by using the similarity between video data frames.

[0042] Specifically, in this embodiment, the similarity between video data frames is used to detect whether the video data content has actually changed. This can be achieved by extracting the current keyframe from the video data. The video data includes the image data of the keyframe and the image data of non-keyframes. The keyframe contains complete image data. Therefore, the current keyframe is extracted from the video data first, and the similarity between keyframes is used to detect changes in the video data content. Non-keyframes are often interpolation frames, which contain less information and are not conducive to identifying changes in the video data content.

[0043] In this embodiment, the similarity between keyframes is calculated by calculating the perceptual hash value of the keyframe. The perceptual hash algorithm can be used to calculate the perceptual hash value of the current keyframe, which is then used as the first target hash value.

[0044] Perceptual hashing is a type of hashing algorithm primarily used for searching for similar images. Its key feature is generating a "fingerprint" string for each frame of image data, comparing the fingerprint information of different image data to determine their similarity. The closer the results, the more similar the different image data are considered. Typically, the fingerprint string calculated by perceptual hashing is a 64-bit hash value consisting of 0s and 1s. In this embodiment, both the first target hash value and the subsequent second target hash value have 64 bits.

[0045] After obtaining the first target hash value, the perceptual hash value of the next prediction frame can be calculated by referring to the previous multi-frame image data in the video data, which serves as the second target hash value. The next prediction frame can be the prediction frame corresponding to the next key frame that is adjacent to the current key frame. By detecting the similarity between the prediction frame and the subsequent key frame, it can be determined whether the content of the video data has changed from the current key frame to the subsequent key frame.

[0046] In this embodiment, calculating the perceptual hash value of the next prediction frame as the second target hash value specifically includes:

[0047] In video data, the most recent frames are selected, with the time intervals between them decreasing progressively. For example, the perceptual hash value of the predicted frame can be calculated by selecting keyframes captured 9 seconds, 5 seconds, 3 seconds, and 1 second before the current keyframe, as well as the three keyframes adjacent to the current keyframe. This selection method is based on the patterns of change in video data content. For instance, if a fire occurs in a production scene, the time it takes for a flame to go from ignition to becoming clearly visible is approximately 9 seconds.

[0048] Then, a perceptual hash value is calculated for each selected frame of image data, resulting in multiple 64-bit perceptual hash values. These perceptual hash values ​​from multiple frames are then merged into the perceptual hash value for the next prediction frame, serving as the second target hash value. In this embodiment, the fusion process can involve selecting the perceptual hash value of any frame of image data as the initial second target hash value representing the next prediction frame. Then, a voting algorithm is used to count the number of each flag value in each digit of the initial second target hash value. In this embodiment, the flag value can be 0 or 1. If a certain flag value has the highest number in a certain digit, then that flag value is set in the initial second target hash value for that digit. For example, if the count shows that 1 has the highest number in a certain digit, then 1 is set as the flag value for that digit in the second target hash value. In this embodiment, the calculation of the second target hash value for the prediction frame using a voting algorithm and selected multiple frames of image data has good time robustness. It can filter out the influence of camera shake and changes in lighting on the calculation of the second target hash value, and is more sensitive to new targets appearing in each frame of image data, improving the responsiveness to changes in video data content.

[0049] In this embodiment, after determining the second target hash value, the similarity between the current frame and the predicted frame can be calculated based on the second target hash value and the first target hash value. However, since cameras can be deployed in different production scenarios, and different production scenarios are affected by different external factors, for example, video data in outdoor production scenarios is easily affected by swaying leaves and changes in sunlight, increasing the similarity threshold can filter out such effects. Therefore, in this embodiment, different similarity thresholds can be set for cameras deployed in different production scenarios. Thus, in this embodiment, while calculating the first target hash value and the second target hash value, it is also necessary to determine the similarity threshold for the camera that captures the key frame pointed to by the first target hash value. Specifically, this can be done by querying the production scenario where the camera is located. In this embodiment, a relationship table between the production scenario monitored by the camera and the similarity threshold can be stored in the camera in advance. After querying the current production scenario of the camera, a similarity threshold adapted to the production scenario can be configured.

[0050] In this embodiment, the similarity between the first target hash value and the second target hash value is calculated by determining the Hamming distance between them. The Hamming distance essentially counts the number of different flag values ​​in the same digit position between the first and second target hash values. If the calculated Hamming distance is less than a predetermined similarity threshold, it can be determined that the content of the video data has not changed, and step S140 is executed. If the Hamming distance is greater than or equal to the similarity threshold, it can be determined that the content of the video data has changed, and step S130 is executed.

[0051] S130, Wake-up Channel.

[0052] In this embodiment, a communication channel is established between the camera and the edge computing node. Due to the limited computing power at the camera and cost considerations, complex calculations are mostly handled by the edge computing node. When the camera detects a change in the video data it is collecting, the communication channel between the camera and the edge computing node can be activated, sending the changed video data to the edge computing node for production status identification of the production scene monitored by the camera. This channel activation reduces the amount of video data processed at the edge computing node, and compared to a polling method, reduces the reaction delay of the edge computing node in identifying the production status of changed video data.

[0053] S140, Keep the channel in an unwake-up state.

[0054] In this embodiment, when the video data monitored by the camera does not change, the communication channel between the camera and the edge computing node is kept in an unwakeable state, thereby reducing the amount of video data transmitted to the edge computing node.

[0055] S150, the edge computing node responds to the awakened channel and detects the production status of the production scene in the video data.

[0056] In this embodiment, the edge computing node can respond to the awakened channel and the video data transmitted through the channel to detect the production status of the production scene monitored by the camera.

[0057] In this embodiment, the production scenario can be, for example, a high-risk area such as the production unit area, storage tank area, loading and unloading room, or pump room; it can also be a warehouse storing radioactive, explosive, highly toxic substances or equipment, as well as various hazardous chemicals; it can be an area requiring hot work, high-altitude work, or confined space work; or an area requiring daily inspections or shutdown maintenance. Finally, the production scenario in this embodiment can also be a monitoring room, control room, or public space. In this embodiment, the detection of the production status of the production scenario in the video data is accomplished by calling a scenario classification model trained for that production scenario stored in the edge computing node. The scenario classification model can detect various target objects in the production scenario in the video data. For example, high-risk areas such as the production unit area, storage tank area, loading and unloading room, and pump room often contain target objects related to abnormal production status, such as people, vehicles, open flames, smoke, and fire extinguishers. The scenario classification model trained for that production scenario can better extract and identify these target objects. As another example, in production scenarios such as monitoring rooms, control rooms, and public spaces, the target object is often only people. In this embodiment, the scene classification model can apply a lightweight convolutional neural network multi-classification model. This model has lower accuracy, requires less computing power, and runs quickly, making it suitable for preliminary identification of target objects in production scenes. After preliminary identification, based on the identified target objects, the target detection model trained for the target objects can be called in the edge computing nodes to detect the production status of the target objects in the video data. In this embodiment, the target detection model can apply a CNN convolutional neural network or a VIT model. These two models have higher accuracy but require more computing power, making them suitable for identifying the production status of target objects. For example, when the target object is a human body, the target detection model can identify whether the human body is off-duty, whether it is wearing protective clothing, a safety helmet, or has climbed over a fence, etc., and can further identify whether there is a gathering of multiple people. When the human body is off-duty, not wearing protective clothing or a safety helmet, or has climbed over a fence, it can be considered that the production status is abnormal. As another example, when the target object is a vehicle, the target detection model can identify the vehicle's license plate number, vehicle type, etc. Finally, in this embodiment, the target detection model can also identify whether a fire has occurred in the production scene, the size of the fire, and whether there is thick smoke.

[0058] S160. When the edge computing node is in an abnormal production state, it will perform an alarm operation.

[0059] In this embodiment, when the edge computing node identifies an abnormal production status through the target detection model, such as when the target object is a human body and the target object is detected to be in an abnormal state of not wearing a safety helmet, an alarm operation is executed.

[0060] Since the production detection method provided in this embodiment can be applied in actual production, this embodiment can also introduce a manual verification step for the production status identified by the scene classification model and the target detection model in actual application, and update the scene classification model and the target detection model at the edge computing node by comparing whether the manual verification and the production status identified by the model are consistent, thereby improving the accuracy of model recognition.

[0061] Specifically, edge computing nodes can receive confirmation information in response to alarm operations. This confirmation information can be generated by a human reviewing video data. When the received confirmation information indicates a genuine alarm, the edge computing node labels the video data as a positive sample; when the confirmation information indicates a false alarm, it labels the video data as a negative sample. Both the video data and the labels are then transmitted to the cloud. The cloud uses the video data and labels to update the scene classification model and the target detection model, respectively. After the cloud updates are complete, the scene classification model and the target detection model are distributed to the edge computing nodes for subsequent identification of the production status of the production scene.

[0062] The technical solution provided by this invention utilizes a cascaded combination of multiple cameras and edge computing nodes deployed in the same production scene. The cameras collect video data from the production scene, which involves production safety, and transmit the video data to the edge computing nodes. When a change in the video data content is detected at the camera based on frame similarity, the channel established between the camera and the edge computing node is activated. The edge computing node responds to the activated channel, detects the production status of the aforementioned production scene in the received video data, and executes an alarm operation when an abnormal production status is detected, thereby prompting action to address the safety situation of the current production scene and improving the safety of the production process.

[0063] The technical solution provided by this invention, compared to methods that analyze video data uploaded from various cameras via a cloud platform video surveillance analysis server, detects changes in the content of video data at the camera and wakes up the channel between the camera that has collected changed video data and the edge computing node. This allows the edge computing node to detect the production status of the production scene through the video data from the woken-up channel, reducing the amount of video data processed at the edge computing node, improving the efficiency of safety production monitoring and the response efficiency to abnormal production states, reducing response latency, and reducing the waste of computing power caused by identifying unchanged video data. When the number of cameras increases, this application's channel wake-up processing method, compared to the method of processing video data collected by cameras indiscriminately during edge computing node round-robin, reduces the processing latency of video data with changed content, improves processing accuracy, and thus meets the requirements for effective safety production monitoring.

[0064] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description.

[0065] Example 2

[0066] Figure 2 This is a schematic diagram of a production testing system provided in Embodiment 2 of the present invention. Figure 2 As shown, the production inspection system includes an edge computing node 210 and multiple cameras 220 set up in the same production scene, and a channel is established between the edge computing node 210 and each of the cameras 220;

[0067] The camera 220 includes:

[0068] The video data acquisition module 2201 is used to acquire video data for the production scenario involving production safety and transmit the video data to the edge computing node;

[0069] Content detection module 2202 is used to detect whether the content in the video data has changed based on the similarity between frames. If it has changed, the channel wake-up module is called; otherwise, the channel maintenance module is called.

[0070] Channel wake-up module 2203 is used to wake up the module;

[0071] Channel maintenance module 2204 is used to maintain the channel in an unwakeable state;

[0072] The edge computing node 210 includes:

[0073] The production status detection module 2101 is used to detect the production status of the production scene in the video data in response to the awakened channel.

[0074] Alarm module 2102 is used to perform an alarm operation when the production status is abnormal.

[0075] Optionally, the content detection module 2202 includes:

[0076] The keyframe extraction module is used to extract the current keyframe from the video data.

[0077] The first hash value calculation module is used to calculate the perceptual hash value of the current key frame as the first target hash value;

[0078] The second hash value calculation module is used to calculate the perceptual hash value of the next predicted frame in the video data by referring to the prior multi-frame image data, as the second target hash value;

[0079] The similarity threshold determination module is used to determine the similarity threshold.

[0080] The Hamming distance calculation module is used to calculate the Hamming distance between the first target hash value and the second target hash value;

[0081] The "No Change Determination Module" is used to determine that the content of the video data has not changed if the Hamming distance is less than the similarity threshold.

[0082] The change determination module is used to determine that the content of the video data has changed if the Hamming distance is greater than or equal to the similarity threshold.

[0083] Optionally, the second hash value calculation module includes:

[0084] An image data extraction module is used to select the earliest multiple frames of image data from the video data, wherein the time interval between the image data is decreasing.

[0085] A perceptual hash value calculation module is used to calculate a perceptual hash value for each frame of image data.

[0086] The perceptual hash value fusion module is used to fuse the perceptual hash values ​​of multiple frames of image data into the perceptual hash value of the next prediction frame, which serves as the second target hash value.

[0087] Optionally, the perceptual hash value fusion module includes:

[0088] An initial value determination module is used to select the perceptual hash value of any frame of the image data as the initial second target hash value representing the perceptual hash value of the next prediction frame;

[0089] The flag value statistics module is used to count the number of flag values ​​for each digit of the perceptual hash value of all the image data for each digit of the initial second target hash value.

[0090] The flag value setting module is used to set the initial second target hash value as the flag value in a certain number of bits if a certain flag value has the largest number of bits in that number of bits.

[0091] Optionally, the similarity threshold determination module includes:

[0092] The first production scenario query module is used to query the production scenario in which the camera is located;

[0093] The threshold configuration module is used to configure a similarity threshold that is adapted to the production scenario.

[0094] Optionally, the production status detection module 2101 includes:

[0095] The second production scenario query module is used to query the production scenario in which the camera is located.

[0096] The first model invocation module is used to invoke a scene classification model trained for the production scene in order to detect target objects in the production scene in the video data;

[0097] The second model invocation module is used to invoke the target detection model trained for the target object in order to detect the production status of the target object in the video data.

[0098] Optionally, the edge computing node 210 in the production testing system further includes:

[0099] The confirmation information receiving module is used to receive confirmation information in response to the alarm operation feedback;

[0100] The positive sample labeling module is used to label the video data as a positive sample when the confirmation information indicates a real police situation;

[0101] The negative sample labeling module is used to label the video data as a negative sample when the confirmation information indicates a false alarm.

[0102] The transmission module is used to transmit the video data and the tag to the cloud;

[0103] The production testing system also includes a cloud platform, which includes:

[0104] The model update module is used to update the scene classification model and the target detection model respectively based on the video data and the tags;

[0105] The model distribution module is used to distribute the scene classification model and the object detection model to the edge computing node when the update is completed.

[0106] The production testing system provided in this embodiment of the invention can execute the production testing method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0107] Example 3

[0108] Figure 3A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0109] like Figure 3 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0110] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0111] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as production inspection methods.

[0112] In some embodiments, the production inspection method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the production inspection method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the production inspection method by any other suitable means (e.g., by means of firmware).

[0113] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0114] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0115] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0116] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0117] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0118] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0119] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0120] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A production testing method, characterized in that, An edge computing node and multiple cameras are set up in the same production scenario, and a channel is established between the edge computing node and each of the cameras. The method includes: The camera collects video data from the production scene involving production safety and transmits the video data to the edge computing node; wherein, an edge computing node and multiple cameras are cascaded together for the same production scene; The camera detects whether the content has changed in the video data based on the similarity between frames. If so, then the channel is activated; If not, the channel remains in an unwakeable state; The edge computing node responds to the awakened channel and detects the production status of the production scene in the video data; When the production status is abnormal, the edge computing node performs an alarm operation; The camera detects whether the content has changed in the video data based on the similarity between frames, including: Extract the current keyframe from the video data; Calculate the perceptual hash value for the current keyframe and use it as the first target hash value; The perceptual hash value of the next predicted frame is calculated by referring to the prior multi-frame image data in the video data, and is used as the second target hash value; Determine the similarity threshold; Calculate the Hamming distance between the first target hash value and the second target hash value; If the Hamming distance is less than the similarity threshold, then it is determined that the content of the video data has not changed; If the Hamming distance is greater than or equal to the similarity threshold, then it is determined that the content of the video data has changed; The step of calculating the perceptual hash value of the next predicted frame in the video data by referencing prior multi-frame image data, as the second target hash value, includes: The video data selects the earliest frames of image data, and the time intervals between the image data show a decreasing trend; Calculate the perceptual hash value for each frame of image data; A voting algorithm is used to fuse the perceptual hash values ​​of multiple frames of image data into the perceptual hash value of the next prediction frame, which is then used as the second target hash value. The step of using a voting algorithm to fuse the perceptual hash values ​​of multiple frames of image data into the perceptual hash value of the next prediction frame, as the second target hash value, includes: Select the perceptual hash value of any frame of the image data as the initial second target hash value representing the perceptual hash value of the next predicted frame; For each digit of the initial second target hash value, count the number of each flag value in each digit of the perceptual hash value of all the image data; If a certain flag value has the largest number of occurrences in a certain number of positions, then the initial second target hash value is set as the flag value in that number of positions.

2. The method according to claim 1, characterized in that, Determining the similarity threshold includes: Query the production scene where the camera is located; Configure a similarity threshold that is adapted to the production scenario.

3. The method according to any one of claims 1-2, characterized in that, The step of detecting the production status of the production scene in the video data includes: Query the production scene where the camera is located; Invoke the scene classification model trained for the production scene to detect target objects in the production scene in the video data; The target detection model trained for the target object is invoked to detect the production status of the target object in the video data.

4. The method according to claim 3, characterized in that, Also includes: The edge computing node receives confirmation information in response to the alarm operation feedback; When the confirmation information indicates a genuine police incident, the edge computing node labels the video data as a positive sample. When the confirmation information indicates a false alarm, the edge computing node labels the video data as a negative sample. The edge computing node transmits the video data and the tag to the cloud; The cloud platform updates the scene classification model and the target detection model based on the video data and the tags, respectively. When the update is complete, the cloud distributes the scene classification model and the target detection model to the edge computing nodes.

5. A production testing system, characterized in that, The system includes edge computing nodes and multiple cameras set up in the same production scene, and a channel is established between the edge computing nodes and each of the cameras. The camera includes: The video data acquisition module is used to acquire video data for the production scenario involving production safety and transmit the video data to the edge computing node; wherein, an edge computing node and multiple cameras are cascaded together for the same production scenario; The content detection module is used to detect whether the content in the video data has changed based on the similarity between frames. If it has, the channel wake-up module is called; otherwise, the channel maintenance module is called. A channel wake-up module is used to wake up the module. A channel maintenance module is used to maintain the channel in an unwakeable state. The edge computing nodes include: A production status detection module is used to respond to the awakened channel and detect the production status of the production scene in the video data; The alarm module is used to perform an alarm operation when the production status is abnormal; The content detection module includes: The keyframe extraction module is used to extract the current keyframe from the video data. The first hash value calculation module is used to calculate the perceptual hash value of the current key frame as the first target hash value; The second hash value calculation module is used to calculate the perceptual hash value of the next predicted frame in the video data by referring to the prior multi-frame image data, as the second target hash value; The similarity threshold determination module is used to determine the similarity threshold. The Hamming distance calculation module is used to calculate the Hamming distance between the first target hash value and the second target hash value; The "No Change Determination Module" is used to determine that the content of the video data has not changed if the Hamming distance is less than the similarity threshold. The change determination module is used to determine that the content of the video data has changed if the Hamming distance is greater than or equal to the similarity threshold. The second hash value calculation module includes: An image data extraction module is used to select the earliest multiple frames of image data from the video data, wherein the time interval between the image data is decreasing. A perceptual hash value calculation module is used to calculate a perceptual hash value for each frame of image data. The perceptual hash value fusion module is used to fuse the perceptual hash values ​​of multiple frames of image data into the perceptual hash value of the next prediction frame using a voting algorithm, as the second target hash value; The perceptual hash value fusion module includes: An initial value determination module is used to select the perceptual hash value of any frame of the image data as the initial second target hash value representing the perceptual hash value of the next prediction frame; The flag value statistics module is used to count the number of flag values ​​for each digit of the perceptual hash value of all the image data for each digit of the initial second target hash value. The flag value setting module is used to set the initial second target hash value as the flag value in a certain number of bits if a certain flag value has the largest number of bits in that number of bits.

6. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the production testing method according to any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that are used to cause a processor to execute the production testing method according to any one of claims 1-4.

Images