Image detection method and device

Abstract

The application provides an image detection method and device, applied to the technical field of Internet of Things, the method comprises the following steps: extracting first image data; based on a first algorithm model, using an inference framework to infer and detect the first image data, and obtaining a first image detection result; in the case that the confidence of the first image detection result is less than a preset value, controlling a target camera to increase the focal length, and controlling the target camera after increasing the focal length to collect a second monitoring video of a first monitoring position; extracting second image data of the second monitoring video, and obtaining a second image detection result; in the case that the confidence of the second image detection result is greater than the preset value, sending the second image detection result to a cloud platform; extracting third image data of a third monitoring video, and in the case that the confidence of the second image detection result is less than the preset value, based on a second algorithm model, obtaining a third image detection result, which is beneficial to improving the efficiency of image detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of Internet of Things, and in particular to an image detection method and device. BACKGROUND

[0002] Edge intelligent gateways are widely used in actual life and work scenarios. In one application mode, an edge intelligent gateway can be connected to a camera for image detection. In existing image detection methods, the edge gateway usually collects monitoring videos of each monitoring position and performs corresponding monitoring tasks on each monitoring video based on monitoring requirements. However, in actual application scenarios, the monitoring tasks of each monitoring position are associated. For example, when performing the monitoring task of personnel intrusion on multiple monitoring positions, if no personnel intrusion is detected at the monitoring position of the gate, it can be concluded that personnel intrusion is generally not detected at other monitoring positions. The existing image detection method performs the same image detection process on the monitoring videos of each monitoring position, so the detection efficiency of the existing image detection method is low. SUMMARY

[0003] The embodiments of the present application provide an image detection method and device to solve the problem of low detection accuracy in the existing image detection method.

[0004] To solve the above technical problems, the present application is implemented as follows:

[0005] In a first aspect, the embodiments of the present application provide an image detection method. The method comprises:

[0006] controlling the target camera to sequentially collect monitoring videos of each monitoring position in the plurality of monitoring positions based on a preset time interval;

[0007] extracting first image data of a first monitoring video of a first monitoring position, the first monitoring position being any one of the plurality of monitoring positions;

[0008] performing inference detection on the first image data based on a first algorithm model using an inference framework to obtain a first image detection result, the first algorithm model being an algorithm model called when performing a first monitoring task on the first monitoring position;

[0009] in a case where the confidence of the first image detection result is less than a preset value, controlling the target camera to increase the focal length and controlling the target camera with the increased focal length to collect a second monitoring video of the first monitoring position;

[0010] extracting second image data of the second monitoring video and performing inference detection on the second image data based on the first algorithm model using the inference framework to obtain a second image detection result;

[0011] In a case where the confidence of the second image detection result is greater than the preset value, the second image detection result is sent to a cloud platform.

[0012] The target camera is controlled to capture a third monitoring video of a second monitoring position, the second monitoring position being a monitoring position other than the first monitoring position among the plurality of monitoring positions.

[0013] Third image data of the third monitoring video is extracted, and in a case where the confidence of the second image detection result is less than the preset value, the third image data is detected by inference based on a second algorithm model using the inference framework to obtain a third image detection result, the second algorithm model being an algorithm model called when performing a monitoring task other than the first monitoring task on the second monitoring position.

[0014] Optionally, the control of the target camera to sequentially capture the monitoring video of each monitoring position among the plurality of monitoring positions based on the preset time interval comprises:

[0015] A wireless network name and a wireless network password are configured.

[0016] In a case where the target camera is connected to the gateway management platform based on the wireless network name and the wireless network password, the target camera is controlled to sequentially capture the monitoring video of each monitoring position among the plurality of monitoring positions based on a preset time interval.

[0017] Optionally, the extraction of the first image data of the first monitoring video comprises:

[0018] A ffmpeg fast motion picture expert group open source library is called to pull stream and frame the first monitoring video to obtain raw stream data of the first monitoring video.

[0019] The raw stream data is decoded into YUY format data by a decoding chip.

[0020] The YUY format data is encoded into BGR format data by an encoding chip, and the first image data comprises the BGR format data.

[0021] Optionally, before the inference detection of the first image data based on the first algorithm model using the inference framework to obtain the first image detection result, the method further comprises:

[0022] First algorithm parameters are configured based on the first monitoring task, the first algorithm parameters comprising at least one of a first algorithm threshold, a video stream address, and an alarm interval.

[0023] Create a first algorithm container based on the first algorithm parameters;

[0024] The first algorithm model is loaded through the first algorithm container, and the first algorithm model is generated based on the YOLOx network model training method.

[0025] Optionally, when at least two algorithm containers are configured at the plurality of monitoring locations, loading the first algorithm model through the first algorithm container includes:

[0026] Send a start message to the first algorithm container;

[0027] Upon receiving the startup message, the first algorithm container loads the first algorithm model.

[0028] Optionally, the method further includes:

[0029] Extract the third image data from the third monitoring video, and if the confidence level of the second image detection result is greater than the preset value, perform inference detection on the third image data based on the first algorithm model and the inference framework to obtain the third image detection result.

[0030] Secondly, embodiments of this application also provide an image detection apparatus. The image detection apparatus includes:

[0031] The first control module is used to control the target camera to sequentially acquire monitoring video from each of the multiple monitoring locations based on a preset time interval;

[0032] The first extraction module is used to extract the first image data of the first monitoring video at the first monitoring location, where the first monitoring location is any one of the plurality of monitoring locations.

[0033] The first inference module is used to perform inference detection on the first image data based on the first algorithm model and using an inference framework to obtain the first image detection result. The first algorithm model is the algorithm model called when performing the first monitoring task on the first monitoring location.

[0034] The second control module is used to control the target camera to increase its focal length when the confidence level of the first image detection result is less than a preset value, and to control the target camera after increasing its focal length to acquire the second monitoring video of the first monitoring position.

[0035] The second inference module is used to extract the second image data of the second monitoring video, and based on the first algorithm model, use the inference framework to perform inference detection on the second image data to obtain the second image detection result;

[0036] The first sending module is used to send the second image detection result to the cloud platform when the confidence level of the second image detection result is greater than the preset value;

[0037] The third control module is used to control the target camera to acquire a third monitoring video at a second monitoring location, where the second monitoring location is one of the multiple monitoring locations other than the first monitoring location.

[0038] The third inference module is used to extract the third image data of the third monitoring video, and when the confidence of the second image detection result is less than the preset value, it uses the inference framework based on the second algorithm model to perform inference detection on the third image data to obtain the third image detection result. The second algorithm model is the algorithm model called when performing other monitoring tasks other than the first monitoring task on the second monitoring location.

[0039] Optionally, the first control module includes:

[0040] The first configuration unit is used to configure the wireless network name and wireless network password;

[0041] The first control unit is configured to, when the target camera is connected to the gateway management platform based on the wireless network name and the wireless network password, control the target camera to sequentially collect monitoring videos from each of the multiple monitoring locations based on a preset time interval.

[0042] Optionally, the first extraction module includes:

[0043] The first calling unit is used to call the ffmpeg fast-forwarding dynamic image expert group open source library to pull and extract frames from the first monitoring video to obtain the raw stream data of the first monitoring video;

[0044] The first decoding unit is used to decode the raw stream data into YUY format data through a decoding chip;

[0045] The first encoding unit is used to encode the YUY format data into BGR format data through an encoding chip, wherein the first image data includes the BGR format data.

[0046] Optionally, the device further includes:

[0047] The first configuration module is used to configure first algorithm parameters based on the first monitoring task. The first algorithm parameters include at least one of the following: a first algorithm threshold, a video stream address, and an alarm interval.

[0048] The first creation module is used to create a first algorithm container based on the first algorithm parameters;

[0049] The first loading module is used to load the first algorithm model through the first algorithm container. The first algorithm model is generated based on the YOLOx network model training method.

[0050] Optionally, when at least two algorithm containers are configured at the plurality of monitoring locations, the first loading module includes:

[0051] The first sending unit is used to send a start message to the first algorithm container;

[0052] The first loading unit is used to load the first algorithm model through the first algorithm container when the first algorithm container receives the start message.

[0053] Optionally, the device further includes:

[0054] The third inference module is used to extract the third image data of the third monitoring video, and when the confidence level of the second image detection result is greater than the preset value, it uses the inference framework based on the first algorithm model to perform inference detection on the third image data to obtain the third image detection result.

[0055] Thirdly, embodiments of this application also provide an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the image detection method described above.

[0056] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the image detection method described above.

[0057] The image detection method of this application embodiment includes: controlling a target camera to sequentially acquire monitoring videos of each of a plurality of monitoring locations based on a preset time interval; extracting first image data from the first monitoring video of a first monitoring location; performing inference detection on the first image data using an inference framework based on a first algorithm model to obtain a first image detection result, wherein the first algorithm model is the algorithm model called when performing a first monitoring task on the first monitoring location; if the confidence level of the first image detection result is less than a preset value, controlling the target camera to increase its focal length, and controlling the target camera with increased focal length to acquire a second monitoring video of the first monitoring location; and extracting the second monitoring video... The method uses a second image data set and, based on the first algorithm model, performs inference detection on the second image data using an inference framework to obtain a second image detection result. If the confidence level of the second image detection result is greater than a preset value, the second image detection result is sent to the cloud platform. The method then controls the target camera to acquire a third monitoring video from the second monitoring location. The third image data from the third monitoring video is extracted, and if the confidence level of the second image detection result is less than a preset value, the method uses the second algorithm model and an inference framework to perform inference detection on the third image data to obtain a third image detection result. The second algorithm model is the algorithm model invoked when performing other monitoring tasks on the second monitoring location besides the first monitoring task. This method improves the efficiency of the image detection method by repeatedly judging the image detection results when performing the first monitoring task on the first monitoring location. If the image monitoring results at the first monitoring location are repeatedly less than the confidence level, the first monitoring task is no longer performed on the second monitoring location. Attached Figure Description

[0058] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 This is a flowchart of the image detection method provided in the embodiments of this application;

[0060] Figure 2 This is a system architecture diagram of the edge gateway provided in the embodiments of this application;

[0061] Figure 3 This is a structural diagram of an image detection device provided in an embodiment of this application;

[0062] Figure 4 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0063] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0064] This application provides an image detection method applied to a gateway management platform. (See also...) Figure 1 , Figure 1 This is a flowchart of the image detection method provided in the embodiments of this application, such as... Figure 1 As shown, it includes the following steps:

[0065] Step 101: Based on a preset time interval, control the target camera to sequentially acquire monitoring videos of each of the multiple monitoring locations;

[0066] In this step, the target camera can be a high-definition pan-tilt camera, typically mounted high up on a structure like a tower. The gateway management platform is the management platform for an edge gateway, usually installed at the base of such a structure. The edge gateway connects to the target camera via Ethernet cable, or wirelessly via technologies like Wireless Fidelity (Wi-Fi) or Bluetooth. Multiple monitoring locations are set up within the monitoring area where the target camera is located. The gateway management platform can log into the camera management page to pre-configure multiple monitoring locations, assigning each location a unique number. Then, based on preset time intervals, it controls the target camera to sequentially capture video from each of these multiple monitoring locations.

[0067] Step 102: Extract the first image data of the first monitoring video at the first monitoring location, where the first monitoring location is any one of the plurality of monitoring locations;

[0068] In this step, the first monitoring location can be any one of multiple pre-set monitoring locations based on monitoring requirements. The gateway management platform extracts key image frames from the first monitoring video by pulling the stream and then extracts the first image data of the key image frames.

[0069] Step 103: Based on the first algorithm model, use the inference framework to perform inference detection on the first image data to obtain the first image detection result. The first algorithm model is the algorithm model called when performing the first monitoring task on the first monitoring location.

[0070] Different monitoring tasks correspond to different algorithm models. Examples of monitoring tasks include forest fire detection and intrusion detection. When the target camera rotates to the first monitoring position, if fire detection is required at that position, the algorithm model corresponding to forest fire detection is invoked. It should be noted that multiple monitoring tasks can be executed simultaneously at the same monitoring position. For example, while performing fire detection at the first monitoring position, intrusion detection can also be performed simultaneously. Therefore, two algorithm models can be invoked concurrently.

[0071] The aforementioned inference framework can be the rnkk inference framework. It should be noted that the rnkk inference framework is embedded in a hardware neural network processing unit (NPU) chip. Assuming the first algorithm model is a forest fire prevention algorithm model, using the inference framework to perform fire detection inference on the first image data, the resulting first image detection result can be the probability that a fire has occurred at the first monitoring location.

[0072] Step 104: If the confidence level of the first image detection result is less than a preset value, control the target camera to increase its focal length, and control the target camera after increasing its focal length to collect the second monitoring video of the first monitoring position.

[0073] In this step, referring to the example in the above steps, when the confidence level that a fire has occurred at the first monitoring location is less than a preset value, that is, when the probability of a fire occurring is low, the gateway management platform controls the target camera to increase its focal length, and controls the target camera with increased focal length to further acquire a second monitoring video of the first monitoring location, so as to obtain a more detailed monitoring image of the first monitoring location.

[0074] Step 105: Extract the second image data of the second monitoring video, and based on the first algorithm model, use the inference framework to perform inference detection on the second image data to obtain the second image detection result;

[0075] In this step, the gateway management platform extracts key image frames from the second monitoring video by pulling the stream and then extracts the first image data of the key image frames.

[0076] Step 106: If the confidence level of the second image detection result is greater than the preset value, send the second image detection result to the cloud platform;

[0077] Referring to the example steps above, if the confidence level of a fire occurring at the first monitoring location is greater than a preset value based on the second image detection result (i.e., the probability of a fire occurring is relatively high), the gateway management platform sends the second image detection result to the cloud platform. Additionally, an alarm message can be sent to the backend management platform, notifying the video frame extraction service to save the video 5 seconds before and after the fire. Video saving involves reading the YUV data cached in memory, compressing and encoding it into h264 format, and then saving it using the ffmpeg open-source library.

[0078] Step 107: Control the target camera to acquire a third monitoring video at the second monitoring location, where the second monitoring location is any monitoring location other than the first monitoring location among the plurality of monitoring locations;

[0079] In this step, after monitoring the first monitoring location, the target camera is controlled to collect monitoring videos from the other monitoring locations.

[0080] Step 108: Extract the third image data of the third monitoring video, and if the confidence level of the second image detection result is less than the preset value, perform inference detection on the third image data based on the second algorithm model and the inference framework to obtain the third image detection result. The second algorithm model is the algorithm model called when performing other monitoring tasks besides the first monitoring task on the second monitoring location.

[0081] In this step, the third image data of the third monitoring video is extracted using the same method as described above. It is particularly important to note that in practical applications, the monitoring status of each monitoring location is correlated. For example, if the first monitoring location is the gate, and no intrusion is detected at the first monitoring location, then the intrusion monitoring task can be discontinued at other monitoring locations.

[0082] Based on the above scenario, the gateway management platform can be pre-configured so that if the confidence level of the second image monitoring result is still less than the preset value, it can be demonstrated that no one has indeed entered the first monitoring location. In order to save computing resources and improve monitoring efficiency, the first monitoring task will no longer be executed at other monitoring locations besides the first monitoring location.

[0083] In addition, it should be noted that there is no limit to the number of times the monitoring video is collected at each monitoring location. In order to obtain more accurate image detection, the target camera can be controlled to zoom in multiple times to collect monitoring video.

[0084] Optionally, controlling the target camera to sequentially acquire monitoring video from each of the plurality of monitoring locations based on a preset time interval includes:

[0085] Configure the wireless network name and password;

[0086] When the target camera connects to the gateway management platform based on the wireless network name and the wireless network password, the target camera is controlled to sequentially collect monitoring videos from each of the multiple monitoring locations based on a preset time interval.

[0087] Because wired connections are difficult or costly to deploy in application scenarios such as high mountains and towers, the gateway management platform and the target camera preferably use a wireless connection in this embodiment. First, the gateway management platform needs to be configured with a wireless network name and password. The target camera is configured with the same wireless network name and password and connects to the gateway management platform. Then, the gateway management platform can control the target camera to collect monitoring video. Specifically, the gateway management platform can interact with the target camera through the Open Network Video Interface Forum (ONVIF).

[0088] In this embodiment, the gateway management platform and the target camera are wirelessly connected, which helps to reduce geographical restrictions and allows the gateway management platform to flexibly control the target camera to collect monitoring videos of the monitoring location.

[0089] Optionally, extracting the first image data from the first monitoring video includes:

[0090] The ffmpeg fast-forwarding dynamic image expert group open source library is called to pull and extract frames from the first monitoring video to obtain the raw stream data of the first monitoring video;

[0091] The raw stream data is decoded into YUY format data using a decoding chip;

[0092] The YUY format data is encoded into BGR format data using an encoding chip, wherein the first image data includes the BGR format data.

[0093] In the image detection method of this application embodiment, the gateway management platform pulls and extracts frames from the first monitoring video by calling the ffmpeg fast-forwarding dynamic image expert group open-source library to obtain the raw stream data of the first monitoring video. Then, the raw stream data is decoded into YUY format data by a decoding chip, and then the YUY format data is encoded into BGR format data by an encoding chip. It should be noted that after obtaining the first image data, the gateway management platform can put the first image data into a cache and call it at any time when needed.

[0094] The image detection method in this application extracts the first image data from the first monitoring video, which is beneficial for subsequent inference detection based on the first image data to obtain the first image detection result.

[0095] Optionally, before obtaining the first image detection result by performing inference detection on the first image data using an inference framework based on the first algorithm model, the method further includes:

[0096] Based on the first monitoring task, configure the first algorithm parameters, which include at least one of the following: a first algorithm threshold, a video stream address, and an alarm interval;

[0097] Create a first algorithm container based on the first algorithm parameters;

[0098] The first algorithm model is loaded through the first algorithm container, and the first algorithm model is generated based on the YOLOx network model training method.

[0099] In the image detection method of this application embodiment, before the gateway management platform performs inference detection on the first image data based on the first algorithm model, it first configures corresponding algorithm parameters for each monitoring task. These algorithm parameters include at least one of the following: algorithm threshold, video stream address, and alarm interval. A corresponding algorithm container is created based on the algorithm parameters corresponding to the algorithm task. When the algorithm model is called, it is first loaded into the algorithm container. Creating an algorithm container helps to encapsulate the deep learning algorithm model, enabling it to run on different computing platforms and improving its performance.

[0100] In addition, the algorithm model can be generated by training a YOLOx network model. Specifically, a pre-trained model is first trained using the MOCO (Momentum Contrast for Unsupervised Visual Representation Learning) algorithm to initialize the weights for YOLOx-L training. This allows the collected unlabeled data to improve performance. After generating the YOLOx-L model, a YOLOx-S model is distilled from this model for use in embedded devices. This method of using a large YOLOx-L model to guide a small YOLOx-S model can enable the trained algorithm model to have better performance.

[0101] Optionally, when at least two algorithm containers are configured at the plurality of monitoring locations, loading the first algorithm model within the first algorithm container includes:

[0102] Send a start message to the first algorithm container;

[0103] Upon receiving the startup message, the first algorithm container loads the first algorithm model.

[0104] In the image detection method of this application embodiment, in one application scenario, multiple monitoring locations are configured with at least two algorithm containers, indicating that some monitoring locations correspond to different algorithm containers and require different monitoring tasks. For example, the gateway management platform pre-creates two algorithm containers: one related to forest fire prevention monitoring tasks, and the other related to personnel intrusion detection monitoring tasks. Different port mappings are applied to each algorithm container to facilitate subsequent communication between the gateway management platform and the algorithm containers. For example, if the first monitoring location needs to perform a forest fire prevention monitoring task, when the gateway management platform controls the target camera to move to the first monitoring location, it can send a start message to the first algorithm container. The start message can be an HTTP message. Upon receiving the start message, the first algorithm container loads the first algorithm model and then performs inference detection on the image data of the monitoring video at the first monitoring location based on the first algorithm model.

[0105] In this embodiment of the application, when at least two algorithm containers are configured at multiple monitoring locations, a start message is sent to the algorithm container corresponding to the first monitoring location to load the first algorithm model through the first algorithm container, and then the corresponding monitoring task is executed on the monitoring video of the first monitoring location, which is beneficial for targeted image detection of the monitoring video of each monitoring location.

[0106] It should be noted that in another application scenario, the detection requirements of multiple monitoring locations are the same. In this case, it is only necessary to create an algorithm container, and when the target camera moves to each monitoring location, the same monitoring task is executed.

[0107] Optionally, the method further includes:

[0108] Extract the third image data from the third monitoring video, and if the confidence level of the second image detection result is greater than the preset value, perform inference detection on the third image data based on the first algorithm model and the inference framework to obtain the third image detection result.

[0109] In the image detection method of this application embodiment, for example, when the first algorithm model is an algorithm model related to personnel intrusion, the second image detection result represents the probability of personnel intrusion. If the confidence level of the second image detection result is greater than a preset value, it indicates that the probability of personnel intrusion is relatively high.

[0110] Furthermore, in order to obtain the trajectory of the intruder, inference monitoring can be performed on other monitoring locations based on the first algorithm model, which is beneficial for persistently tracking the image detection results obtained above.

[0111] The edge gateway where the gateway management platform mentioned in the various embodiments of this application is located can be designed based on a high-performance RISC machine (Advanced RISC Machines, ARM) architecture chip. Its hardware structure can specifically include a core board and a routing board. The core board mainly includes a Central Processing Unit (CPU), memory, a video processing chip, an image processing chip, an NPU neural network computing chip, a wired network card, and device interfaces such as RJ45, RS485 / RS232, AI / DI, DO, and Universal Serial Bus (USB). To meet the actual business needs of wireless internet access and Wi-Fi camera access, a 4 / 5G dial-up internet module and a Wi-Fi access module are specifically added. The routing board provides hardware routing functionality for the gateway, pre-converted to the OpenWRT system, and transmits data with the core board through a network interface. The edge intelligent gateway adapts different types of hardware drivers by tailoring and streamlining the Ubuntu system, enabling artificial intelligence (AI) applications to run normally.

[0112] See Figure 2 The software design scheme for the edge gateway can be as follows:

[0113] 1. The overall functionality adopts a layered architecture and modular design to reduce the coupling between modules, support on-demand configuration and dynamic loading of functions, and adapt to different business scenarios.

[0114] 2. The front-end pages use a static page approach, retrieving data from the backend via JavaScript for secondary rendering. All static pages are hosted by an Nginx service, which reverse-proxies Hypertext Transfer Protocol (HTTP) requests to the corresponding business processes.

[0115] 3. The system's business function modules adopt a standardized interface design, which facilitates subsequent function iteration and expansion, and achieves efficient adaptation between the northbound cloud platform, upper-layer intelligent application software, and intelligent algorithm applications.

[0116] 4. The algorithm is packaged and run using Docker, which facilitates unified management and maintenance.

[0117] The edge gateway functional module may include:

[0118] 1. System management module, responsible for sub-functional modules such as device information, time configuration, serial port settings, user management, upgrade and maintenance.

[0119] 2. Network Management Module: Responsible for the following sub-modules: Wide Area Network (WAN) port settings, Virtual Local Area Network (VLAN) settings, Dynamic Host Configuration Protocol (DHCP) settings, Dynamic Domain Name Server (DDNS) settings, Virtual Private Network (VPN) settings, Network Address Translation (NAT) settings, static routing settings, network diagnostics, and network backup and recovery.

[0120] 3. The algorithm management module is responsible for algorithm-related functions such as algorithm download, creation, and version switching; task-related functions such as task creation, configuration, inspection, and start / stop; and alarm-related functions such as receiving, storing, querying, deleting, forwarding, resuming, and linking with devices.

[0121] 4. The multimedia management module is responsible for the unified management of video stream requests and video stream decoding. The decoded BGR data is stored in shared memory for all algorithms to read and share, avoiding multiple algorithms from repeatedly pulling the same stream.

[0122] 5. Platform Integration Module: Responsible for integrating with the Internet of Things (IoT) platform to enable device registration, heartbeat, information reporting, receiving cloud commands, creating and orchestrating monitoring tasks, and forwarding AI alarms.

[0123] 6. The message queue module uses a built-in Message Queuing Telemetry Transport (MQTT) server for local message subscription and publishing, and message queue control functions. The database used is MongoDB.

[0124] 4. Algorithm application module: Based on the specific algorithm applications developed for actual business, it reads images for analysis, reports detection results, and receives instructions to control the camera pan-tilt unit.

[0125] See Figure 3 , Figure 3 This is a structural diagram of an image detection device provided in another embodiment of this application, as shown below. Figure 3 As shown, the image detection device 300 includes:

[0126] The first control module 301 is used to control the target camera to sequentially collect monitoring videos of each of the multiple monitoring locations based on a preset time interval;

[0127] The first extraction module 302 is used to extract the first image data of the first monitoring video at the first monitoring location, wherein the first monitoring location is any one of the plurality of monitoring locations.

[0128] The first inference module 303 is used to perform inference detection on the first image data based on the first algorithm model and using an inference framework to obtain the first image detection result. The first algorithm model is the algorithm model called when performing the first monitoring task on the first monitoring location.

[0129] The second control module 304 is used to control the target camera to increase its focal length when the confidence level of the first image detection result is less than a preset value, and to control the target camera after increasing its focal length to acquire the second monitoring video of the first monitoring position.

[0130] The second inference module 305 is used to extract the second image data of the second monitoring video, and based on the first algorithm model, use the inference framework to perform inference detection on the second image data to obtain the second image detection result.

[0131] The first sending module 306 is used to send the second image detection result to the cloud platform when the confidence level of the second image detection result is greater than the preset value;

[0132] The third control module 307 is used to control the target camera to acquire a third monitoring video at a second monitoring location, wherein the second monitoring location is one of the multiple monitoring locations other than the first monitoring location;

[0133] The third inference module 308 is used to extract the third image data of the third monitoring video, and when the confidence of the second image detection result is less than the preset value, it uses the inference framework based on the second algorithm model to perform inference detection on the third image data to obtain the third image detection result. The second algorithm model is the algorithm model called when performing other monitoring tasks other than the first monitoring task on the second monitoring location.

[0134] Optionally, the first control module includes:

[0135] The first configuration unit is used to configure the wireless network name and wireless network password;

[0136] The first control unit is configured to, when the target camera is connected to the gateway management platform based on the wireless network name and the wireless network password, control the target camera to sequentially collect monitoring videos from each of the multiple monitoring locations based on a preset time interval.

[0137] Optionally, the first extraction module includes:

[0138] The first calling unit is used to call the ffmpeg fast-forwarding dynamic image expert group open source library to pull and extract frames from the first monitoring video to obtain the raw stream data of the first monitoring video;

[0139] The first decoding unit is used to decode the raw stream data into YUY format data through a decoding chip;

[0140] The first encoding unit is used to encode the YUY format data into BGR format data through an encoding chip, wherein the first image data includes the BGR format data.

[0141] Optionally, the device further includes:

[0142] The first configuration module is used to configure first algorithm parameters based on the first monitoring task. The first algorithm parameters include at least one of the following: a first algorithm threshold, a video stream address, and an alarm interval.

[0143] The first creation module is used to create a first algorithm container based on the first algorithm parameters;

[0144] The first loading module is used to load the first algorithm model through the first algorithm container. The first algorithm model is generated based on the YOLOx network model training method.

[0145] Optionally, when at least two algorithm containers are configured at the plurality of monitoring locations, the first loading module includes:

[0146] The first sending unit is used to send a start message to the first algorithm container;

[0147] The first loading unit is used to load the first algorithm model through the first algorithm container when the first algorithm container receives the start message.

[0148] Optionally, the device further includes:

[0149] Extract the third image data from the third monitoring video, and if the confidence level of the second image detection result is greater than the preset value, perform inference detection on the third image data based on the first algorithm model and the inference framework to obtain the third image detection result.

[0150] See Figure 4 , Figure 4 This is a structural diagram of an electronic device provided in another embodiment of this application, such as... Figure 4 As shown, the electronic device includes: a processor 401, a communication interface 402, a communication bus 404, and a memory 403, wherein the processor 401, the communication interface 402, and the memory 403 interact with each other through the communication bus 404.

[0151] The system includes: a memory 403 for storing computer programs; a processor 401 for controlling the target camera to sequentially acquire monitoring videos from each of the plurality of monitoring locations based on a preset time interval; extracting first image data from the first monitoring video at a first monitoring location, where the first monitoring location is any one of the plurality of monitoring locations; performing inference detection on the first image data using an inference framework based on a first algorithm model to obtain a first image detection result, where the first algorithm model is the algorithm model called when performing a first monitoring task on the first monitoring location; controlling the target camera to increase its focal length, and controlling the target camera with the increased focal length to acquire second monitoring videos from the first monitoring location; extracting second image data from the second monitoring video, and performing the extraction when the confidence level of the first image detection result is less than a preset value. Based on the first algorithm model, the second image data is inferred and detected using the inference framework to obtain a second image detection result. If the confidence level of the second image detection result is greater than the preset value, the second image detection result is sent to the cloud platform. If the confidence level of the second image detection result is less than the preset value, the target camera is controlled to acquire a third monitoring video at a second monitoring location, where the second monitoring location is one of the multiple monitoring locations other than the first monitoring location. The third image data of the third monitoring video is extracted, and based on the second algorithm model, the third image data is inferred and detected using the inference framework to obtain a third image detection result. The second algorithm model is the algorithm model called when performing monitoring tasks other than the first monitoring task at the second monitoring location.

[0152] Optionally, the processor 401 is specifically used for:

[0153] Configure the wireless network name and password;

[0154] When the target camera connects to the gateway management platform based on the wireless network name and the wireless network password, the target camera is controlled to sequentially collect monitoring videos from each of the multiple monitoring locations based on a preset time interval.

[0155] Optionally, the processor 401 is specifically used for:

[0156] The ffmpeg fast-forwarding dynamic image expert group open source library is called to pull and extract frames from the first monitoring video to obtain the raw stream data of the first monitoring video;

[0157] The raw stream data is decoded into YUY format data using a decoding chip;

[0158] The YUY format data is encoded into BGR format data using an encoding chip, wherein the first image data includes the BGR format data.

[0159] Optionally, the processor 401 is further configured to:

[0160] Based on the first monitoring task, configure the first algorithm parameters, which include at least one of the following: a first algorithm threshold, a video stream address, and an alarm interval;

[0161] Create a first algorithm container based on the first algorithm parameters;

[0162] The first algorithm model is loaded through the first algorithm container, and the first algorithm model is generated based on the YOLOx network model training method.

[0163] Optionally, when at least two algorithm containers are configured at the plurality of monitoring locations, the processor 401 is specifically used for:

[0164] Send a start message to the first algorithm container;

[0165] Upon receiving the startup message, the first algorithm container loads the first algorithm model.

[0166] Optionally, the processor 401 is further configured to:

[0167] Extract the third image data from the third monitoring video, and if the confidence level of the second image detection result is greater than the preset value, perform inference detection on the third image data based on the first algorithm model and the inference framework to obtain the third image detection result.

[0168] The communication bus 404 mentioned in the aforementioned electronic device can be a Peripheral Component Interconnect (PCT) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus 404 can be divided into an address bus, a data bus, a control bus, etc. For ease of identification, it is represented by only one thick line in the diagram, but this does not indicate that there is only one bus or one type of data.

[0169] Communication interface 402 is used for communication between the aforementioned terminal and other devices.

[0170] The memory 403 may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory 403 may also be at least one storage device located remotely from the aforementioned processor 401. The aforementioned processor 401 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0171] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, this computer program implements the various processes of the above-described image detection method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0172] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0173] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0174] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An image detection method, characterized in that, Applied to a gateway management platform, the method includes: Based on a preset time interval, the target camera is controlled to sequentially collect monitoring videos from each of the multiple monitoring locations; Extract the first image data of the first monitoring video at the first monitoring location, where the first monitoring location is any one of the plurality of monitoring locations; Based on the first algorithm model, the first image data is inferred and detected using an inference framework to obtain the first image detection result. The first algorithm model is the algorithm model called when performing the first monitoring task on the first monitoring location. If the confidence level of the first image detection result is less than a preset value, the target camera is controlled to increase its focal length, and the target camera with increased focal length is controlled to acquire the second monitoring video of the first monitoring position. Extract the second image data from the second monitoring video, and based on the first algorithm model, use the inference framework to perform inference detection on the second image data to obtain the second image detection result; If the confidence level of the second image detection result is greater than the preset value, the second image detection result is sent to the cloud platform. The target camera is controlled to acquire a third monitoring video at a second monitoring location, where the second monitoring location is one of the multiple monitoring locations other than the first monitoring location. Extract the third image data from the third monitoring video, and if the confidence level of the second image detection result is less than the preset value, perform inference detection on the third image data based on the second algorithm model and the inference framework to obtain the third image detection result. The second algorithm model is the algorithm model called when performing other monitoring tasks besides the first monitoring task on the second monitoring location. Extract the third image data from the third monitoring video, and if the confidence level of the second image detection result is greater than the preset value, perform inference detection on the third image data based on the first algorithm model and the inference framework to obtain the third image detection result.

2. The image detection method according to claim 1, characterized in that, The step of controlling the target camera to sequentially acquire monitoring video from each of the multiple monitoring locations based on a preset time interval includes: Configure the wireless network name and password; When the target camera connects to the gateway management platform based on the wireless network name and the wireless network password, the target camera is controlled to sequentially collect monitoring videos from each of the multiple monitoring locations based on a preset time interval.

3. The image detection method according to claim 1, characterized in that, The extraction of the first image data from the first monitoring video includes: The ffmpeg fast-forwarding dynamic image expert group open source library is called to pull and extract frames from the first monitoring video to obtain the raw stream data of the first monitoring video; The raw stream data is decoded into YUY format data using a decoding chip; The YUY format data is encoded into BGR format data using an encoding chip, wherein the first image data includes the BGR format data.

4. The image detection method according to claim 1, characterized in that, Before obtaining the first image detection result by performing inference detection on the first image data using an inference framework based on the first algorithm model, the method further includes: Based on the first monitoring task, configure the first algorithm parameters, which include at least one of the following: a first algorithm threshold, a video stream address, and an alarm interval; Create a first algorithm container based on the first algorithm parameters; The first algorithm model is loaded through the first algorithm container, and the first algorithm model is generated based on the YOLOx network model training method.

5. The image detection method according to claim 4, characterized in that, When at least two algorithm containers are configured at the multiple monitoring locations, loading the first algorithm model through the first algorithm container includes: Send a start message to the first algorithm container; Upon receiving the startup message, the first algorithm container loads the first algorithm model.

6. An image detection device, characterized in that, The device includes: The first control module is used to control the target camera to sequentially collect monitoring videos from each of the multiple monitoring locations based on a preset time interval. The first extraction module is used to extract the first image data of the first monitoring video at the first monitoring location, where the first monitoring location is any one of the plurality of monitoring locations. The first inference module is used to perform inference detection on the first image data based on the first algorithm model and using an inference framework to obtain the first image detection result. The first algorithm model is the algorithm model called when performing the first monitoring task on the first monitoring location. The second control module is used to control the target camera to increase its focal length when the confidence level of the first image detection result is less than a preset value, and to control the target camera after increasing its focal length to acquire the second monitoring video of the first monitoring position. The second inference module is used to extract the second image data of the second monitoring video, and based on the first algorithm model, use the inference framework to perform inference detection on the second image data to obtain the second image detection result; The first sending module is used to send the second image detection result to the cloud platform when the confidence level of the second image detection result is greater than the preset value; The third control module is used to control the target camera to acquire a third monitoring video at a second monitoring location, where the second monitoring location is one of the multiple monitoring locations other than the first monitoring location. The third inference module is used to extract the third image data of the third monitoring video, and when the confidence of the second image detection result is less than the preset value, it uses the inference framework based on the second algorithm model to perform inference detection on the third image data to obtain the third image detection result. The second algorithm model is the algorithm model called when performing other monitoring tasks besides the first monitoring task on the second monitoring location. Extract the third image data from the third monitoring video, and if the confidence level of the second image detection result is greater than the preset value, perform inference detection on the third image data based on the first algorithm model and the inference framework to obtain the third image detection result.

7. The image detection device according to claim 6, characterized in that, The first control module includes: The first configuration unit is used to configure the wireless network name and wireless network password; The first control unit is used to control the target camera to acquire the first monitoring video at the first monitoring location when the target camera is connected to the gateway management platform based on the wireless network name and the wireless network password.

8. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the image detection method as described in any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the image detection method as described in any one of claims 1 to 5.

Images