Intelligent visual inspection system and method

Abstract

The present disclosure provides an intelligent visual inspection system and method, the system comprising: a visual model management platform, an intelligent visual inspection device and a process customization platform, the visual model management platform being configured to receive a download instruction from a user and execute a first visual model download operation; the intelligent visual inspection device being configured to identify collected image data according to one or more downloaded visual models to obtain a target recognition image; in response to determining that the target recognition image has a target feature, generating an alarm prompt message; the process customization platform being configured to determine the execution order of the selected multiple visual models in response to the received user instruction, and in response to determining that the target recognition images obtained by the selected visual model all have the target feature, record the target scene by a preset recording method, and send the recorded target scene information to a preset receiving end.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent detection technology, and in particular to an intelligent visual detection system and method. Background Art

[0002] With the advancement of science and technology and the continuous improvement of the level of new urbanization, it has become an inevitable trend to build a modern infrastructure system and accelerate the pace of digital society construction. By using edge algorithms and machine learning technologies, we can give full play to the advantages of massive data and rich application scenarios, and tap into the huge application potential of key digital technology innovations such as artificial intelligence in daily production and life scenarios.

[0003] At present, an intelligent visual detection device can only detect one type of behavior in different scenarios. For the detection of other types of behaviors in other scenarios, corresponding intelligent visual devices need to be designed, resulting in low utilization rate of intelligent visual devices and high implementation costs, which ultimately creates a gap between machine vision models and actual applications. Summary of the invention

[0004] In view of this, the purpose of the present disclosure is to provide an intelligent visual inspection system and method for solving the above problems.

[0005] Based on the above purpose, the first aspect of the present disclosure provides an intelligent visual inspection system, including: a visual model management platform, an intelligent visual inspection device and a process customization platform, wherein the intelligent visual monitoring device is connected to the visual management platform and the process customization platform by wire and / or wireless; wherein,

[0006] The visual model management platform is configured to receive a download instruction from a user and execute a first visual model download operation;

[0007] The intelligent visual detection device is configured to identify the collected image data according to the downloaded one or more visual models to obtain a target recognition image; in response to determining that the target recognition image has a target feature, generate an alarm prompt information, wherein the visual model corresponds to the target feature one by one;

[0008] The process customization platform is configured to determine the execution order of the selected multiple visual models in response to the received user instructions, and in response to determining that the target recognition images obtained by the selected visual models all have the target features, record the target scene through a preset recording method, and send the recorded target scene information to a preset receiving end.

[0009] Furthermore, the system further comprises: a data cloud processing platform, the data cloud processing platform is connected to the intelligent visual detection device by wire and / or wirelessly, and the data cloud processing platform is configured to receive the target recognition image and display it to the user in a graphical manner; and

[0010] In response to determining that the target recognition image has the target feature, a text message prompt, a phone prompt or an instant messaging software prompt is generated and sent to a preset client.

[0011] Furthermore, the intelligent visual detection device includes: a central processing module, an image sensor, a storage module and a wireless communication module, and the central processing module is connected to the image sensor, the storage module and the wireless communication module by wires; wherein,

[0012] The image sensor is configured to collect image data of a target scene;

[0013] The central processing module is configured to recognize the image data through the downloaded visual model to obtain a target recognition image;

[0014] The storage module is configured to store the image data and the downloaded visual model;

[0015] The wireless communication module is configured to enable data transmission between the central processing module and the visual model management platform and the data cloud processing platform.

[0016] Furthermore, the intelligent visual detection device further comprises: an alarm prompt module, the alarm prompt module is wiredly connected to the central processing module, and the central processing module is further configured to send the prompt instruction to the alarm prompt module in response to determining that the target feature exists in the target recognition image;

[0017] The alarm prompt module is configured to generate sound / light alarm prompt information in response to the prompt instruction of the central processing module.

[0018] Furthermore, the intelligent visual detection device further includes: a power supply module and a protection circuit module, wherein the power supply module is wiredly connected to other modules in the intelligent visual detection device, and the protection circuit module is wiredly connected to the central processing module; wherein,

[0019] The power module is configured to supply power to other modules in the intelligent visual detection device;

[0020] The protection circuit module is configured to cut off the corresponding power supply circuit when the central processing module is in the event of overvoltage or overcurrent.

[0021] Furthermore, the system also includes an interface service platform, which is wirelessly connected to the intelligent visual detection device. The interface service platform is configured to provide a corresponding interface for user operations of the intelligent visual detection device and authenticate the interface through JWT.

[0022] Furthermore, the transmission protocol of the wireless communication module is the message queue telemetry transmission MQTT protocol.

[0023] Furthermore, the visual model management platform is also configured to receive an upload instruction from a user and execute a second visual model upload operation.

[0024] Based on the same inventive concept, the second aspect of the present disclosure proposes an intelligent visual detection method, comprising:

[0025] Acquire image data of the target scene;

[0026] Based on one or more preset visual models, the image data is recognized to obtain a target recognition image;

[0027] In response to determining that the target feature exists in the target recognition image, a corresponding target recognition image is output.

[0028] Furthermore, when the target feature exists in the target recognition image, an alarm prompt information is generated.

[0029] From the above, it can be seen that the intelligent visual detection system and method provided by the present invention dynamically updates a variety of dynamic visual models through a visual model management platform. Users can download corresponding visual models to the intelligent visual detection device according to actual needs, so that the intelligent visual detection device can recognize images of various target scenes, and issue warning prompts to users for target recognition images with target features, thereby improving the utilization rate of the intelligent visual detection device and reducing the implementation cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0031] Figure 1 It is a structural schematic diagram of an intelligent visual inspection system according to an embodiment of the present disclosure;

[0032] Figure 2 It is a schematic diagram of the structure of an intelligent visual detection device according to an embodiment of the present disclosure;

[0033] Figure 3 This is a flow chart of the intelligent visual detection method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0034] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0035] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0036] As described in the background technology section, the intelligent visual detection equipment in the relevant technology is still difficult to meet the needs. An intelligent visual detection device can only detect one type of behavior in different scenarios, such as: elderly fall detection, driver seat belt detection, smart construction site safety helmet detection, smoking detection, mask detection, uncovered trash can detection, mobile phone playing detection, glove detection, reflective clothing detection. An intelligent visual detection device cannot realize two or more behavior detections, resulting in low utilization of the intelligent visual detection equipment and increasing the implementation cost of the intelligent visual detection equipment.

[0037] In view of this, the embodiments of the present disclosure provide an intelligent visual detection system. By setting up a visual model management platform, users can download the required visual models and upload the trained visual models to the visual model management platform for other users to download. Secondly, an intelligent visual detection device is also provided, which has edge computing capabilities and can perform target recognition without relying on the background system. In addition, the process customization platform enables users to select multiple visual models to execute according to the detection requirements of the actual scene, for example: after performing a driver's seat belt test, a mobile phone test is performed, and the target detection results are sent to the receiving end set by the user.

[0038] The technical solution of the present disclosure is described in detail below through specific embodiments.

[0039] refer to Figure 1The intelligent visual inspection system of an embodiment of the present invention includes: a visual model management platform 103, an intelligent visual inspection device 105 and a process customization platform 101, and the intelligent visual monitoring device 105 is connected to the visual management platform 103 and the process customization platform 101 by wire and / or wirelessly.

[0040] The visual model management platform 103 is configured to receive a download instruction from a user and execute a first visual model download operation. The user can download the corresponding visual model according to actual detection requirements, making it more convenient for the user to apply various visual models.

[0041] The intelligent visual detection device 105 is configured to identify the collected image data according to the downloaded one or more visual models to obtain a target recognition image; in response to determining that the target recognition image has a target feature, generate an alarm prompt information, wherein the visual model corresponds to the target feature one by one. The intelligent visual detection device 105 has edge computing capabilities, and there is no need to transmit the image data back to other background systems for data processing, which greatly improves the detection efficiency.

[0042] The process customization platform 101 is configured to determine the execution order of the selected multiple visual models in response to the received user instructions, and in response to determining that the target recognition images obtained by the selected visual models all have the target features, record the target scene through a preset recording method, and send the recorded target scene information to a preset receiving end.

[0043] Through the process customization platform 101, users can customize the reasoning results of the visual model in the intelligent visual monitoring device 105. For example, when a user uses a face recognition model and a smoking detection model, the smoking detection model is executed first, and the face recognition model is executed second. After the smoking behavior of a person is identified, the face recognition model is used to identify the face of the person smoking, and then the camera is used to take a photo and record it, and then send it to the administrator's mailbox. The user can issue a warning notification or deduct performance from the smoker based on the received image data. It should be noted that the recording method can also select microphone recording, or set other hardware on the intelligent visual detection device 105 according to actual needs to record the information of the target scene, and set the number and recording time of the corresponding hardware, which are not specifically limited here.

[0044] It can be seen that users can download corresponding visual models to the intelligent visual detection device 105 according to actual needs, so that the intelligent visual detection device 105 can recognize images of various target scenes and issue warning prompts to users for target recognition images with target features, thereby improving the utilization rate of the intelligent visual detection device 105 and reducing implementation costs.

[0045] In some embodiments, reference Figure 1 , the intelligent visual inspection system also includes: a data cloud processing platform 102, the data cloud processing platform 102 is wirelessly connected to the intelligent visual inspection device 105, and the data cloud processing platform 102 is configured to receive the target recognition image and display it to the user in a graphical manner; and in response to determining that the target recognition image has target features, generate a text message prompt, a phone prompt or an instant messaging software prompt and send it to a preset client. Through the data cloud processing platform 102, the user can perform a secondary analysis on the reasoning results of the visual model to further improve the accuracy of recognition.

[0046] In some embodiments, in combination Figure 2 The intelligent visual detection device includes: a central processing module 201, an image sensor 203, a storage module 202 and a wireless communication module 204, and the central processing module is connected to the image sensor, the storage module and the wireless communication module by wire.

[0047] The image sensor 203 is configured to collect image data of a target scene. Specifically, the image sensor 203 may be IMX 219, which uses a small chip to facilitate the application of small-volume devices. The sensor can accurately complete image capture.

[0048] The central processing module 201 is configured to identify the image data through the downloaded visual model to obtain a target recognition image. Specifically, the central processing module 201 can select an NVIDIA Jetson Nano small computer, which can run multiple neural network models in parallel and process images collected by multiple image sensors 203 at the same time.

[0049] The storage module 202 is configured to store the image data and the downloaded visual model. Specifically, the storage module 202 can be a micro-SD 64GB memory card.

[0050] The wireless communication module 204 is configured to enable data transmission between the central processing module 201 and the visual model management platform 103 and the data cloud processing platform 102. Specifically, the wireless communication module 204 can select a 5G module. The 5G module is small in size, has a wide working range, and can transmit data stably.

[0051] In some embodiments, in combination Figure 2The intelligent visual detection device 105 also includes: an alarm prompt module 205, which is wired to the central processing module 201, and the central processing module 201 is also configured to send the prompt instruction to the alarm prompt module 205 in response to determining that the target feature exists in the target recognition image.

[0052] The alarm prompt module 205 is configured to generate sound / light alarm prompt information in response to the prompt instruction of the central processing module 201. Specifically, the alarm prompt module 205 can select LED lights and / or VS1053 players, and the VS1053 player can support user-defined prompt sounds.

[0053] When the user chooses to provide feedback on the intelligent visual inspection device 105, after receiving the prompt instruction from the central processing module 201, the alarm prompt module 205 flashes the LED light and the VS1053 player plays the default prompt sound or the user-defined prompt sound.

[0054] In some embodiments, in combination Figure 2 The intelligent visual detection device 105 also includes: a power supply module 207 and a protection circuit module 206, the power supply module 207 is wiredly connected to other modules in the intelligent visual detection device 105, and the protection circuit module is wiredly connected to the central processing module.

[0055] The power supply module 207 is configured to supply power to other modules in the intelligent visual detection device 105 ; the protection circuit module 206 is configured to cut off the corresponding power supply circuit when the central processing module 201 is in the case of overvoltage or overcurrent.

[0056] In some embodiments, in combination Figure 1 The intelligent visual inspection system further includes an interface service platform 104, the interface service platform 104 is wirelessly connected to the intelligent visual inspection device 105, and the interface service platform 104 is configured to provide a corresponding interface for user operations of the intelligent visual inspection device 105, and authenticate the interface through JWT. User operations include but are not limited to: registering a device, removing a device, and issuing instructions.

[0057] In some embodiments, the transmission protocol of the wireless communication module 204 is the Message Queuing Telemetry Transport (MQTT) protocol.

[0058] MQTT is a client-server based message publishing / subscribing transmission protocol, which is suitable for restricted environments (such as weak network signals and low hardware performance). The MQTT protocol provides network connection based on the TCP / IP protocol, so it has strong compatibility.

[0059] In some embodiments, the visual model management platform is further configured to receive an upload instruction from a user and perform a second visual model upload operation. The user uploads the visual model trained by the user to the visual model management platform and sets the visibility permissions for other users.

[0060] In some embodiments, the intelligent visual inspection system includes a user management platform, which is configured to respond to user requests, authenticate users, and store user account information.

[0061] It should be noted that the above describes some embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0062] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment systems, an intelligent visual detection method proposed in the present disclosure is executed using the systems described in the above-mentioned embodiments.

[0063] refer to Figure 3 , the intelligent visual detection method may include the following steps:

[0064] Step S301, acquiring image data of a target scene.

[0065] Step S302: Recognize the image data based on one or more preset visual models to obtain a target recognition image.

[0066] Step S303: in response to determining that the target recognition image has target features, outputting a corresponding target recognition image.

[0067] As an optional embodiment, the aforementioned step S103 may further include:

[0068] When the target feature exists in the target recognition image, an alarm prompt information is generated.

[0069] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Based on the concept of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present disclosure as described above, which are not provided in detail for the sake of simplicity.

[0070] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present disclosure difficult to understand, the known power / ground connections to the integrated circuit (IC) chips and other components may or may not be shown in the provided figures. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present disclosure difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present disclosure will be implemented (that is, these details should be fully within the scope of understanding of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present disclosure, it is apparent to those skilled in the art that the embodiments of the present disclosure can be implemented without these specific details or with changes in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0071] Although the present disclosure has been described in conjunction with specific embodiments of the present disclosure, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0072] The embodiments of the present disclosure are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. An intelligent visual inspection system, characterized in that: include: A visual model management platform, an intelligent visual detection device and a process customization platform, wherein the intelligent visual detection device is connected to the visual management platform and the process customization platform by wire and / or wireless; wherein, The visual model management platform is configured to receive a download instruction from a user and execute a first visual model download operation; The intelligent visual detection device is configured to identify the collected image data based on the downloaded one or more visual models to obtain a target recognition image; and generate an alarm prompt message in response to determining that the target recognition image contains a target feature, wherein the visual model corresponds to the target feature in a one-to-one manner; The process customization platform is configured to, in response to a received user instruction, determine an execution order of the selected multiple visual models, and, in response to determining that the target recognition images obtained by the selected visual models all contain the target features, record the target scene using a preset recording method, and send the recorded target scene information to a preset receiving end; Also included: a data cloud processing platform, the data cloud processing platform is connected to the intelligent visual detection device by wire and / or wirelessly, and the data cloud processing platform is configured to receive the target recognition image and display it to the user in a graphical manner; and In response to determining that the target feature exists in the target recognition image, a text message prompt, a phone prompt, or an instant messaging software prompt is generated and sent to a preset client.

2. The system according to claim 1, wherein: The intelligent visual detection device includes: a central processing module, an image sensor, a storage module and a wireless communication module, wherein the central processing module is connected to the image sensor, the storage module and the wireless communication module by wires; wherein, The image sensor is configured to collect image data of a target scene; The central processing module is configured to recognize the image data using the downloaded visual model to obtain a target recognition image; The storage module is configured to store the image data and the downloaded vision model; The wireless communication module is configured to enable data transmission between the central processing module, the visual model management platform and the data cloud processing platform.

3. The system according to claim 2, characterized in that The intelligent visual detection device further includes: an alarm prompt module, the alarm prompt module being connected to the central processing module by wire, the central processing module being further configured to send a prompt instruction to the alarm prompt module in response to determining that the target feature exists in the target recognition image; The alarm prompt module is configured to generate sound / light alarm prompt information in response to the prompt instruction of the central processing module.

4. The system according to claim 3, characterized in that The intelligent visual detection device further includes: a power supply module and a protection circuit module, wherein the power supply module is connected to other modules in the intelligent visual detection device by wire, and the protection circuit module is connected to the central processing module by wire; wherein, The power module is configured to supply power to other modules in the intelligent visual detection device; The protection circuit module is configured to cut off the corresponding power supply circuit when the central processing module is in the event of overvoltage or overcurrent.

5. The system according to claim 1, wherein: It also includes an interface service platform, which is wirelessly connected to the intelligent visual detection device. The interface service platform is configured to provide a corresponding interface for user operations of the intelligent visual detection device and authenticate the interface through JWT.

6. The system according to claim 2, wherein: The transmission protocol of the wireless communication module is the message queue telemetry transmission MQTT protocol.

7. The system according to claim 1, wherein: The visual model management platform is further configured to receive an upload instruction from a user and execute a second visual model upload operation.

8. An intelligent visual detection method, characterized in that: Executed using the system according to any one of claims 1 to 7, including: Acquire image data of the target scene; Recognize the image data based on one or more preset visual models to obtain a target recognition image; In response to determining that the target feature exists in the target recognition image, a corresponding target recognition image is output.

9. The method according to claim 8, characterized in that Also includes: When the target feature exists in the target recognition image, an alarm prompt message is generated.

Images