Inspection method and inspection system

Abstract

The present invention provides an inspection method and an inspection system. The inspection method comprises: acquiring relative position information between a plurality of image capturing devices; on the basis of a target event and the relative position information, determining an inspection route satisfying the target event, wherein the plurality of image capturing devices comprised in the inspection route are set as a plurality of inspection devices; and on the basis of the inspection route, controlling real-time video signals of the inspection devices to be presented to a display device.

Description

Inspection method and inspection system Technical Field

[0001] The present invention relates to a monitoring method and system, and in particular to an inspection method and system based on multiple imaging devices. Background Art

[0002] Traditional surveillance systems using multiple cameras display each camera's image separately, limiting viewing angles and locations to fixed positions and preventing the ability to link multiple camera images. Furthermore, the sheer number of cameras creates a plethora of images, making it difficult for users to locate their target. When switching between camera views, the difference in viewing angle between cameras prevents users from experiencing a fully immersive experience.

[0003] Another drawback of 360-degree camera surveillance systems is their high degree of freedom, requiring users to manually turn the camera to locate the target. Even with multiple 360-degree cameras, users still have to select a single camera to view. Traditional surveillance systems are unable to quickly locate a specific target.

[0004] Summary of the Invention

[0005] The present invention provides an inspection method and an inspection system, which can condense and watch real-time video signals that meet target events.

[0006] The inspection method of the present invention is suitable for execution using an electronic device. The inspection method includes: obtaining relative position information between multiple imaging devices; determining an inspection route based on a target event and the relative position information, wherein the inspection route satisfies the target event, and multiple imaging devices passed through the inspection route are set as multiple inspection devices; and controlling the real-time video signals of each inspection device to be presented to a display device based on the inspection route.

[0007] According to an embodiment of the present invention, the above-mentioned inspection method also includes: establishing relative position information between the imaging devices, including: providing a plan view corresponding to the space in which the imaging device is set; based on user operation, marking the actual position of the imaging device set in the space in the plan view corresponding to multiple planar positions of the plan view; and calculating the relative position information between the imaging devices based on the planar positions.

[0008] According to an embodiment of the present invention, the above-mentioned inspection method also includes: establishing relative position information between the imaging devices, including: obtaining multiple images corresponding to the imaging devices from the imaging devices respectively; and calculating the relative position information between the imaging devices by finding corresponding feature points in each two images.

[0009] According to an embodiment of the present invention, the target event includes an event indicating that a designated object has been captured. The step of determining an inspection route includes: executing an artificial intelligence (AI) model to perform a target detection algorithm on the real-time video signal received by each imaging device to determine whether the imaging device has captured the designated object; and, in response to multiple target devices in the imaging device capturing the designated object, determining an inspection device based on relative position information and the target device capturing the designated object, wherein the number of inspection devices included in the inspection route is greater than or equal to the number of target devices.

[0010] According to an embodiment of the present invention, after determining whether the imaging device has captured the designated object, it also includes: in response to only the first imaging device among the imaging devices capturing the designated object, based on the relative position information and the first imaging device that captured the designated object, determining an inspection route, wherein the inspection route includes at least the first imaging device and the second imaging device corresponding to the predetermined position.

[0011] According to an embodiment of the present invention, after determining whether the imaging device has captured a designated object, the method further includes: in response to the designated object being a device, after detecting the presence of the device in the real-time video signal through the execution of a target detection algorithm by an artificial intelligence model, obtaining real-time information about the device and recording the real-time information. The step of controlling the real-time video signal of each inspection device to be presented to a display device also includes: in response to the presence of the designated object in the real-time video signal, simultaneously presenting the corresponding real-time information on the display device when the real-time video signal is presented to the display device.

[0012] According to an embodiment of the present invention, after determining whether the imaging device has captured a designated object, the system further includes: in response to the designated object being a human body, after detecting the presence of a human body in the real-time video signal through the execution of a target detection algorithm by an artificial intelligence model, determining whether the human body is in a dangerous state through the artificial intelligence model, and recording a warning message when the human body is determined to be in a dangerous state. The step of controlling the real-time video signal of each inspection device to be presented to the display device also includes: in response to the presence of a designated object in the real-time video signal and the designated object having a warning message, simultaneously presenting the warning message on the display device when the real-time video signal is presented to the display device.

[0013] According to an embodiment of the present invention, after determining whether the imaging device has captured a designated object, the method further includes: in response to the designated object being a human body, after detecting the presence of a human body in the real-time video signal through the execution of a target detection algorithm by the artificial intelligence model, generating a selection frame for selecting the human body through the artificial intelligence model. The step of controlling the real-time video signal of each inspection device to be presented to the display device also includes: in response to the presence of the designated object in the real-time video signal, simultaneously presenting a selection frame on the display device to select the human body when the real-time video signal is presented to the display device.

[0014] According to an embodiment of the present invention, the above-mentioned step of controlling the real-time video signal of each inspection device to be presented to the display device based on the inspection route includes: switching the display screen of the display device from the real-time video signal of the first inspection device in the inspection device to the real-time video signal of the second inspection device in the inspection device that captures the specified object, which includes: controlling the first inspection device to turn to the first direction toward the second inspection device to take an image, and controlling the second inspection device to turn to the first direction; presenting the real-time video signal of the first inspection device facing the first direction to the display screen; performing a zoom operation on the real-time video signal of the first inspection device in the display screen; and after performing the zoom operation, controlling the second inspection device to turn from the first direction to the second direction toward the specified object to take an image, and synchronously switching the display screen to the real-time video signal of the second inspection device during the turning process of the second inspection device.

[0015] According to an embodiment of the present invention, the target event includes an event indicating an inspection of at least one work area. The step of determining an inspection route includes: selecting at least one target device corresponding to the at least one work area in the imaging device; and determining the inspection device based on the relative position information and the at least one target device.

[0016] According to an embodiment of the present invention, the above-mentioned inspection method also includes: determining the inspection devices included in the inspection route based on the target event and at least another target event; and determining the inspection order of the inspection devices based on the relative position information with reference to the event sequence of the target event and at least another target event.

[0017] According to an embodiment of the present invention, the step of determining the inspection route includes: determining the inspection order of the inspection devices based on the relative position information and the priority of the imaging devices.

[0018] According to an embodiment of the present invention, in the process of displaying the video signals of each inspection device on the display device in sequence based on the inspection order, it also includes: in response to detecting a new event that meets the target event, reselecting multiple devices from the imaging device as multiple new inspection devices; based on relative position information, the imaging device corresponding to the video signal currently displayed by the display device is used as the inspection starting point, and the new inspection route of the new inspection device is re-determined; and based on the new inspection route, the real-time video signals of each of the new inspection devices are controlled to be presented to the display device.

[0019] According to an embodiment of the present invention, the inspection method further includes: providing an inspection result interface to a display device, wherein the inspection result interface includes a video block, a floor plan block, an inspection screenshot block, and an information block. The video block is used to play a real-time video signal in real time. The floor plan block is used to display a floor plan corresponding to the space where the imaging device is located, and the floor plan includes a plurality of position information corresponding to the floor plan corresponding to the actual position of the imaging device in the space and a trajectory based on the inspection sequence. The inspection screenshot block is used to display a screenshot corresponding to the target event. The information block is used to display real-time information corresponding to the target event.

[0020] According to an embodiment of the present invention, in the process of controlling the real-time video signals of each inspection device to be presented to a display device based on an inspection route, in response to receiving a position selected in the real-time video signal presented on the display device, real-time information corresponding to the specified object or work area included in the position is simultaneously presented on the display device.

[0021] The inspection system of the present invention includes: a plurality of imaging devices; a display device; and a processor coupled to the imaging devices and the display device. The processor is configured to: obtain relative position information between the plurality of imaging devices; determine an inspection route based on a target event and the relative position information, wherein the inspection route satisfies the target event and the plurality of imaging devices passing through the inspection route are defined as a plurality of inspection devices; and control, based on the inspection route, to display real-time video signals from each of the inspection devices to the display device.

[0022] Based on the above, the present invention provides an inspection method and system that selects a device that meets a target event from multiple imaging devices, generates an inspection route based on the selected device, and then displays the content captured by the imaging device based on the inspection route. This allows for a concentrated viewing of real-time video signals that meet the target event. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings are included to provide a further understanding of the present invention and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments of the present invention and together with the description serve to explain the principles of the present invention.

[0024] FIG1 is a schematic diagram of an inspection system according to an embodiment of the present invention.

[0025] FIG. 2 is a schematic diagram of the architecture of an inspection system according to an embodiment of the present invention.

[0026] FIG3 is a flow chart of an inspection method according to an embodiment of the present invention.

[0027] FIG. 4 is a schematic diagram of a plan view according to an embodiment of the present invention.

[0028] 5A to 5C are schematic diagrams of setting an inspection sequence according to an embodiment of the present invention.

[0029] FIG6 is a schematic diagram of setting an inspection route according to an embodiment of the present invention.

[0030] FIG. 7 is a schematic diagram of an inspection movement method according to an embodiment of the present invention.

[0031] FIG. 8 is a schematic diagram of an inspection movement method according to an embodiment of the present invention.

[0032] FIG. 9 is a schematic diagram of a display screen according to an embodiment of the present invention.

[0033] FIG. 10 is a schematic diagram of a display screen according to an embodiment of the present invention.

[0034] FIG. 11 is a schematic diagram of an inspection result interface according to an embodiment of the present invention.

[0035] FIG. 12 is a schematic diagram of a display screen according to an embodiment of the present invention.

[0036] FIG. 13 is a schematic diagram of a display screen according to an embodiment of the present invention.

[0037] FIG. 14 is a schematic diagram of an inspection route according to an embodiment of the present invention.

[0038] FIG. 15 is a schematic diagram of an inspection route according to an embodiment of the present invention.

[0039] Explanation of Figure Numbers

[0040] 100: Inspection system

[0041] 110: Processor

[0042] 120: Memory

[0043] 130: Display device

[0044] 140, 140-1 to 140-N, 4C1 to 4C5, 5C1 to 5C4, 6C1 to 6C5, 7C1 to 7C2, 8C1 to 8C3, 14C1 to 14C4, 15C1 to 15C2: Imaging device

[0045] 100A: Electronic devices

[0046] 210: Streaming Server

[0047] 220: Artificial Intelligence Model

[0048] 230: Receiving device

[0049] 241: Event Server

[0050] 250: Inspection module

[0051] 400: Floor Plan

[0052] 71d, 72d, 81d, 82d, 83d, 14d1-14d6, 15d1-15d2: Direction

[0053] 900, 1000, 1200, 1300: Display screen

[0054] 910-930, 1210-1250: Text box

[0055] 1010-1040, 12F1-12F5, 12W1-12W2, 13W1-13W3: Select box

[0056] 1100: Inspection results interface

[0057] 1110: Video Block

[0058] 1120: Floor plan block

[0059] 1130: Inspection screenshot block

[0060] 1140: Information block

[0061] 1510, 1520: Equipment

[0062] T: Object

[0063] U, U1, U2, 14U1 to 14U4, 15U1 to 15U4: User

[0064] V2~VN: Real-time video signal

[0065] S305~S315: Steps of inspection method DETAILED DESCRIPTION

[0066] FIG1 is a schematic diagram of an inspection system according to one embodiment of the present invention. Referring to FIG1 , inspection system 100 includes a processor 110, a memory 120, a display device 130, and N (N is an integer greater than or equal to 2) imaging devices 140 - 1 through 140 -N (collectively, imaging devices 140 ). Processor 110 is coupled to memory 120, display device 130, and imaging devices 140 - 1 through 140 -N.

[0067] In this embodiment, the processor 110, memory 120, and display device 130 may be integrated into the same electronic device 100A. The electronic device 100A may be, for example, a smartphone, tablet, laptop, personal computer, car navigation system, or other computing device. The imaging devices 140-1 through 140-N are connected to the electronic device 100A via wired or wireless communication, enabling data transmission between the imaging devices 140-1 through 140-N and the processor 110.

[0068] In another embodiment, the processor 110 and the memory 120 may be integrated into a single electronic device with computing capabilities, such as a smartphone, tablet computer, laptop computer, personal computer, or car navigation system. The display device 130 and the imaging devices 140 - 1 to 140 -N are connected to the electronic device via wired or wireless communication.

[0069] The processor 110 is, for example, a central processing unit (CPU), a graphics processing unit (GPU), a physical processing unit (PPU), a programmable microprocessor, an embedded control chip, a digital signal processor (DSP), an application specific integrated circuit (ASIC), or other similar devices.

[0070] The memory 120 may be, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, a hard disk, or other similar device, or a combination of these devices. The memory 120 may also include one or more code segments. Once installed, these code segments cause the processor 110 to execute the inspection method described below.

[0071] The display device 130 is implemented by, for example, a liquid crystal display (LCD), a plasma display (Plasma Display), an organic light-emitting diode (OLED) display, a projection system, etc.

[0072] The imaging devices 140-1 to 140-N are cameras, cameras, etc. that use charge coupled device (CCD) lenses, complementary metal oxide semiconductor transistors (CMOS) lenses. For example, the imaging devices 140-1 to 140-N are omnidirectional cameras. A panoramic camera (also known as a 360-degree camera) is a camera whose imaging perspective can cover the entire sphere or at least can cover a circular field of view on the horizontal plane. Its types include full spherical panoramic cameras and semi-spherical panoramic cameras. In addition, the imaging devices 140-1 to 140-N can also be wide-angle cameras. In actual applications, multiple imaging devices 140-1 to 140-N are deployed in a space, and then a monitoring network is established based on the relationship between the imaging devices 140-1 to 140-N.

[0073] FIG2 is a schematic diagram illustrating the architecture of an inspection system according to one embodiment of the present invention. Referring to FIG2 , the inspection system 100 further includes a streaming server 210, an artificial intelligence (AI) model 220, a receiving device 230, an event server 240, and an inspection module 250. The streaming server 210 is a separate server from the electronic device 100A and communicates with the electronic device 100A via a wired or wireless connection. The streaming server 210 is used to store real-time video signals from the imaging devices 140-1 to 140-N and transmit the real-time video signals to the inspection module 250.

[0074] The artificial intelligence model 220 is an application program comprised of one or more code segments installed in the memory 120 of the electronic device 100A and executed by the processor 110. The artificial intelligence model 220 uses an object detection algorithm to detect real-time video signals received by each imaging device 140, thereby determining whether the imaging device 140 has captured a designated object. Alternatively, the artificial intelligence model 220 may be installed in another electronic device different from the electronic device 100A, which establishes a communication connection with the electronic device 100A via a wired or wireless method.

[0075] The inspection module 250 is an application program composed of one or more code segments and stored in the memory 120 of the electronic device 100A. The inspection module 250 is executed by the processor 110 to implement the inspection method described below.

[0076] The receiving device 230 is used to receive real-time information from the equipment and transmit the real-time information to the event server 240 for storage. The receiving device 230 can be a sensor or programmable logic controller (PLC) installed on each equipment to monitor the operation status of the equipment in real time.

[0077] Event server 240 can be a database system installed in electronic device 100A, used to store real-time information transmitted by receiving device 230 and the recognition results of artificial intelligence model 220. Alternatively, event server 240 can be a separate server from electronic device 100A, communicating with electronic device 100A via wired or wireless means. Event server 240 provides the recognition results of artificial intelligence model 220 and / or real-time information obtained by receiving device 230 to inspection module 250 as required by inspection module 250.

[0078] The following describes the steps of the inspection method in conjunction with the inspection system 100. FIG3 is a flow chart of the inspection method according to one embodiment of the present invention. Referring to FIG1 to FIG3, in step S305, the processor 110 obtains relative position information between the plurality of imaging devices 140-1 to 140-N.

[0079] In one embodiment, the processor 110 obtains a plan view corresponding to the space in which the imaging devices 140-1 to 140-N are located. This plan view includes multiple position information corresponding to the actual positions of the imaging devices 140-1 to 140-N in the space. Specifically, the processor 110 may display this plan view corresponding to the space on the display device 130 and, in response to user input such as a keyboard, mouse, or touchpad, mark the positions of the imaging devices 140-1 to 140-N on the plan view. Subsequently, the processor 110 obtains relative position information between the imaging devices 140-1 to 140-N based on this position information.

[0080] For example, Figure 4 is a schematic diagram of a plan view according to one embodiment of the present invention. In Figure 4, five imaging devices 4C1-4C5 are used for illustration. Plan view 400 can be a simple plan view or a spatial design drawing in a format such as DXF or DWG created using computer-aided design (CAD) software.

[0081] Referring to Figure 4 , a user can manually set the actual spatial positions of imaging devices 4C1-4C5 relative to the planar positions of plan view 400. For example, imaging device 4C1 can be positioned at a doorway, while imaging device 4C2 can be positioned at the corner of the entrance. After determining the planar positions of imaging devices 4C1-4C5 within plan view 400, processor 110 can calculate relative position information between imaging devices 4C1-4C5 based on the planar positions of imaging devices 4C1-4C5 within plan view 400. For example, imaging device 4C2 can determine the relative position of imaging device 4C1.

[0082] Alternatively, the processor 110 can automatically calculate the relative position information between the imaging devices 4C1-4C5 based on the images captured by each of the imaging devices 4C1-4C5. For example, the processor 110 obtains corresponding multiple images from the imaging devices 4C1-4C5 (one image captured by each imaging device) and calculates the relative position information between the imaging devices 4C1-4C5 by finding corresponding feature points between each pair of images. For example, assuming that the imaging ranges of the imaging devices 4C1 and 4C2 cover the same area, by finding the same feature values ​​in the two images, the corresponding relationship between the two can be determined. For example, the relative position of the imaging device 4C2 relative to the imaging device 4C1 can be determined.

[0083] The scale-invariant feature transform (SIFT) method or the optical flow method can be used to find the feature points of the same target object in the two images. The two images are then rotated, translated, zoomed in, and zoomed out to match the feature points of the target object in the two images, thereby obtaining the relative position information between the imaging device 4C1 and the imaging device 4C2. Each imaging device 140 is perspectively projected 90 degrees from front to back, left to right, and then a corresponding relationship is found between these perspective-projected images. For example, the relative position information can be a homography transformation matrix between the two imaging devices. The homography transformation matrix can be used to determine the angle between the imaging devices 4C1 and 4C2, as well as the distance between them.

[0084] In addition, the above two methods can also be combined to obtain relative position information. For example, after obtaining relative position information using the plan view, the corresponding angle of the plan view marking method is used to perform perspective projection and then perform comparison.

[0085] Then, in step S310, the processor 110 determines the inspection route based on the target event and the relative position information. Here, the determined inspection route will satisfy the target event, and the multiple imaging devices passed through in the inspection route are set as multiple inspection devices. That is, the selected multiple inspection devices can satisfy the content of the target event. For example, the target event can be an event indicating that a specified object is photographed. For example, the specified object can be a human body, an animal, a plant, a home appliance, an electronic instrument, equipment, a building material, etc. Alternatively, the target event can also be an event indicating at least one working area (such as a test area, a production area, a packaging area). The processor 110 can further determine the inspection order of the inspection devices based on the relative position information and the priority of the imaging devices 140-1 to 140-N. In other embodiments, the user can also manually set the inspection order of the inspection devices.

[0086] For a target event indicating the capture of a designated object, the processor 110 executes an artificial intelligence model 220 to perform a target detection algorithm on the real-time video signals received by each imaging device 140 to determine whether each imaging device 140 captured the designated object. In response to multiple target devices among imaging devices 140-1 to 140-N capturing the designated object, multiple inspection devices are determined based on relative position information and the target devices capturing the designated object. The number of inspection devices included in the inspection route is greater than or equal to the number of target devices.

[0087] In addition, in response to only one imaging device (the first imaging device) capturing the designated object, at least two inspection devices are determined based on the relative position information and the first imaging device capturing the designated object. That is, the inspection route includes at least the first imaging device and the second imaging device corresponding to the predetermined position.

[0088] The following description is based on the configuration of the imaging devices corresponding to the architecture of FIG. 2 and the plan view 400 of FIG. 4 , and assumes that the designated object is a human body. The processor 110 uses the artificial intelligence model 220 to execute a target detection algorithm on the real-time video signals of each of the imaging devices 4C1 to 4C5, thereby determining whether the imaging devices 4C1 to 4C5 have captured the designated object. As shown in FIG. 4 , the artificial intelligence model 220 determines that the imaging devices 4C2 and 4C5 have captured users U1 and U2, respectively. Next, the processor 110 further determines the inspection devices in the inspection route based on the relative position information and the imaging devices 4C2 and 4C5. Here, the processor 110 sets the imaging devices 4C2 and 4C5 as inspection devices. Furthermore, since the shooting ranges of the imaging device 4C2 and the imaging device 4C5 do not overlap, the processor 110 further selects the imaging device 4C3 and the imaging device 4C4 as inspection devices based on a route setting rule (for example, along the aisle of the space or along the route of the production line, or the priority of the imaging devices 4C1 to 4C5) and relative position information.

[0089] In addition, in this embodiment, a predetermined location is further set as the starting or ending location for the inspection. For example, the "entrance" of the inspection space is set as the predetermined location. In Figure 4, the predetermined location is the doorway, which corresponds to imaging device 4C1. Processor 110 further associates the predetermined location with imaging device 4C1 and sets it as the inspection device. Based on the relative position information, processor 110 then determines the inspection route, with the inspection order being imaging device 4C1, imaging device 4C2, imaging device 4C3, imaging device 4C4, and imaging device 4C5.

[0090] After the inspection order of imaging devices 4C1-4C5 is set, the processor 110 can further control the display screen to rotate toward the direction of users U1 and U2 during the inspection process. For example, in Figure 4, the display screen of display device 130 switches from the real-time video signal of imaging device 4C1 to the real-time video signal of imaging device 4C2, then rotates toward the direction of user U1, and then sequentially switches to the real-time video signals of imaging devices 4C3, 4C4, and 4C5, before finally rotating toward the direction of user U2.

[0091] Furthermore, if only one imaging device (e.g., imaging device 4C1) captures a designated object, to achieve a patrol inspection effect, processor 110 may designate imaging device 4C2 that captured the designated object and imaging device 4C1 corresponding to a predetermined location (e.g., an entrance corresponding to the space) as patrol inspection devices. Processor 110 then determines a patrol inspection route that includes imaging device 4C1 and imaging device 4C2 (as patrol inspection devices) based on the relative position information. Alternatively, in response to only one imaging device (the first imaging device) capturing the designated object, processor 110 may determine at least three patrol inspection devices based on the relative position information, the first imaging device that captured the designated object, and the second and third imaging devices corresponding to two predetermined locations (the patrol start location and the patrol end location).

[0092] Furthermore, returning to FIG. 1 , assuming that the target event is an event indicating an inspection of at least one work area, the processor 110 selects the imaging devices corresponding to one or more designated work areas as target devices based on the distribution of the imaging devices 140 - 1 to 140 -N; and determines the inspection devices based on the relative position information and the target devices. A predetermined position may be further set as the inspection start position or the inspection end position. The processor 110 determines multiple inspection devices based on the relative position information, the target devices, and the imaging devices corresponding to the predetermined positions (the inspection start position or the inspection end position).

[0093] After determining the inspection device, the processor 110 further determines the inspection order. For example, the inspection order can be determined by moving clockwise, counterclockwise, at the minimum rotation angle, or at the shortest path. For example, Figures 5A to 5C are schematic diagrams of setting the inspection order according to an embodiment of the present invention. Please refer to Figures 5A to 5C. In this embodiment, it is assumed that there are four imaging devices 5C1 to 5C4 that meet the target event. The inspection order of the imaging devices 5C1 to 5C4 can be counterclockwise movement as shown in Figure 5A, or clockwise movement as shown in Figure 5B. In addition, as shown in Figure 5C, the inspection order of the imaging devices 5C1 to 5C4 can be set based on each imaging device rotating at the minimum angle to transition to the next one. Alternatively, the inspection order of the imaging devices 5C1 to 5C4 can also be set based on the shortest motion path.

[0094] Furthermore, to connect two imaging devices that meet the target event, an imaging device that does not meet the target event may also be selected as a patrol device. For example, if the imaging ranges of two patrol devices (imaging devices that meet the target event) do not overlap, the transition from the real-time video signal of one patrol device to the real-time video signal of the other patrol device will cause a discontinuity in the image. Therefore, to connect the two patrol devices, at least one imaging device between the two patrol devices may be further selected as a patrol device.

[0095] FIG6 is a schematic diagram illustrating setting up an inspection route according to an embodiment of the present invention. Referring to FIG6 , assume that three imaging devices 6C1 through 6C3 meet the target event. Assume that the imaging ranges of imaging device 6C1 and imaging device 6C2 do not overlap, and that the imaging ranges of imaging device 6C1 and imaging device 6C3 do not overlap. Therefore, in addition to designating imaging devices 6C1 through 6C3 as inspection devices, imaging device 6C4 can be selected between imaging device 6C1 and imaging device 6C2 as an inspection device, and imaging device 6C5 can be selected between imaging device 6C1 and imaging device 6C3 as an inspection device.

[0096] Furthermore, when there are multiple target events, the inspection order can also be determined based on the order of the target events. The processor 110 determines the multiple inspection devices included in the inspection route based on the multiple target events. Then, based on the order of the target events and the relative position information, the inspection order of the multiple inspection devices is determined. For example, if the target events include a first event of photographing a human body and a second event of photographing a designated device, and the order of the first event takes precedence over the order of the second event, the inspection device that meets the first event will be placed before the inspection device that meets the second event.

[0097] After the inspection route is determined, in step S315, the processor 110 controls the display device 130 to display the real-time video signals of each inspection device based on the inspection route. That is, based on the inspection order, the processor 110 switches the display screen of the display device 130 from the real-time video signal of the first inspection device to the real-time video signal of the second inspection device, then switches the display screen of the display device 130 to the real-time video signal of the third inspection device, and so on, until the display screen of the display device 130 switches to the real-time video signal of the last inspection device.

[0098] The switching between the two real-time video signals can be combined with transition effects to make the displayed images visually coherent.

[0099] Figure 7 is a schematic diagram of an inspection movement method according to an embodiment of the present invention. Referring to Figure 7 , this embodiment is described using the imaging device 7C1 performing an inspection toward the imaging device 7C2. Furthermore, the description assumes that an object T is located in direction 72d, but this is not limiting. The processor 110 controls the imaging device 7C1 to rotate from direction 71d to capture images in direction 72d, which is toward the imaging device 7C2, and controls the imaging device 7C2 to face direction 72d. Next, the processor 110 displays the real-time video signal V1 obtained by the imaging device 7C1 in direction 72d on a display screen, controls the imaging device 7C1 to perform a zoom-in operation, and sequentially displays real-time video signals V2 to VN on the display screen. The display screen then switches to the real-time video signal from the imaging device 7C2. This ensures a coherent visual presentation.

[0100] In addition to amplifying the real-time video signal of image capture device 7C1 to its maximum limit and then transitioning to the real-time video signal of image capture device 7C2, a patrol method can also be used to freely move within a certain distance (limited range) from the center of the field of view of an image capture device. For example, in FIG7 , during the transition from image capture device 7C1 to image capture device 7C2, the display screen can also move within the limited range of image capture device 7C1 toward image capture device 7C2. After reaching the limit, the real-time video signal of image capture device 7C1 is amplified to the limit through the aforementioned amplification operation, and then transitioned to the real-time video signal of image capture device 7C2. This approach allows viewers to feel more immersed, as if they are walking in the scene.

[0101] Figure 8 is a schematic diagram of an inspection movement method according to an embodiment of the present invention. Referring to Figure 8 , this embodiment illustrates the inspection sequence of imaging device 8C1, imaging device 8C2, and imaging device 8C3, with user U located at imaging device 8C2. Processor 110 controls imaging device 8C1 to rotate toward direction 81d toward imaging device 8C2 to capture images, and controls imaging device 8C2 to rotate toward direction 81d. Next, processor 110 displays the real-time video signal from imaging device 8C1 in direction 81d on a display screen and amplifies the real-time video signal from imaging device 8C1 in the display screen. After amplifying to the maximum limit, processor 110 controls imaging device 8C2 to rotate from direction 81d toward direction 82d toward a designated object (i.e., user U) to capture images. During the rotation of imaging device 8C2, the display screen is synchronously switched to the real-time video signal from imaging device 8C2. Consequently, the display screen presents a visual effect of a rotation from direction 81d to 82d. The processor 110 then controls the image capturing device 8C2 to rotate from direction 82d toward direction 83d toward the image capturing device 8C3 to capture images, and displays the real-time video signal captured by the image capturing device 8C2 on the display screen. The processor 110 then displays the real-time video signal from the image capturing device 8C2 in direction 83d on the display screen and amplifies the real-time video signal from the image capturing device 8C2 in the display screen. After the amplification operation reaches its maximum limit, the display screen is switched to the real-time video signal from the image capturing device 8C3.

[0102] During the inspection process, the processor 110 may further utilize the artificial intelligence model 220 to obtain real-time information about a specified object, or may receive real-time information about a device from the receiving device 230 and further present it on a display screen. For example, real-time information may be presented using an on-screen display (OSD), a warning light, a pop-up notification, the Internet of Things (IoT), a manufacturing execution system (MES), or the like.

[0103] In response to the designated object being a device, after the AI ​​model 220 executes an object detection algorithm and detects the presence of the device in the real-time video signal, the processor 110 obtains and records the real-time information of the device from the receiving device 230. Subsequently, when controlling the display screen to present the real-time video signal from each imaging device, in response to the presence of the designated object (device) in the real-time video signal, the corresponding real-time information is simultaneously displayed on the display device 130 when the real-time video signal is presented to the display device 130.

[0104] FIG9 is a schematic diagram of a display screen according to an embodiment of the present invention. Referring to FIG9 , display screen 900 currently displays a real-time video signal representing three devices (a pickling device 1, a pickling device 2, and a degreasing tank). Accordingly, processor 110 simultaneously displays three text boxes 910 - 930 on display screen 900 to display real-time information corresponding to the three devices, respectively.

[0105] In another embodiment, during the execution of step S315, in response to receiving a selection of a location in the real-time video signal for presentation on the display device 130, real-time information corresponding to a designated object or work area at the location may be simultaneously presented on the display device 130. Specifically, a user may select a location in the real-time video signal presented on the display device 130 and determine the information to be presented at the location. Alternatively, the processor 110 may further identify whether the selected location in the real-time video signal corresponds to a designated object (e.g., a household appliance, electronic instrument, equipment, building material) or work area (e.g., a testing area, production area, or packaging area). If the selected location is determined to correspond to the designated object or work area, the processor 110 may simultaneously display the real-time information corresponding to the designated object or work area on the display device 130.

[0106] Furthermore, in response to the designated object being a human body, after the AI ​​model 220 executes an object detection algorithm and detects the presence of a human body in the real-time video signal, the AI ​​model 220 generates a selection frame to select the human body. Subsequently, when the display screen is controlled to present the real-time video signal from each imaging device, in response to the presence of the designated object (human body) in the real-time video signal, a selection frame is simultaneously displayed on the display device 130 to select the human body when the real-time video signal is presented to the display device 130. Furthermore, a selection frame can be further generated to select a specific part of the body, such as a palm.

[0107] FIG10 is a schematic diagram of a display screen according to an embodiment of the present invention. Referring to FIG10 , a human body is present in the video signal currently displayed on display screen 1000 . Accordingly, processor 110 simultaneously displays a selection box 1010 on display screen 1000 to select the human body, further displays a selection box 1020 to select the head of the human body, and displays selection boxes 1030 and 1040 to select the palms of the human body.

[0108] Furthermore, in response to the designated object being a human body, after the AI ​​model 222 executes an object detection algorithm and detects the presence of a human body in the real-time video signal, the AI ​​model 222 determines whether the human body is in a dangerous state (e.g., falling, not wearing a helmet, entering a dangerous area, etc.). If the human body is determined to be in a dangerous state, a warning message is recorded. Subsequently, when controlling the display screen to present the real-time video signal of each imaging device, in response to the presence of the designated object in the real-time video signal and the designated object having warning information, the warning message is simultaneously presented on the display device 130 when the real-time video signal is presented to the display device 130. Furthermore, the system can also be configured to simultaneously present real-time information related to the designated object when the real-time video signal is presented to the display device 130.

[0109] After determining the inspection route and inspecting multiple video signals via the display screen, in response to detecting a new event that meets the currently specified target event, the processor 110 will further reselect multiple new inspection devices from the imaging devices 140. For example, during the inspection process, if the artificial intelligence model 220 detects that another user has entered the imaging range of one of the imaging devices, the processor 110 will re-execute steps S310 and S315. Based on the relative position information, the processor 110 will re-determine a new inspection route for the new inspection devices, starting with the imaging device corresponding to the video signal currently displayed on the display device 130 as the inspection starting point. Based on the new inspection route, the processor 110 will control the display device 130 to display the real-time video signals of each new inspection device. In other words, the inspection system 100 can change the inspection route at any time based on the current situation.

[0110] The processor 110 can also be further configured to provide an inspection result interface to the display device 130. Figure 11 is a schematic diagram of an inspection result interface according to an embodiment of the present invention. Referring to Figure 11, the inspection result interface 1100 includes a video block 1110, a floor plan block 1120, an inspection screenshot block 1130, and an information block 1140. The video block 110 is used to play real-time video signals in real time. The floor plan block 1120 is used to display a floor plan corresponding to the space where the imaging device is located, and the floor plan includes position information corresponding to the actual position of each imaging device set in the space and a trajectory based on the inspection order. The inspection screenshot block 1130 is used to display screenshots corresponding to the target event, for example, displaying screenshots of each device. The information block 1140 is used to display real-time information corresponding to the target event. For example, the embodiment of Figure 11 specifies two target events, namely, events in which a human body is photographed and events in which a device is photographed. Therefore, the real-time information of each device will be synchronously displayed in the information block 1140, such as "Device #03 repaired", "Device #06 started", and when it is determined that the human body is in a dangerous state, the corresponding warning information will be displayed, such as "Area A: Person not wearing a helmet".

[0111] In another embodiment, real-time information and / or warning information can be directly superimposed on the real-time video signal presented on the display screen. Figure 12 is a schematic diagram of a display screen according to one embodiment of the present invention. Referring to Figure 12, display screen 1200 simultaneously presents selection boxes 12F1-12F5, selection boxes 12W1-12W2, and text boxes 1210-1250. Selection boxes 12F1-12F5 are used to select a human body in the real-time video signal. Selection boxes 12W1-12W2 are used to select a specific building material in the real-time video signal. Text box 1210 is used to display the number of people detected in the currently displayed real-time video signal. Text box 1220 corresponds to selection box 12F2 and is used to display a warning signal for the human body selected by selection box 12F2, such as "Not wearing a safety helmet." Text box 1230 is used to display real-time information about the detected equipment (e.g., the pickling tank), such as operating conditions such as temperature and concentration. The text box 1240 and the text box 1250 correspond to the selection boxes 12W1 - 12W2 respectively, and are used to present real-time information of the building materials selected by the selection boxes 12W1 - 12W2 , such as the operation content and production capacity status of the building materials.

[0112] Figure 13 is a schematic diagram of a display screen according to an embodiment of the present invention. Referring to Figure 13 , display screen 1300 simultaneously displays selection boxes 13W1-13W3. Selection boxes 13W1 and 13W2 select the hook, while selection box 13W3 selects the building material. Processor 110 can further detect the angle between the hoisted building material and the horizontal plane and simultaneously display this angle information on display screen 1300, dynamically changing the displayed angle information as the actual operating angle changes.

[0113] FIG14 is a schematic diagram of an inspection route according to an embodiment of the present invention. Referring to FIG14 , in this embodiment, imaging devices 14C1-14C4 are installed in a space, and users 14U1-14U4 are present in this space. Since imaging device 14C2 did not capture any person, imaging device 14C2 is not set as an inspection device. After processor 110 determines whether the inspection devices are imaging devices 14C1, 14C3, or 14C4, and the inspection order, processor 110 first controls the display screen of display device 130 to display the real-time video signal of imaging device 14C1, then controls the display screen to turn toward user 14U1's direction 14d1, and then controls the display screen to display the real-time video signal of imaging device 14C1 turning toward imaging device 14C3's direction 14d2.

[0114] The processor 110 then controls the display device 130 to switch its display screen to the real-time video signal from the imaging device 14C3 (towards direction 14d2), then controls the display screen to rotate toward direction 14d3 of the user 14U2, and then toward direction 14d4 of the imaging device 14C4. The processor 110 then controls the display device 130 to switch its display screen to the real-time video signal from the imaging device 14C4 (towards direction 14d4), then controls the display screen to rotate toward direction 14d5 of the user 14U3, and then toward direction 14d6 of the user 14U4.

[0115] Figure 15 is a schematic diagram of an inspection route according to an embodiment of the present invention. Referring to Figure 15 , in this embodiment, devices 1510 and 1520, as well as imaging devices 15C1 and 15C2, are installed in a space, and users 14U1-14U4 are present in this space. Processor 110 determines the inspection order for imaging devices 15C1 and 15C2 to be from imaging device 15C1 to imaging device 15C2, and sequentially executes the following steps A-D. In step A, processor 110 controls imaging device 15C1 to capture images toward device 1510, thereby displaying the captured real-time video signal on the display screen of display device 130. Afterwards, in step B, the processor 110 controls the imaging device 15C1 to capture images in the direction 15d1 of the imaging device 15C2, and controls the imaging device 15C2 to capture images in the direction 15d1, so that the display screen switches from the real-time video signal of the imaging device 15C1 to the real-time video signal of the imaging device 15C2 (towards the direction 15d1).

[0116] Next, in step C, processor 110 controls imaging device 15C2 to capture images toward device 1520, thereby presenting the captured real-time video signal on the display screen of display device 130. Finally, in step D, processor 110 controls imaging device 15C2 to capture images toward direction 15d2 of imaging device 15C1, and controls imaging device 15C1 to also capture images toward direction 15d2, so that the display screen switches from the real-time video signal of imaging device 15C2 to the real-time video signal of imaging device 15C1 (toward direction 15d2). Steps A through D are then repeated. Since the target event in this embodiment is capturing the designated devices 1510 and 1520, imaging devices 15C1 and 15C2 do not specifically turn toward the user. In the embodiment shown in FIG15 , for example, while imaging device 15C1 is turning from direction 15d2 toward device 1510, user 15U2 is captured and displayed on the display screen.

[0117] Alternatively, a picture-in-picture (PIP) display can be used to present users who are not on the inspection route. For example, in FIG15 , after the display screen in step D is switched to the real-time video signal of the imaging device 15C1 (towards direction 15d2 ), the PIP display can be used to display users 15U1 , 15U2 , and 15U3 .

[0118] In summary, the present invention provides an inspection method and system that selects a device that meets a target event from multiple imaging devices, generates an inspection route based on the selected device, and then displays the content captured by the imaging devices based on the inspection route. This allows rapid capture of images matching the target event from multiple real-time video signals and displays them on a display device.

Claims

1. A patrol inspection method, suitable for being executed by an electronic device, characterized in that The inspection method includes: Obtaining relative position information between multiple imaging devices; Determining an inspection route based on a target event and the relative position information, where the inspection route meets the target event, and multiple imaging devices passed by in the inspection route are set as multiple inspection devices; and Controlling real-time video signals of the multiple inspection devices to be presented on a display device based on the inspection route.

2. The inspection method according to claim 1, wherein It further includes: Establishing the relative position information between the multiple imaging devices, including: Providing a floor plan corresponding to the space where the multiple imaging devices are set; Based on user operations, marking multiple planar positions corresponding to the actual positions of the multiple imaging devices set in the space in the floor plan; and Calculating the relative position information between the multiple imaging devices based on the multiple planar positions.

3. The inspection method according to claim 1, characterized in that It further includes: Establishing the relative position information between the multiple imaging devices, including: Respectively obtaining multiple images corresponding to the multiple imaging devices from the multiple imaging devices; And Calculating the relative position information between the multiple imaging devices by finding corresponding feature points in every two images.

4. The inspection method according to claim 1, wherein The target event includes an event used to indicate that a specified object is photographed, The step of determining the inspection route includes: Judging whether the multiple imaging devices photograph the specified object by executing a target detection algorithm on the real-time video signals received by each of the multiple imaging devices through an artificial intelligence model; and In response to multiple target devices among the multiple imaging devices photographing the specified object, determining the multiple inspection devices based on the relative position information and the multiple target devices that photograph the specified object, where the number of the multiple inspection devices included in the inspection route is greater than or equal to the number of the multiple target devices.

5. The inspection method according to claim 4, where after judging whether the multiple imaging devices photograph the specified object, it further includes: In response to only the first imaging device among the multiple imaging devices photographing the specified object, determining the inspection route based on the relative position information and the first imaging device that photographs the specified object, where the inspection route at least includes the first imaging device and a second imaging device corresponding to a predetermined position.

6. The inspection method according to claim 4, where after judging whether the multiple imaging devices photograph the specified object, it further includes: In response to the specified object being a device, after detecting the device in the real-time video signal by executing the target detection algorithm through the artificial intelligence model, obtaining real-time information of the device and recording the real-time information; Where the step of controlling real-time video signals of the multiple inspection devices to be presented on the display device further includes: In response to the specified object existing in the real-time video signal, when the real-time video signal is presented on the display device, presenting the corresponding real-time information on the display device at the same time.

7. The inspection method according to claim 4, wherein after determining whether the plurality of imaging devices have captured the specified object, it further includes: In response to the specified object being a human body, after detecting the presence of the human body in the real-time video signal by executing the target detection algorithm through the artificial intelligence model, determining whether the human body is in a dangerous state through the artificial intelligence model, and recording a warning message when it is determined that the human body is in the dangerous state; Wherein the step of controlling the real-time video signals of the plurality of inspection devices to be presented to the display device further includes: In response to the presence of the specified object in the real-time video signal and the specified object having the warning message, when the real-time video signal is presented to the display device, presenting the warning message in the display device at the same time.

8. The inspection method according to claim 4, wherein after determining whether the plurality of imaging devices have captured the specified object, it further includes: In response to the specified object being a human body, after detecting the presence of the human body in the real-time video signal by executing the target detection algorithm through the artificial intelligence model, generating a selection box that frames the human body through the artificial intelligence model; Wherein the step of controlling the real-time video signals of the plurality of inspection devices to be presented to the display device further includes: In response to the presence of the specified object in the real-time video signal, when the real-time video signal is presented to the display device, presenting the selection box in the display device at the same time to frame the human body.

9. The inspection method according to claim 4, wherein the step of controlling the real-time video signals of the plurality of inspection devices to be presented to the display device based on the inspection route includes: Switching the display screen of the display device from the real-time video signal of the first inspection device among the plurality of inspection devices to the real-time video signal of the second inspection device that has captured the specified object among the plurality of inspection devices, including: Controlling the first inspection device to turn to capture images in the first direction towards the second inspection device, and controlling the second inspection device to turn to the first direction; Presenting the real-time video signal of the first inspection device in the first direction to the display screen; Performing a zoom operation on the real-time video signal of the first inspection device in the display screen; and After performing the zoom operation, controlling the second inspection device to turn from the first direction towards the second direction of the specified object to capture images, and synchronously switching the display screen to the real-time video signal of the second inspection device during the turning process of the second inspection device.

10. The inspection method according to claim 1, wherein the target event includes an event for instructing the inspection of at least one working area, The step of determining the inspection route includes: Selecting at least one target device corresponding to the at least one working area among the plurality of imaging devices; And Determining the plurality of inspection devices based on the relative position information and the at least one target device.

11. The inspection method according to claim 1, further includes: Determine the plurality of inspection devices included in the inspection route based on the target event and at least one other target event; And Determine the inspection order of the plurality of inspection devices with reference to the event order of the target event and at least one other target event and based on the relative position information.

12. The inspection method according to claim 1, wherein the step of determining the inspection route includes: Determine the inspection order of the plurality of inspection devices based on the relative position information and the priorities of the plurality of imaging devices.

13. The inspection method according to claim 1, wherein in the process of sequentially displaying the video signals of each of the plurality of inspection devices to the display device based on the inspection route, it further includes: In response to detecting that a new event meets the target event, reselect a plurality of the plurality of imaging devices as a plurality of new inspection devices; Based on the relative position information, starting from the imaging device corresponding to the video signal currently displayed on the display device as the inspection starting point, re-determine the new inspection route of the plurality of new inspection devices; And Based on the new inspection route, control the real-time video signals of the plurality of new inspection devices to be presented to the display device.

14. The inspection method according to claim 1, further includes: Provide an inspection result interface to the display device, wherein the inspection result interface includes a video block, a floor plan block, an inspection screenshot block, and an information block, The video block is used to play the real-time video signal in real time, The floor plan block is used to display the floor plan corresponding to the space where the plurality of imaging devices are located, and the floor plan includes a plurality of position information corresponding to the actual positions of the plurality of imaging devices arranged in the space corresponding to the floor plan and a trajectory based on the inspection order, The inspection screenshot block is used to display the screenshot corresponding to the target event, The information block is used to display the real-time information corresponding to the target event.

15. The inspection method according to claim 1, wherein in the process of controlling the real-time video signals of the plurality of inspection devices to be presented to the display device based on the inspection route, it further includes: In response to receiving a selected position in the real-time video signal presented on the display device, simultaneously present the real-time information corresponding to the specified object or work area included in the position on the display device.

16. An inspection system, characterized in that, Includes: A plurality of imaging devices; A display device; And A processor, coupled to the plurality of imaging devices and the display device, wherein the processor is configured to: Obtain the relative position information between the plurality of imaging devices; Determine an inspection route based on the target event and the relative position information, wherein the inspection route meets the target event, and the multiple imaging devices passed through in the inspection route are set as a plurality of inspection devices; And Based on the inspection route, control the real-time video signals of the plurality of inspection devices to be presented to the display device.

Images