Intelligent monitoring method, device and medium for production workshop

Abstract

The present application relates to the field of path planning adjustment, in particular to an intelligent monitoring method and device for a production workshop and a medium. First, a straight line path of a to-be-monitored person in a preset future period is planned as a first prediction path. If the path intersects with a planned running path of an intelligent forklift, a second prediction path with higher accuracy is generated. Then, first and second path vectors are generated according to the number of times that the two paths pass through each grid in a two-dimensional grid navigation map. When the cosine similarity of the two path vectors is greater than a threshold value and the second prediction path still intersects with the planned path of the forklift, the part of the path of the forklift in the preset period is re-planned to avoid the person. The method can quickly estimate the straight line prediction path of the person, judge whether the path intersects with the path of the forklift, accurately predict the path of the person when a possible collision occurs, further accurately judge the risk of hitting the person, timely control the forklift to avoid obstacles, and prevent the forklift from hitting the pedestrian.

Description

Technical Field

[0001] This invention relates to the field of path planning and adjustment, and in particular to an intelligent monitoring method, equipment and medium for production workshops. Background Technology

[0002] Goods handling is an indispensable part of many industries, including logistics, manufacturing, and retail. It involves moving goods from one location to another. With technological advancements, goods handling methods have gradually evolved from traditional manual handling to automation and intelligence. Intelligent forklifts combine advanced technologies such as automation, robotics, sensor technology, and artificial intelligence to achieve automated material handling tasks. The transfer of goods is involved in large warehouses or between factory production lines.

[0003] Multiple intelligent forklifts are typically deployed in a manufacturing workshop to jointly complete goods transfer tasks. Therefore, multiple intelligent forklifts may be operating simultaneously within the same workshop area at the same time. To prevent collisions, the intelligent forklifts communicate and locate each other to monitor their operating paths in real time, and use obstacle avoidance algorithms to prevent collisions. However, a few inspection personnel often move around the same workshop area. Because personnel have greater freedom of movement and typically cannot communicate and locate the intelligent forklifts, intelligent forklifts may collide with pedestrians during inspections. Summary of the Invention

[0004] To address one of the aforementioned technical problems, the present invention adopts the following technical solution:

[0005] According to one aspect of the present invention, an intelligent monitoring method for a production workshop is provided, the method comprising the following steps:

[0006] Based on the current direction and speed of movement of the personnel to be monitored in the workshop area, a first predicted path is planned for the personnel to be monitored in a preset future time period; the first predicted path is a straight path.

[0007] If the first predicted path intersects with the planned running path of any intelligent forklift in the workshop area in a preset future time period, then a second predicted path for the person to be monitored in the preset future time period is generated based on the historical motion image information of the person to be monitored; the accuracy of the second predicted path is greater than that of the first predicted path.

[0008] Based on the number of times each grid cell is traversed in the two-dimensional grid navigation map corresponding to the first and second predicted paths in the workshop area, a first path vector and a second path vector corresponding to the first and second predicted paths are generated respectively; each element in the path vector is the number of times the corresponding predicted path traverses each grid cell.

[0009] If the cosine similarity between the first path vector and the second path vector is greater than a preset similarity threshold, and the second predicted path intersects with the planned running path of any intelligent forklift in the workshop area in a preset future time period, then a local path replanning operation is performed on the corresponding part of the intelligent forklift's path in the preset future time period to avoid the personnel to be monitored.

[0010] According to a second aspect of the present invention, a non-transitory computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the above-described intelligent monitoring method for a production workshop.

[0011] According to a third aspect of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described intelligent monitoring method for a production workshop.

[0012] This invention has at least one of the following beneficial effects:

[0013] In this invention, a predicted straight path (i.e., the first predicted path) of the monitored person in a preset future time period can be quickly estimated based on the person's current movement direction and speed in real time. Then, using this straight path as an initial criterion, it is quickly determined whether it intersects with the planned operating path of an intelligent forklift in the workshop area, thereby quickly determining whether a collision is possible. If a collision is possible, a more accurate predicted path (i.e., the second predicted path) is generated based on the person's historical motion image information. Furthermore, by assessing the similarity between the first and second predicted paths, and whether the second predicted path intersects with the planned operating path of the intelligent forklift, a more accurate judgment is made regarding whether the intelligent forklift will hit a pedestrian, allowing for timely obstacle avoidance control and preventing collisions.

[0014] Furthermore, this invention employs a two-stage judgment method to determine the likelihood of a collision between the intelligent forklift and a pedestrian. Typically, the walking routes of inspection personnel in a workshop are set within safe zones, and due to the relatively regular layout of production workshops, these routes usually consist of multiple straight segments. Therefore, the first stage of this invention uses a simple, rough prediction of the straight-line trajectory to determine whether a collision between a person and the forklift is possible. If a collision is deemed likely, a second, more precise scheme is activated to predict the trajectory of the personnel being monitored, thus further refining the collision risk assessment. This not only controls the computational load consumed in collision detection but also improves the accuracy of collision detection. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A flowchart illustrating an intelligent monitoring method for a production workshop provided in an embodiment of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] As one possible embodiment of the present invention, such as Figure 1 As shown, an intelligent monitoring method for a production workshop includes the following steps:

[0019] S100: Based on the current direction and speed of movement of the personnel to be monitored in the workshop area, plan the first predicted path for the personnel to be monitored in a preset future time period. The first predicted path is a straight path.

[0020] Typically, workshop floors are rectangular spaces, and to improve production efficiency, optimize logistics processes, and ensure a safe and convenient working environment, the placement areas for equipment and other components are planned, resulting in a more compact and tidy overall layout. Correspondingly, the drivable areas (the areas for intelligent forklifts and personnel to be monitored) are also usually more regular in shape. The inspection routes of personnel to be monitored generally consist of multiple straight segments, so they mostly walk in a straight line. Therefore, in this embodiment, the first segment uses a simple and rough straight-line trajectory prediction to determine whether a collision between personnel and vehicles is likely. Based on this, not only can a certain level of path prediction accuracy be guaranteed, but the prediction speed can also be improved. Specifically, the first predicted path can be obtained using the following two methods.

[0021] Firstly, S100 includes:

[0022] S111: Obtain the first and last image frames, including the image information of the person to be detected, from the historical video corresponding to the current time.

[0023] Specifically, historical videos are videos acquired up to a preset duration before the current time. In this embodiment, the preset duration can be 2 seconds. For example, if the current time is 10:00:00, the corresponding historical video is a video segment acquired by a surveillance camera between 09:59:58 and 10:00:00.

[0024] In this embodiment, multiple surveillance cameras can be installed at various locations above the production workshop to acquire real-time video of the workshop, with each camera having a fixed shooting angle. Historical videos are acquired by one of these cameras. Furthermore, when selecting videos, it is preferable to choose those captured by cameras that can capture the full body and facial information of individuals.

[0025] After obtaining historical video footage, publicly available technologies can be used to identify and track people (i.e., those to be monitored) in the video. For example, existing software with person recognition and tracking capabilities, such as OpenCV, DeepSORT, or Viso.ai, can be used to identify the position of the person to be monitored in each frame of the video and mark the bounding box.

[0026] S112: Obtain the center pixel position of the first bounding box and the center pixel position of the second bounding box corresponding to the person to be monitored from the first image frame and the last image frame, respectively.

[0027] S113: Map the center pixel positions of the first bounding box and the second bounding box to the two-dimensional grid navigation map respectively, and connect them to form a reference path.

[0028] Since the entire time span of historical videos is extremely short, the size of the images of people in the videos will not change significantly. Based on this, the pixel position of the person to be monitored in each frame can be represented by the center of the bounding box of the person's image. Then, according to the mapping relationship between the pixel coordinate system of the monitoring camera and the coordinate system of the two-dimensional grid navigation map, two coordinate points are determined in the two-dimensional grid navigation map and a reference path is formed by connecting them with straight lines.

[0029] S114: Determine the movement direction of the person to be monitored based on the extension direction of the reference path.

[0030] Specifically, S114 includes:

[0031] S104: Based on the center pixel positions of the first bounding box and the second bounding box, generate the time sequence to determine the extension direction of the reference path.

[0032] S115: Determine the movement speed of the person to be monitored based on the length of the reference path and the time interval between the first and last image frames.

[0033] Therefore, given that the length and direction of the reference path and its corresponding duration are known, the movement speed and direction of the person to be monitored can be obtained. Based on this, the straight path formed in this direction and at this speed within a preset future time period (5 seconds) can be predicted.

[0034] Secondly

[0035] Multiple Bluetooth receivers are installed in the workshop area, and all of them communicate with the same clock unit. The device worn by the person being monitored is equipped with a Bluetooth transmitter, which is used to broadcast location information, including the information transmission time and the corresponding identity information of the person being monitored.

[0036] This method requires Bluetooth communication for positioning. Specifically, multiple Bluetooth receivers can be suspended from the ceiling or deployed on the floor of the workshop. The deployment area should be close to the inspection path of the personnel being monitored. Specifically, a Bluetooth receiver and transmitter with a maximum power of 2.5mW can be selected. Bluetooth devices at this power level have low power consumption and a communication distance of approximately 10 meters, making them suitable for this scenario.

[0037] Since subsequent Bluetooth transmitter positioning is based on the Time Difference of Arrival (TDOA) method, it is necessary to connect all Bluetooth receivers to the same clock unit to provide unified time synchronization for all Bluetooth receivers, thereby improving positioning accuracy.

[0038] In this embodiment, the Bluetooth transmitter can be installed in the safety protective equipment that the personnel to be monitored must wear to enter the workshop, such as a safety helmet. Furthermore, the information broadcast by the Bluetooth transmitter includes the identity information of the personnel to be monitored. Therefore, the solution in this embodiment can not only obtain the location of personnel in real time, but also track them using their identity information.

[0039] S100 may also include:

[0040] S121: Generate the location information of the person to be monitored at the current moment based on the reception time corresponding to the same location information received by at least three Bluetooth receivers.

[0041] S122: Based on the location information of the person to be monitored at the current time and the previous time, generate the movement direction and speed of the person to be monitored.

[0042] By measuring the time difference between the arrival of Bluetooth signals at different receivers, the position of the Bluetooth transmitter relative to these receivers can be calculated. This allows for the determination of the actual spatial location of the person being monitored at different times, forming a corresponding reference path on a two-dimensional grid navigation map, ultimately yielding the first predicted path.

[0043] S200: If the first predicted path intersects with the planned operating path of any intelligent forklift in the workshop area within a preset future time period, then a second predicted path is generated based on the historical motion image information of the person to be monitored within the preset future time period. The accuracy of the second predicted path is greater than that of the first predicted path.

[0044] First, a two-dimensional grid navigation map needs to be created based on the operable area of ​​the workshop where the intelligent forklift is located. Since the spatial distribution of a factory workshop is usually rectangular, the operating ground for the intelligent forklift is also rectangular. Therefore, when constructing the two-dimensional grid navigation map, the continuous rectangular space can be directly divided into discrete small areas (i.e., grids). Each grid can contain information about that area (such as whether it is passable, terrain type, etc.). The completed two-dimensional grid navigation map can be stored in the form of a two-dimensional array. Specifically, each element in the array corresponds to the state of a grid. Since the state of the workshop ground is basically the same, and its impact on the passage of the intelligent forklift is basically the same, it is not necessary to distinguish the ground state when constructing the two-dimensional grid navigation map; only whether the corresponding grid area has obstacles needs to be considered. For example, corresponding numbers can be used in the two-dimensional array to indicate whether there are obstacles in the area, such as 0 indicating a passable area and 1 indicating an obstacle.

[0045] After constructing the corresponding two-dimensional grid navigation map, a scheduling path from the starting point to the destination point can be planned based on the transfer task currently received by each intelligent forklift. Specifically, in this embodiment, the A* algorithm is used to plan the scheduling path in the corresponding two-dimensional grid navigation map.

[0046] The A* (A*) path search algorithm (also known as the A* algorithm) is an algorithm for finding the shortest path between two points on a graphical plane, given a starting point and an ending point.

[0047] Specifically, in this embodiment, the A* path search algorithm is used. During the path planning process on the two-dimensional grid navigation map, each node in the path is the center point of the grid.

[0048] During the execution of the A* path search algorithm, the core of its search control is the setting of the cost function (g(n)) and the heuristic function (h(n)).

[0049] g(n) represents the actual cost from the starting point to the current node n. This value can be adjusted according to the actual application scenario. In this embodiment, since the state of the entire passable ground is basically the same, all passable grids have the same passable cost. The cost of each step can be simply set to 1, that is, g(n) can be calculated based on the distance traveled.

[0050] h(n) is an estimate representing the expected cost from the current node n to the target node. In this embodiment, the estimated cost can be calculated using either Manhattan distance or Euclidean distance, depending on the limitations of the intelligent forklift's movement direction. Manhattan distance is suitable for situations where movement is limited to the four basic directions (up, down, left, and right). Euclidean distance is suitable for situations where movement is possible in diagonal directions.

[0051] In addition, S200 generates a second predicted path for the monitored person in a preset future time period based on the person's historical motion image information. This can be obtained by referring to the methods described in existing technologies CN111639591A or CN111797751A.

[0052] Specifically, whether the first predicted path intersects with the planned operating path of any intelligent forklift in the workshop area during a preset future time period is determined by the following steps:

[0053] S201: Obtain the coordinates A(X1,Y1) of the lower left corner and B(X2,Y2) of the upper right corner of the bounding box corresponding to the first predicted path in the two-dimensional grid navigation map.

[0054] S202: If the i-th node Ci(Xi, Yi) in the planned operation path corresponding to any intelligent forklift in the workshop area within a preset future period satisfies the conditions: X1 < Xi < X2, Y1 < Yi < Y2, then take the planned operation path corresponding to the intelligent forklift in the preset future period as the initial interference path. Each node in the planned operation path corresponding to the intelligent forklift is the center point of the corresponding grid in the two-dimensional grid navigation map.

[0055] S203: Take the position of each grid passed by each initial interference path in the two-dimensional grid navigation map as a tuple, and generate a path set L1 corresponding to the initial interference path.

[0056] S204: Take the position of each grid passed by the first predicted path in the two-dimensional grid navigation map as a tuple, and generate a path set L2 corresponding to the first predicted path.

[0057] S205: If there is an intersection between L2 and L1 corresponding to any initial interference path, then obtain the time T1 for the intelligent forklift corresponding to the initial interference path to reach the intersection position point from the current position, and the time T2 for the person to be monitored to reach the intersection position point from the current position.

[0058] S206: If |T1 - T2| < T, determine that there is an intersection between the initial interference path and the first predicted path. T is a preset duration threshold. T can be 3 seconds.

[0059] S207: If |T1 - T2| ≥ T, determine that there is no intersection between the initial interference path and the first predicted path.

[0060] In this embodiment, when determining whether there is an intersection between two trajectories, first, through X1 < Xi < X2, Y1 < Yi < Y2, all irrelevant paths that do not pass through the bounding box corresponding to the first predicted path can be quickly removed with fewer comparisons, so as to reduce the data processing volume when specifically locating the trajectory intersection by the set intersection method later. Then, after determining the intersection point, determine whether there may be a collision situation by the time difference between the pedestrian and the intelligent forklift when they reach the intersection point, that is, whether there is an intersection between the initial interference path and the first predicted path in this embodiment.

[0061] S300: Generate the first path vector and the second path vector corresponding to the first predicted path and the second predicted path respectively according to the number of times each grid is passed by the first predicted path and the second predicted path in the two-dimensional grid navigation map corresponding to the workshop area. Each element in the path vector is the number of times the corresponding predicted path passes through each grid.

[0062] In this embodiment, due to the existence of a unified reference system—a two-dimensional grid navigation map—each running trajectory can be more easily vectorized. Specifically, the number of times each trajectory passes through each grid is used as the element value of each grid in the entire vector. For example, if the two-dimensional grid navigation map is a 3×3 grid map, its corresponding vector includes 9 elements. If a straight line trajectory passes through all the grids in the second row once, the corresponding vector is (0, 0, 0, 1, 1, 1, 0, 0, 0).

[0063] Therefore, spatial paths can be transformed into vectors, which facilitates similarity calculation.

[0064] S400: If the cosine similarity between the first path vector and the second path vector is greater than the preset similarity threshold, and the second predicted path intersects with the planned running path of any intelligent forklift in the workshop area in the preset future time period, then a local path replanning operation is performed on the corresponding part of the intelligent forklift's path in the preset future time period to avoid the personnel to be monitored.

[0065] In this step, if the cosine similarity between the first path vector and the second path vector is less than or equal to the preset similarity threshold, it is not necessary to determine whether the second predicted path intersects with the planned operation path of any intelligent forklift in the workshop area in the preset future time period. Instead, a local path replanning operation is directly performed on the corresponding part of the intelligent forklift's path in the preset future time period to avoid the personnel to be monitored.

[0066] Specifically, local path replanning operations include:

[0067] Reduce the moving speed of the intelligent forklift and / or change the local path of the intelligent forklift.

[0068] In this embodiment, a predicted straight path (i.e., the first predicted path) of the monitored person in a preset future time period can be quickly estimated based on the person's current movement direction and speed in real time. Then, using this straight path as an initial criterion, it is quickly determined whether it intersects with the planned operating path of an intelligent forklift in the workshop area, thereby quickly determining whether a collision between the intelligent forklift and the monitored person is possible. If a collision is possible, a more accurate predicted path (i.e., the second predicted path) is predicted based on the monitored person's historical motion image information. Furthermore, by assessing the similarity between the first and second predicted paths, and whether the second predicted path intersects with the planned operating path of the intelligent forklift, a more accurate judgment is made regarding whether the intelligent forklift will hit a person, allowing for timely obstacle avoidance control and preventing collisions.

[0069] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0070] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0071] In an exemplary embodiment of this disclosure, an electronic device capable of implementing the above-described method is also provided.

[0072] Those skilled in the art will understand that various aspects of the present invention can be implemented as systems, methods, or program products. Therefore, various aspects of the present invention can be specifically implemented in the following forms: entirely hardware implementations, entirely software implementations (including firmware, microcode, etc.), or implementations combining hardware and software aspects, collectively referred to herein as “circuits,” “modules,” or “systems.”

[0073] An electronic device according to this embodiment of the invention. The electronic device is merely an example and should not be construed as limiting the functionality or scope of the embodiments of the invention.

[0074] Electronic devices are manifested in the form of general-purpose computing devices. Components of an electronic device may include, but are not limited to: at least one processor, at least one memory, and buses connecting different system components (including memory and processor).

[0075] The memory stores program code that can be executed by a processor, causing the processor to perform the steps described in the "Exemplary Methods" section above, according to various exemplary embodiments of the present invention.

[0076] The storage may include readable media in the form of volatile storage, such as random access memory (RAM) and / or cache memory, and may further include read-only memory (ROM).

[0077] The storage may also include programs / utilities having a set (at least one) of program modules, including but not limited to: an operating system, one or more applications, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0078] A bus can represent one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus that uses any of the various bus architectures.

[0079] The electronic device can also communicate with one or more external devices (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable a user to interact with the electronic device, and / or any device that enables the electronic device to communicate with one or more other computing devices (e.g., routers, modems, etc.). This communication can be performed via input / output (I / O) interfaces. Furthermore, the electronic device can communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter. The network adapter communicates with other modules of the electronic device via a bus. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0080] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0081] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible embodiments, various aspects of the present invention may also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps of the various exemplary embodiments of the present invention described in the "Exemplary Methods" section above.

[0082] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0083] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0084] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0085] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0086] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0087] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0088] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for intelligent monitoring of a production workshop, characterized in that, The method includes the following steps: Based on the current direction and speed of movement of the personnel to be monitored in the workshop area, a first predicted path is planned for the personnel to be monitored in a preset future time period; the first predicted path is a straight path. If the first predicted path intersects with the planned operating path of any intelligent forklift in the workshop area during a preset future time period, a second predicted path is generated based on the historical motion image information of the person to be monitored during the preset future time period; the accuracy of the second predicted path is greater than that of the first predicted path. Based on the number of times each grid cell is traversed in the two-dimensional grid navigation map corresponding to the first and second predicted paths in the workshop area, a first path vector and a second path vector corresponding to the first and second predicted paths are generated respectively; each element in the path vector is the number of times the corresponding predicted path traverses each grid cell. If the cosine similarity between the first path vector and the second path vector is greater than a preset similarity threshold, and the second predicted path intersects with the planned operating path of any intelligent forklift in the workshop area in a preset future time period, then a local path replanning operation is performed on the corresponding part of the path of the intelligent forklift in the preset future time period to avoid the personnel to be monitored.

2. The method according to claim 1, characterized in that, Based on the movement direction and speed of the personnel to be monitored in the workshop area, determine the first predicted path of the personnel to be monitored in a preset future time period, including: From the historical video corresponding to the current time, obtain the first and last image frames, which include the image information of the person to be detected; the historical video is a video obtained before the current time with a preset duration. The center pixel position of the first bounding box and the center pixel position of the second bounding box corresponding to the person to be monitored are obtained from the first image frame and the last image frame, respectively. The center pixel positions of the first bounding box and the center pixel positions of the second bounding box are mapped onto a two-dimensional grid navigation map, and then connected to form a reference path; The direction of movement of the person to be monitored is determined based on the extension direction of the reference path; The movement speed of the person to be monitored is determined based on the length of the reference path and the time interval between the first and last image frames.

3. The method according to claim 2, characterized in that, Before determining the movement direction of the person to be monitored based on the extension direction of the reference path, the method further includes: Based on the center pixel positions of the first and second bounding boxes, the order in which time events are generated determines the extension direction of the reference path.

4. The method according to claim 2, characterized in that, The historical videos were acquired by cameras with a fixed shooting direction and field of view.

5. The method according to claim 1, characterized in that, Multiple Bluetooth receivers are installed in the workshop area, and all of the Bluetooth receivers are connected to the same clock unit. The device worn by the person to be monitored is equipped with a Bluetooth transmitter, which is used to broadcast location information. The location information includes the information transmission time and the identity information of the person to be monitored. Based on the movement direction and speed of the personnel to be monitored in the workshop area, determine the first predicted path of the personnel to be monitored in a preset future time period, including: Generate the position information of the person to be monitored at the current moment based on the reception times corresponding to the same positioning information received by at least three Bluetooth receivers; Generate the movement direction and moving speed of the person to be monitored based on the position information of the person to be monitored corresponding to the current moment and the previous moment respectively.

6. The method according to claim 1, characterized in that, Determine whether there is an intersection between the first predicted path and the planned operation path corresponding to any intelligent forklift in the workshop area during a preset future period according to the following steps: Obtain the coordinates A(X1, Y1) of the lower left corner point and the coordinates B(X2, Y2) of the upper right corner point of the bounding box corresponding to the first predicted path in the two-dimensional grid navigation map; If the i-th node Ci(Xi, Yi) in the planned operation path corresponding to any intelligent forklift in the workshop area during the preset future period satisfies the condition: X1 < Xi < X2, Y1 < Yi < Y2, then regard the planned operation path corresponding to the intelligent forklift during the preset future period as the initial interference path; each node in the planned operation path corresponding to the intelligent forklift is the center point of the corresponding grid in the two-dimensional grid navigation map; Take the position of each grid passed by each initial interference path in the two-dimensional grid navigation map as a tuple, and generate a path set L1 corresponding to the initial interference path; Take the position of each grid passed by the first predicted path in the two-dimensional grid navigation map as a tuple, and generate a path set L2 corresponding to the first predicted path; If there is an intersection between L2 and L1 corresponding to any initial interference path, obtain the time T1 for the intelligent forklift corresponding to the initial interference path to reach the intersection position point from the current position, and the time T2 for the person to be monitored to reach the intersection position point from the current position; If |T1 - T2| < T, determine that there is an intersection between the initial interference path and the first predicted path; T is a preset duration threshold.

7. The method according to claim 6, characterized in that, After obtaining T1 and T2, the method further includes: If |T1 - T2| ≥ T, determine that there is no intersection between the initial interference path and the first predicted path.

8. The method according to claim 1, characterized in that, The local path replanning operation includes: Reduce the moving speed of the intelligent forklift and / or change the local path of the intelligent forklift.

9. A non-transitory computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements an intelligent monitoring method for a production workshop as described in any one of claims 1 to 8.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements an intelligent monitoring method for a production workshop as described in any one of claims 1 to 8.

Images