Production monitoring method, controller, production area monitoring system and storage medium

Abstract

The embodiment of the invention provides a production monitoring method, a controller, a production area monitoring system and a storage medium, and belongs to the technical field of production line management. The production area monitoring system comprises at least one acquisition device. The method comprises the following steps: acquiring a monitoring video of an acquisition device; performing picture detection on the monitoring video through a preset algorithm detection system to obtain a target product and a product identifier, and performing position detection on the target product through the preset algorithm detection system; when it is detected that the target product does not arrive at the preset station, behavior analysis is carried out on the monitoring video based on a preset operation guidance process, and detection action information is obtained; and when it is determined that the detection action information satisfies a preset operation guidance condition, binding the product identifier with a target product, and intercepting the monitoring video according to the detection start time and the detection end time to obtain a target detection fragment. According to the embodiment of the invention, the monitoring of the whole detection link of the product can be realized, and the product quality is improved.

Description

Technical Field

[0001] The present application relates to the technical field of production line management, and in particular to a production monitoring method, a controller, a production area monitoring system and a storage medium. Background Art

[0002] In the automated equipment production line, there are some manual workstations for product quality inspection, and operators need to conduct product quality inspection according to the product inspection work instructions. However, there is a lack of effective supervision and recording mechanism for whether the operators follow the product inspection work instructions for quality inspection and whether the necessary inspections are carried out. This lack of monitoring leads to the inability to track and evaluate the work behavior of operators in real time, making it impossible to ensure that every product has undergone strict quality inspections. The outflow of defective products not only affects the final quality of the product, but may also lead to a decline in customer trust and damage the company's reputation.

[0003] At present, some companies install surveillance cameras at manual workstations to record videos 24 hours a day. When problems occur, they check the surveillance playback. This has the problem of low efficiency in post-analysis and video review, and due to the lack of a traceability mechanism, it is difficult to determine the responsibility once a product problem occurs, which leads to lax product quality control and increased recall costs. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to propose a production monitoring method, a controller, a production area monitoring system and a storage medium, which can monitor the overall detection link of the product and improve the quality of the product.

[0005] To achieve the above-mentioned purpose, a first production monitoring method of an embodiment of the present application is applied to a production area monitoring system, wherein the production area monitoring system includes at least one acquisition device; the method includes:

[0006] Acquire a monitoring video of the acquisition device, wherein the monitoring video is obtained by the acquisition device acquiring images of products on a production line;

[0007] Performing image detection on the surveillance video through a preset algorithm detection system to obtain a target product and a product identification corresponding to the target product, and performing position detection on the target product through a preset algorithm detection system;

[0008] When it is detected that the target product has not arrived at the preset workstation, a behavior analysis is performed on the monitoring video based on a preset operation guidance process to obtain detection action information, wherein the detection action information includes a detection start time and a detection end time;

[0009] When it is determined that the detection action information meets the preset operation guidance conditions, the product identification is bound to the target product, and the monitoring video is intercepted according to the detection start time and the detection end time to obtain a target detection segment.

[0010] In some embodiments, the detecting the surveillance video by using a preset algorithm detection system to obtain a target product and a product identification corresponding to the target product includes:

[0011] Inputting the surveillance video into the algorithm detection system so that the algorithm detection system performs video decomposition on the surveillance video and outputs a plurality of regional video frames;

[0012] For each of the regional video frames, performing region detection on the regional video frame by using the algorithm detection system to obtain a boundary region;

[0013] Perform product recognition on the regional video frame based on the boundary region to obtain a target product;

[0014] The algorithm detection system assigns a product number to the target product to obtain a product identification corresponding to the target product.

[0015] In some embodiments, the position detection of the target product by a preset algorithm detection system includes:

[0016] The target product is labeled by the algorithm detection system to obtain a detection frame;

[0017] The position of the target product is tracked according to the detection frame and the product identification.

[0018] In some embodiments, the operation guidance process includes multiple target actions; the behavior analysis of the monitoring video based on the preset operation guidance process to obtain the detection action information includes:

[0019] Extracting features from the surveillance video to obtain a detection area;

[0020] When there is a preset operating object in the detection area, action detection is performed on the detection area according to all the target actions to obtain a detection set, wherein the detection set includes detection results and detection times of video frames corresponding to the target actions one by one;

[0021] According to the detection result, all video frames in the detection set are classified and counted to obtain at least one classification result;

[0022] Determine the detection start time and detection end time of the monitoring video according to the detection time;

[0023] Detection action information is obtained according to the detection start time, the detection end time, the classification result and the detection time.

[0024] In some embodiments, the operation guidance condition includes a detection sequence condition and a duration interval corresponding to the target action; and determining that the detection action information satisfies the preset operation guidance condition includes:

[0025] Determine the action duration corresponding to each of the classification results according to the detection time in the detection action information;

[0026] When the detection time of each video frame in the detection action information meets the sequence condition and the action duration is within the duration interval, it is determined that the detection action information meets the preset operation guidance condition.

[0027] In some embodiments, after the target product is detected by a preset algorithm detection system, the method further includes:

[0028] When it is detected that the target product arrives at the preset workstation, a first alarm message is generated.

[0029] In some embodiments, after performing behavior analysis on the surveillance video to obtain detection action information, the method further includes:

[0030] When it is determined that the detection action information does not meet the preset operation guidance condition, determining a violation event that does not meet the operation guidance condition according to the detection action information;

[0031] The violation event is recorded and a second alarm message is generated.

[0032] In a second aspect, an embodiment of the present application provides a controller, comprising a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor is used to execute the production monitoring method as described in the first aspect.

[0033] In a third aspect, an embodiment of the present application provides a production area monitoring system, wherein the controller includes at least one acquisition device and a controller as described in the second aspect.

[0034] In a fourth aspect, an embodiment of the present application provides a storage medium, which is a computer-readable storage medium, and the computer-readable storage medium stores a computer program. When the computer program is executed by a computer, the computer is used to execute the production monitoring method as described in any one of the embodiments of the first aspect of the present application.

[0035] The production monitoring method, controller, production area monitoring system and storage medium proposed in the embodiments of the present application have the following beneficial effects: the production area monitoring system of the embodiments of the present application includes at least one acquisition device, which is used to acquire images of products on the production line. In the process of production monitoring of products, the embodiments of the present application first obtains the monitoring video of the acquisition device, and then performs image detection on the monitoring video through a preset algorithm detection system to obtain the target product and the product identification corresponding to the target product, thereby realizing real-time monitoring and detection of the target product, facilitating the identification and tracking of the position and status of the product through the product identification, and performing position detection on the target product through a preset algorithm detection system to realize real-time tracking of the position of the target product, thereby realizing comprehensive monitoring of the product detection process. When it is detected that the target product has not arrived at the preset workstation, the monitoring video is processed based on the preset operation guidance process. For analysis, the overall inspection process of the target product is identified to obtain the inspection action information, thereby realizing the identification of the staff's inspection actions, and being able to judge whether the staff's actions are standardized and whether the product inspection is carried out in accordance with the requirements, further realizing the overall monitoring of the inspection process of the target product, improving the safety of the production process, and improving the quality of the inspected products. When it is determined that the inspection action information meets the preset operating guidance conditions, it means that the target product has passed the inspection, that is, the target product meets the production specification requirements. The product identification can be bound to the target product, and the monitoring video can be intercepted according to the inspection start time and the inspection end time to obtain the target inspection clip, so as to realize the recording of the entire inspection process of the target product, ensure that each product is produced in accordance with the prescribed standards and processes, and when there is a problem with the product, the marked inspection clip can be used to quickly locate the specific production link, which is convenient for responsibility tracing and problem solving. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a specific flow chart of the production monitoring method provided in the embodiment of the present application;

[0037] Figure 2 It is a flowchart of a specific method for performing picture detection on a surveillance video through a preset algorithm detection system provided in an embodiment of the present application;

[0038] Figure 3 It is a flowchart of a specific method for performing position detection on a target product through a preset algorithm detection system provided by an embodiment of the present application;

[0039] Figure 4 It is a flowchart of a specific method for performing behavior analysis on a surveillance video based on a preset operation guidance process provided by an embodiment of the present application;

[0040] Figure 5is a flowchart of a specific method for determining whether detection action information satisfies a preset operation guidance condition provided in an embodiment of the present application;

[0041] Figure 6 is a flow chart of a production monitoring method provided by another embodiment of the present application;

[0042] Figure 7 is a flow chart of a production monitoring method provided by another embodiment of the present application;

[0043] Figure 8 It is a schematic diagram of the hardware structure of the controller provided in the embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0045] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0046] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0047] A production monitoring method provided in an embodiment of the present application can be applied to a terminal, can be applied to a server side, and can also be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, or a smart watch, etc.; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDNs), and big data and artificial intelligence platforms; the software can be an application that implements the above method, etc., but is not limited to the above forms.

[0048] Embodiments of the present application can be used in numerous general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer controllers, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0049] In the automated equipment production line, there are some manual workstations for product quality inspection, and operators need to conduct product quality inspection according to the product inspection work instructions. However, there is a lack of effective supervision and recording mechanism for whether the operators follow the product inspection work instructions for quality inspection and whether the necessary inspections are carried out. This lack of monitoring leads to the inability to track and evaluate the work behavior of operators in real time, making it impossible to ensure that every product has undergone strict quality inspections. The outflow of defective products not only affects the final quality of the product, but may also lead to a decline in customer trust and damage the company's reputation.

[0050] At present, some companies install surveillance cameras at manual workstations to record videos 24 hours a day. When problems occur, they check the surveillance playback. This has the problem of low efficiency in post-analysis and video review, and due to the lack of a traceability mechanism, it is difficult to determine the responsibility once a product problem occurs, which leads to lax product quality control and increased recall costs.

[0051] In order to solve the above problems, the present embodiment proposes a production monitoring method, a controller, a production area monitoring system and a storage medium. The production area monitoring system of the embodiment of the present application includes at least one acquisition device, which is used to acquire images of products on the production line. In the process of production monitoring of products, the embodiment of the present application first obtains the monitoring video of the acquisition device, and then performs image detection on the monitoring video through a preset algorithm detection system to obtain the target product and the product identification corresponding to the target product, thereby realizing real-time monitoring and detection of the target product, facilitating the identification and tracking of the position and status of the product through the product identification, and performing position detection on the target product through a preset algorithm detection system to realize real-time tracking of the position of the target product, thereby realizing comprehensive monitoring of the product detection process, and when it is detected that the target product has not arrived at the preset workstation, the monitoring video is processed based on the preset operation guidance process. For analysis, the overall inspection process of the target product is identified to obtain the inspection action information, thereby realizing the identification of the staff's inspection actions, and being able to judge whether the staff's actions are standardized and whether the product inspection is carried out in accordance with the requirements, further realizing the overall monitoring of the inspection process of the target product, improving the safety of the production process, and improving the quality of the inspected products. When it is determined that the inspection action information meets the preset operating guidance conditions, it means that the target product has passed the inspection, that is, the target product meets the production specification requirements. The product identification can be bound to the target product, and the monitoring video can be intercepted according to the inspection start time and the inspection end time to obtain the target inspection clip, so as to realize the recording of the entire inspection process of the target product, ensure that each product is produced in accordance with the prescribed standards and processes, and when there is a problem with the product, the marked inspection clip can be used to quickly locate the specific production link, which is convenient for responsibility tracing and problem solving.

[0052] Please refer to Figure 1 , Figure 1 It is a flowchart of a specific method of the production monitoring method provided in an embodiment of the present application. In some embodiments, the production monitoring method is applied but not limited to a production area monitoring system, and the method includes but is not limited to steps S101 to S104.

[0053] In some embodiments, the production area monitoring system includes at least one acquisition device, which is used to acquire product videos on the production line, and the production area monitoring system also includes a transportation device, which is used to transport the products to different detection modules.

[0054] It can be understood that the transportation device in the embodiment of the present application can be a conveyor belt, a mobile robot, etc., and the embodiment of the present application does not make specific limitations.

[0055] Step S101, obtaining a monitoring video from a collection device, where the monitoring video is obtained by collecting images of products on a production line by the collection device.

[0056] In step S101 of some embodiments, a monitoring video of a collection device is acquired to facilitate subsequent analysis of the monitoring video to determine the detection status of products on the production line.

[0057] It should be noted that the acquisition device in the embodiment of the present application can be an analog camera, a network camera, an infrared camera, a panoramic camera, etc., and the embodiment of the present application does not make any specific restrictions.

[0058] Step S102, performing image detection on the surveillance video through a preset algorithm detection system to obtain a target product and a product identification corresponding to the target product, and performing position detection on the target product through a preset algorithm detection system.

[0059] In step S102 of some embodiments, the monitoring video is detected by a preset algorithm detection system, which can quickly locate the object in the video, improve the analysis efficiency of the monitoring video, and accurately identify whether there are products being transported on the conveyor belt. The algorithm detection system then assigns a product number to the target product, obtains the target product and the product identification corresponding to the target product, facilitates subsequent detection operations through the product identification, and detects the position of the target product through the preset algorithm detection system, thereby realizing real-time monitoring of the target product, achieving safety and quality control, providing more refined, efficient and reliable production management means, and further improving resource utilization.

[0060] Step S103, when it is detected that the target product has not arrived at the preset workstation, a behavior analysis is performed on the monitoring video based on a preset operation guidance process to obtain detection action information, wherein the detection action information includes a detection start time and a detection end time.

[0061] In step S103 of some embodiments, when it is detected that the target product has not arrived at the preset workstation, a behavioral analysis is performed on the monitoring video based on the preset operation instruction process, that is, it is determined whether the detection action of the target product in the monitoring video is consistent with the step process and the target action in the operation instruction process, and the detection time of the target product is recorded to obtain the detection action information, so as to realize the monitoring of the whole detection process of the target product, and can record the action information and time information of the target product detection process, so as to facilitate the subsequent judgment of whether the detection process of the target product is complete and the specific action details in the detection process, and facilitate the responsibility tracing and problem solving.

[0062] It should be noted that the operation instruction process refers to the specific step process of standard operation. The operation instruction process provides a clear workflow and operation steps, and the standard operation that should be performed in each operation step. Specifically, the operation instruction process of the embodiment of the present application includes multiple detection steps, and each detection step corresponds to a different target action. Performing behavioral analysis on the target product in accordance with the operation instruction process can ensure the quality of work, reduce errors and quality problems, and further improve the quality of products or services.

[0063] Step S104, when it is determined that the detection action information meets the preset operation guidance conditions, the product identification is bound to the target product, and the monitoring video is intercepted according to the detection start time and the detection end time to obtain the target detection segment.

[0064] In step S104 of some embodiments, when it is determined that the detection action information satisfies the preset operating instructions, it means that the detection action information satisfies all the requirements in the operating instructions, that is, the target product meets the detection requirements and standards. At this time, the product identification is bound to the target product to achieve one-to-one binding of the product identification and the target product, which is convenient for subsequent rapid tracing of the detection process of the target product, and the monitoring video is captured according to the detection start time and the detection end time, that is, the overall process of detecting the target product is captured, and the entire detection process of the target product is recorded to obtain a target detection fragment, ensuring that each product is produced in accordance with the prescribed standards and processes, and when a problem occurs with the product, the specific production link can be quickly located through the labeled detection fragment, which is convenient for responsibility tracing and problem solving.

[0065] It should be noted that in the process of binding the product identification with the target product, the embodiment of the present application may bind the product identification with the product code or graphic code of the target product. After obtaining the target detection fragment, the embodiment of the present application may also upload the target detection fragment to the manufacturing management system for subsequent data tracing.

[0066] Please refer to Figure 2 , Figure 2 It is a flowchart of a specific method for performing picture detection on a surveillance video through a preset algorithm detection system provided in an embodiment of the present application, and the method includes but is not limited to steps S201 to S204.

[0067] Step S201: input the monitoring video into the algorithm detection system, so that the algorithm detection system decomposes the monitoring video and outputs multiple regional video frames.

[0068] Step S202: for each regional video frame, perform region detection on the regional video frame through an algorithm detection system to obtain a boundary region.

[0069] Step S203: performing product recognition on the regional video frame based on the boundary region to obtain the target product.

[0070] Step S204: assign a product number to the target product through the algorithm detection system to obtain a product identification corresponding to the target product.

[0071] In steps S201 to S204 of some embodiments, in the process of performing picture detection on the surveillance video by a preset algorithm detection system, the embodiment of the present application first inputs the surveillance video into the algorithm detection system so that the algorithm detection system performs video decomposition on the surveillance video, that is, the surveillance video is decomposed into multiple frame images by the algorithm detection system, and multiple regional video frames are output. By decomposing the video frames, objects in the video can be quickly located, thereby improving the analysis efficiency of the surveillance video. For each regional video frame, the algorithm detection system performs regional detection on the regional video frame. Specifically, the embodiment of the present application first performs edge detection on each regional video frame, divides the effective detection area, and then performs effective detection. The area is delineated to obtain the boundary area, and the video frame is finely divided to facilitate the subsequent identification of the target product. The regional video frame is then identified based on the boundary area. Specifically, the embodiment of the present application can identify the product in the regional video frame through a target detector or a target recognition algorithm to obtain the target product, thereby achieving high-precision detection of the product. It can accurately identify whether there is a product being transported on the conveyor belt, and then assign a product number to the target product through the algorithm detection system to obtain a product identification corresponding to the target product. The product identification of each target product is different, which is convenient for subsequent detection operations through product identification, can reduce manual intervention, and improve production efficiency.

[0072] It is understandable that in the embodiment of the present application, product identification can be performed through the YOLO (You On ly Look Once) detection model, and a product number can be assigned to each target product through the snowflake algorithm, and the embodiment of the present application does not make specific limitations.

[0073] Please refer to Figure 3 , Figure 3 It is a flowchart of a specific method for performing position detection on a target product through a preset algorithm detection system provided in an embodiment of the present application, and the method includes but is not limited to steps S301 to S302.

[0074] Step S301: annotate the target product through an algorithm detection system to obtain a detection frame.

[0075] Step S302: Track the position of the target product according to the detection frame and the product identification.

[0076] In steps S301 to S302 of some embodiments, in the process of performing position detection on the target product through a preset algorithm detection system, the embodiment of the present application first marks the target product through the algorithm detection system, generates a detection frame corresponding to the target product, and then tracks the position of the target product based on the detection frame and the product identification, thereby realizing real-time monitoring of the target product, achieving safety and quality control, providing more refined, efficient and reliable production management means, and further improving resource utilization.

[0077] Please refer to Figure 4 , Figure 4 It is a flowchart of a specific method for performing behavior analysis on surveillance videos based on a preset operation guidance process provided in an embodiment of the present application, and the method includes but is not limited to steps S401 to S405.

[0078] It should be noted that the operation guidance process includes multiple detection steps, and each detection step corresponds to a different target action.

[0079] Step S401, extract features from the surveillance video to obtain a detection area.

[0080] Step S402, when there is a preset operating object in the detection area, action detection is performed on the detection area according to all target actions to obtain a detection set, wherein the detection set includes detection results and detection times of video frames corresponding to the target actions one by one.

[0081] Step S403: classify and count all video frames in the detection set according to the detection result to obtain at least one classification result.

[0082] Step S404, determining the detection start time and detection end time of the surveillance video according to the detection time.

[0083] Step S405, obtaining detection action information according to the detection start time, detection end time, classification result and detection time.

[0084] In steps S401 to S405 of some embodiments, in the process of performing behavioral analysis on the monitoring video based on the preset operation guidance process, the embodiment of the present application first performs feature extraction on the monitoring video to extract the area where the manual inspection process is located to obtain the inspection area, so as to facilitate subsequent targeted inspection of the inspection area to avoid interference from other areas, and then determines whether there is a preset operation object in the inspection area. When there is a preset operation object in the inspection area, it means that there is a preset operation object in the inspection area that is inspecting the product. At this time, action detection is performed on the inspection area according to all target actions, that is, it is determined whether the action of the preset operation object in the inspection area is consistent with the target action, the detected action is labeled, a detection label corresponding to the action is generated, and the inspection time of each preset operation object detecting the target product is recorded to obtain a detection set, thereby realizing the recording of the entire product inspection process, and being able to accurately record the specific actions and inspection results of the product inspection process. The detection time can be further judged whether the staff operates according to the requirements or whether the operation is standardized, and then all the video frames in the detection set are classified and counted according to the detection results to realize the classification of the video frames corresponding to the target action, and obtain at least one classification result, so as to realize accurate classification of different video frames, so as to determine the number of video frames with different target actions. After that, the detection start time and the detection end time of the monitoring video are determined according to the detection time, that is, the time when the staff starts the detection and the time when the detection ends, so as to realize the recording of the start and end time of the detection, and then the detection action information is obtained according to the detection start time, the detection end time, the classification result and the detection time, so as to realize the monitoring of the whole detection process of the target product, and can record the action information and time information during the detection process of the target product, so as to facilitate the subsequent judgment of whether the detection process of the target product is complete and the specific action details during the detection process, so as to facilitate responsibility tracing and problem solving.

[0085] It should be noted that in the embodiments of the present application, the target product is detected by a preset operating object, wherein the preset operating object can be an object operated by a human body, a machine, etc., and the embodiments of the present application do not impose specific restrictions. Specifically, the embodiments of the present application take the preset operating object as a human hand as an example for explanation. When the preset operating object exists in the detection area, it means that the human hand in the detection area is in contact with the target product.

[0086] It is worth noting that the embodiment of the present application monitors the detection process of the target product according to the operation guidance process, and the operation guidance process includes multiple detection steps, each detection step includes different target actions. The embodiment of the present application takes the target action as the detection object, monitors whether the preset operation object in the detection area performs the target action on the target product, and records the specific time of performing the target action, so as to realize full monitoring of the detection process and facilitate data tracing.

[0087] Please refer to Figure 5 , Figure 5 It is a flowchart of a specific method for determining that detection action information meets preset operation guidance conditions provided in an embodiment of the present application, and the method includes but is not limited to steps S501 to S502.

[0088] It should be noted that the work guidance conditions include detection sequence conditions and duration intervals corresponding to the target actions, wherein the detection sequence conditions are used to determine whether the detection action information is executed in the order of steps set in the work guidance process, and the duration interval is used to determine the duration of each type of target action in the action information.

[0089] Step S501, determining the action duration corresponding to each classification result according to the detection time in the detection action information.

[0090] Step S502: When the detection time of each video frame in the detection action information meets the sequence condition and the action duration is within the duration interval, it is determined that the detection action information meets the preset operation guidance condition.

[0091] In steps S501 to S502 of some embodiments, in the process of determining whether the detected action information meets the preset job guidance conditions, the embodiments of the present application first determine the action duration corresponding to each classification result according to the detection time in the detected action information, that is, the duration of each type of action, and then judge the occurrence order of the video frames according to the detection time of each video frame in the detected action information, further judge the occurrence order of all actions in the detected action information, and judge whether the action duration corresponding to each classification result meets its corresponding duration interval. When the detection time of each video frame in the detected action information meets the sequence condition and the action duration is in the duration interval, it means that the detected action information is executed in the order of the job guidance process, and the execution duration of each target action meets the specified duration requirement. At this time, it is determined that the detected action information meets the preset job guidance conditions.

[0092] It should be noted that the duration intervals in the embodiment of the present application are set according to the number of target actions. When the number of target actions is three, duration intervals are set for the three target actions respectively, that is, the number of duration intervals is three; when the number of target actions is five, duration intervals are set for the five target actions respectively, that is, the number of duration intervals is five.

[0093] Please refer to Figure 6 , Figure 6 It is a flowchart of a production monitoring method provided by another embodiment of the present application, and the method includes but is not limited to step S601.

[0094] It should be noted that step S601 occurs after the position of the target product is detected by a preset algorithm detection system.

[0095] Step S601: When it is detected that the target product arrives at a preset workstation, a first alarm message is generated.

[0096] In step S601 of some embodiments, after the position of the target product is detected by a preset algorithm detection system, when it is detected that the target product has arrived at the preset workstation, it means that the target product has been directly moved to the next workstation without detection. At this time, a first alarm message is generated, triggering the alarm to automatically alarm, thereby reminding the staff that the target product has not been detected.

[0097] It should be noted that the first alarm information in the embodiment of the present application may be sound alarm information, light alarm information, etc., and the embodiment of the present application does not impose any specific limitation.

[0098] Please refer to Figure 7 , Figure 7 It is a flowchart of a production monitoring method provided by another embodiment of the present application, and the method includes but is not limited to steps S701 to S702.

[0099] It should be noted that step S701 to step S702 occur after the behavior analysis is performed on the surveillance video to obtain the detection action information.

[0100] Step S701, when it is determined that the detection action information does not meet the preset operation guidance conditions, a violation event that does not meet the operation guidance conditions is determined according to the detection action information.

[0101] Step S702, record the violation event and generate second alarm information.

[0102] In steps S701 to S702 of some embodiments, after performing behavioral analysis on the surveillance video, when it is determined that the detection action information does not meet the preset operating instruction conditions, it means that the current detection action information may not be executed in the order of the operating instruction process, or there is a situation where the execution duration of the target action does not meet the specified duration requirement. At this time, the violation event that does not meet the operating instruction conditions is determined based on the detection action information, and the violation event that does not meet the requirements can be quickly discovered and responded to, and the violation event is recorded, and a second alarm information is generated, so that the staff can view the details of the target product, ensuring that only products that meet the quality requirements can pass the production line, thereby improving production efficiency.

[0103] In some embodiments, the production area monitoring system also includes a transportation device. The embodiment of the present application is described using the transportation device as a conveyor belt. When it is determined that the detection action information does not meet the preset operation guidance conditions, the embodiment of the present application will also control the conveyor belt to stop running, so as to prevent products that have not passed the inspection from entering the market and improve the quality of the products.

[0104] It is worth noting that after generating the second alarm information, the embodiment of the present application can re-test the target products that do not meet the operating guidance conditions, thereby improving the detection accuracy, achieving comprehensive detection of the target products, and avoiding the occurrence of missed detection or outflow of undetected products.

[0105] Reference Figure 8 , Figure 8 It is a schematic diagram of the hardware structure of the controller provided in the embodiment of the present application.

[0106] Combine the following Figure 8 The hardware structure of the controller is described in detail. The controller includes: a processor 910 , a memory 920 , an input / output interface 930 , a communication interface 940 and a bus 950 .

[0107] The processor 910 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0108] The memory 920 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 920 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 920, and the processor 910 calls and executes the production monitoring method of the embodiment of this application;

[0109] Input / output interface 930, used to implement information input and output;

[0110] The communication interface 940 is used to realize the communication interaction between the device and other devices, and the communication can be realized by wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WI FI, Bluetooth, etc.); and the bus 950 is used to transmit information between the various components of the device (such as the processor 910, the memory 920, the input / output interface 930 and the communication interface 940);

[0111] The processor 910 , the memory 920 , the input / output interface 930 , and the communication interface 940 are connected to each other in communication within the device via a bus 950 .

[0112] In some embodiments, the present application also provides a production area monitoring system, the production area monitoring system includes at least one acquisition device and Figure 8 Controller shown.

[0113] In some embodiments, the production area monitoring system also includes a transportation device, which is used to transport products to different detection modules. The transportation device can be a conveyor belt, a mobile robot, etc., and the embodiments of the present application do not impose specific limitations.

[0114] An embodiment of the present application also provides a storage medium, which is a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a computer, the computer is used to execute the production monitoring method in the above embodiment of the present application.

[0115] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0116] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0117] It can be understood by those skilled in the art that Figures 2 to 8 The technical solutions shown in the figure do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than those shown in the figure, or a combination of certain steps, or different steps.

[0118] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0119] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0120] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0121] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0122] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0123] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0124] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0125] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including multiple instructions to enable a controller (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0126] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A production monitoring method, characterized in that: Applied to a production area monitoring system, the production area monitoring system includes at least one acquisition device; the method includes: Acquire a monitoring video of the acquisition device, wherein the monitoring video is obtained by the acquisition device acquiring images of products on a production line; Performing image detection on the surveillance video through a preset algorithm detection system to obtain a target product and a product identification corresponding to the target product, and performing position detection on the target product through a preset algorithm detection system; When it is detected that the target product has not arrived at the preset workstation, a behavior analysis is performed on the monitoring video based on a preset operation guidance process to obtain detection action information, wherein the detection action information includes a detection start time and a detection end time; When it is determined that the detection action information meets the preset operation guidance conditions, the product identification is bound to the target product, and the monitoring video is intercepted according to the detection start time and the detection end time to obtain a target detection segment.

2. The production monitoring method according to claim 1, characterized in that: The monitoring video is detected by a preset algorithm detection system to obtain a target product and a product identification corresponding to the target product, including: Inputting the surveillance video into the algorithm detection system so that the algorithm detection system performs video decomposition on the surveillance video and outputs a plurality of regional video frames; For each of the regional video frames, performing region detection on the regional video frame by using the algorithm detection system to obtain a boundary region; Perform product recognition on the regional video frame based on the boundary region to obtain a target product; The algorithm detection system assigns a product number to the target product to obtain a product identification corresponding to the target product.

3. The production monitoring method according to claim 1, characterized in that: The detecting of the position of the target product by a preset algorithm detection system includes: The target product is labeled by the algorithm detection system to obtain a detection frame; The position of the target product is tracked according to the detection frame and the product identification.

4. The production monitoring method according to claim 1, characterized in that: The operation guidance process includes a plurality of target actions; the behavior analysis of the monitoring video based on the preset operation guidance process to obtain the detection action information includes: Extracting features from the surveillance video to obtain a detection area; When there is a preset operating object in the detection area, action detection is performed on the detection area according to all the target actions to obtain a detection set, wherein the detection set includes detection results and detection times of video frames corresponding to the target actions one by one; According to the detection result, all video frames in the detection set are classified and counted to obtain at least one classification result; Determine the detection start time and detection end time of the monitoring video according to the detection time; Detection action information is obtained according to the detection start time, the detection end time, the classification result and the detection time.

5. The production monitoring method according to claim 4, characterized in that: The operation guidance conditions include detection sequence conditions and duration intervals corresponding to the target actions; The determining that the detection action information meets a preset operation guidance condition includes: Determine the action duration corresponding to each of the classification results according to the detection time in the detection action information; When the detection time of each video frame in the detection action information meets the sequence condition and the action duration is within the duration interval, it is determined that the detection action information meets the preset operation guidance condition.

6. The production monitoring method according to claim 1, characterized in that: After the target product is detected by a preset algorithm detection system, the method further includes: When it is detected that the target product arrives at the preset workstation, a first alarm message is generated.

7. The production monitoring method according to claim 1, characterized in that: After performing behavior analysis on the surveillance video to obtain detection action information, the method further includes: When it is determined that the detection action information does not meet the preset operation guidance condition, determining a violation event that does not meet the operation guidance condition according to the detection action information; The violation event is recorded and a second alarm message is generated.

8. A controller, characterized in that: The controller includes a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor is used to execute the production monitoring method according to any one of claims 1 to 7.

9. A production area monitoring system, characterized in that: The device comprises at least one acquisition device and a controller as claimed in claim 8.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and the computer-readable storage medium stores a computer program. When the computer program is executed by a computer, the computer is used to execute the production monitoring method according to any one of claims 1 to 7.

Images