A management method, device, computer device, and storage medium for a micro-nano processing platform

Abstract

The present application provides a management method, device, computer device and storage medium for a micro-nano processing platform. The method includes using at least one anomaly recognition mechanism configured for each camera module to respectively perform anomaly recognition on the monitoring area corresponding to each camera module to obtain an anomaly recognition result; determining an associated entity according to the anomaly recognition result, and updating the anomaly record of the associated entity; if the anomaly record meets the upgrade condition, then raising the processing level for the associated entity, otherwise, maintaining the processing level for the associated entity; and using the processing means corresponding to the current processing level to process the associated entity. This method provides strong support for the efficient unmanned management of the platform, significantly improving the management efficiency and reducing the management cost.

Description

Technical Field

[0001] This application relates to the field of semiconductor micro-nano processing technology, and particularly to a management method, device, computer device, and storage medium for a micro-nano processing platform. Background Art

[0002] With the development of semiconductor technology, micro-nano processing technology has become one of the key technologies in the semiconductor industry. In a micro-nano processing platform, lean management of internal personnel, items, etc. is required to improve the reliability, durability, safety, etc. of the micro-nano processing platform. Traditional management methods require administrators to be deeply involved in identification, processing, etc., which not only takes time and effort but also cannot avoid errors and omissions, greatly increasing management costs and being unfavorable to the long-term development of the micro-nano processing platform. Summary of the Invention

[0003] The purpose of this application aims to at least solve one of the above technical defects, especially the problems of high cost and low efficiency caused by the deep involvement of administrators in platform management in traditional technologies.

[0004] In a first aspect, this application provides a management method for a micro-nano processing platform. The micro-nano processing platform is provided with a plurality of camera modules. The management method includes:

[0005] Using at least one anomaly recognition mechanism configured for each camera module, respectively perform anomaly recognition on the monitoring area corresponding to each camera module to obtain an anomaly recognition result;

[0006] According to the anomaly recognition result, determine the associated entity and update the anomaly record of the associated entity;

[0007] If the anomaly record meets the upgrade condition, then raise the processing level for the associated entity; otherwise, maintain the processing level for the associated entity;

[0008] Adopt the processing means corresponding to the current processing level to process the associated entity.

[0009] In one embodiment, the processing levels include a first processing level and a second processing level from low to high. Adopting the processing means corresponding to the current processing level to process the associated entity includes:

[0010] If the processing level is the first processing level, then send a reminder to the associated entity;

[0011] If the processing level is the second processing level, then report the anomaly record to the administrator.

[0012] In one embodiment, the anomaly record includes the number of anomalies occurring in the current statistical period. The upgrade conditions include:

[0013] If the number of times is greater than a preset threshold, upgrade from the first processing level to the second processing level.

[0014] In one embodiment, the anomaly recognition mechanism includes a personnel anomaly recognition mechanism, and the anomaly recognition result includes a personnel anomaly recognition result. Using the personnel anomaly recognition mechanism to perform anomaly recognition on the monitoring area includes:

[0015] Input the picture frames captured by the camera module into the first detection model, and obtain a first detection result according to the output of the first detection model; the first detection model is used to generate a personnel detection box for the personnel appearing in the picture;

[0016] Assign a unique identifier corresponding to the personnel identity to each first detection result;

[0017] Regard the first detection result containing abnormal items as a personnel anomaly recognition result.

[0018] In one embodiment, the first detection model is further used to generate corresponding abnormal wearing detection boxes for multiple abnormal wearing items respectively, and the first detection result includes a wearing detection result; obtaining the first detection result according to the output of the first detection model includes:

[0019] In the case where the first detection model outputs at least one personnel detection box and at least one abnormal wearing detection box, match the abnormal wearing detection box with the personnel detection box;

[0020] If there is a matching abnormal wearing detection box for the personnel detection box, regard the personnel detection box and its matching abnormal wearing detection box as a wearing detection result;

[0021] If there is no matching abnormal wearing detection box for the personnel detection box, directly regard the personnel detection box as a wearing detection result.

[0022] In one embodiment, before regarding the first detection result containing abnormal items as a personnel anomaly recognition result, it further includes:

[0023] If the wearing detection result includes an abnormal wearing detection box, judge the type of the abnormal wearing detection box;

[0024] If the type of the abnormal wearing detection box is clothing type, add the recognition object corresponding to the abnormal wearing detection box to the abnormal item corresponding to the wearing detection result;

[0025] If the type of the abnormal wearing detection box is work card type, judge whether the personnel corresponding to the personnel detection box is facing forward. If so, add not wearing a work card to the abnormal item corresponding to the wearing detection result.

[0026] In one embodiment, judging whether the personnel corresponding to the personnel detection box is facing forward includes:

[0027] When a face is included in the personnel detection box and the first ratio is greater than the first preset threshold, it is determined that the personnel corresponding to the personnel detection box is facing forward; the first ratio is the ratio between the horizontal distance between the human shoulder key points in the personnel detection box and the width of the personnel detection box.

[0028] In one embodiment, if the type of the abnormal wearing detection box is clothing-related, the recognition objects corresponding to the abnormal wearing detection box include at least one of not wearing a clean suit, not wearing gloves, not wearing safety glasses, and not wearing a mask.

[0029] In one embodiment, matching the abnormal wearing detection box with the personnel detection box includes:

[0030] For any abnormal wearing detection box, determine the first area corresponding to the abnormal wearing detection box;

[0031] Calculate the second area corresponding to the intersection area between the abnormal wearing detection box and the personnel detection box to be matched respectively;

[0032] If the ratio between the second area and the first area is greater than the second preset threshold, it is determined that the abnormal wearing detection box matches the personnel detection box.

[0033] In one embodiment, the first detection result includes a position detection result; according to the output of the first detection model, obtaining the first detection result includes:

[0034] When the first detection model outputs at least one personnel detection box, input the picture frame into the second detection model to obtain at least one group of human key points;

[0035] Match the group of human key points with the personnel detection box;

[0036] Take the ankle key point positions in the personnel detection box and the group of human key points matched therewith as a position detection result.

[0037] In one embodiment, before taking the first detection result including abnormal items as an abnormal recognition result, it further includes:

[0038] If the ankle key point position is within the preset electronic fence area, add position intrusion to the abnormal item corresponding to the position detection result.

[0039] In one embodiment, the personnel in the micro-nano processing platform wears a recognition pattern combination composed of a preset number of different recognition patterns, and the recognition pattern combination corresponds to a unique detection identifier. Assigning a unique identifier corresponding to the personnel identity to each first detection result includes:

[0040] Input the picture frame into the third detection model; the third detection model is used to generate a pattern detection box for the recognition pattern appearing in the picture;

[0041] In the case where the third detection model outputs multiple pattern detection boxes, cluster the pattern detection boxes, and determine the cluster with a preset number of pattern detection boxes as the target cluster;

[0042] Determine the central position of each target cluster, and match each target cluster with the person detection box according to the central position and the position of each person detection box;

[0043] For any person detection box, if there is a matching target cluster, determine a unique detection identifier according to the matching target cluster, and use the unique detection identifier as the unique identifier.

[0044] In one embodiment, assigning a unique identifier corresponding to the person identity to each first detection result further includes:

[0045] Process the picture frame using a multi-object tracking algorithm to obtain the unique tracking identifier corresponding to each pedestrian detection box;

[0046] For any person detection box, if there is a matching target cluster, determine a unique detection identifier according to the matching target cluster, and use the unique detection identifier as the unique identifier, further includes:

[0047] For any person detection box, if there is a matching target cluster, determine a unique detection identifier according to the matching target cluster, and use the unique detection identifier as the unique identifier, and bind the unique detection identifier with the unique tracking identifier;

[0048] If there is no matching target cluster, obtain the unique identifier according to the unique detection identifier bound by the unique tracking identifier of the person detection box.

[0049] In one embodiment, matching each target cluster with the person detection box includes:

[0050] Use the Hungarian algorithm to match each target cluster with the person detection box.

[0051] In one embodiment, a sound generating unit is provided on the camera module to give a reminder to the associated subject, including:

[0052] Take the camera module that reports the first detection result including the abnormal item as the first target camera module, and instruct the first target camera module to give a voice broadcast according to the abnormal item and the person identity corresponding to the unique identifier.

[0053] In one embodiment, a sounding unit and / or a vibration unit are provided on the work card worn by the personnel in the micro-nano processing platform to give a reminder to the associated entity, including:

[0054] Select the work card corresponding to the associated entity as the target work card, and instruct the target work card to give a voice broadcast and / or generate a vibration according to the exception item.

[0055] In one embodiment, the anomaly recognition mechanism includes a desktop anomaly recognition mechanism, the anomaly recognition result includes a desktop anomaly recognition result, and the camera module corresponding to the desktop anomaly recognition mechanism is used to photograph the desktop of the experimental bench. The desktop anomaly recognition mechanism is used to recognize anomalies in the monitoring area, including:

[0056] When the desktop inspection condition is met, input the picture frame captured by the camera module into the fourth detection model to obtain multiple desktop object detection frames; the fourth detection model is used to generate desktop object detection frames for various desktop objects appearing in the picture;

[0057] Match the images within each desktop object detection frame with the reference image library; the reference image library stores the reference images of each desktop tool;

[0058] If there is a desktop object detection frame that cannot be matched, use the experimental bench information corresponding to the camera module and the information of the person who reserved the experimental bench most recently as the desktop anomaly recognition result.

[0059] In one embodiment, a sounding unit is provided on the camera module to give a reminder to the associated entity, including:

[0060] Select the camera module that reports the desktop anomaly recognition result as the starting camera module;

[0061] Obtain the leaving time difference according to the time of reporting the desktop anomaly recognition result and the reservation end time of the experimental bench corresponding to the starting camera module;

[0062] Obtain the leaving distance according to the leaving time difference and the preset walking speed;

[0063] Determine the camera modules whose distances from the starting camera module are within the leaving distance as the second target camera modules;

[0064] Control the second target camera modules to give a voice broadcast according to the experimental bench information and the personnel information.

[0065] In one embodiment, the desktop detection condition includes that the time reaches the end of the reservation time of the experimental bench corresponding to the camera module.

[0066] In one embodiment, the anomaly recognition mechanism includes a channel debris anomaly recognition mechanism, the anomaly recognition result includes a channel debris anomaly recognition result, and the camera module corresponding to the channel debris anomaly recognition mechanism is used to capture the platform channel. Using the channel debris anomaly recognition mechanism to perform anomaly recognition on the monitoring area includes:

[0067] Using the picture frames captured by the camera module to update the long-term background model and the short-term background model respectively; both the long-term background model and the short-term background model are used to extract the background image in the picture, and the learning rate of the long-term background model is lower than that of the short-term background model;

[0068] Inputting the picture frame into the long-term background model to obtain a first image, and inputting the picture frame into the short-term background model to obtain a second image;

[0069] Comparing the first image and the second image to determine whether the difference region in the obtained difference image is a debris region;

[0070] According to the determined debris region, obtain the channel debris anomaly recognition result.

[0071] In one embodiment, determining whether the difference region is a debris region includes:

[0072] Inputting the difference image into a fifth detection model; the fifth detection model is used to generate a filtering detection box for the target filtering image in the image;

[0073] If there is a corresponding filtering detection box for each difference region in the difference image, it is determined that there is no debris region in the difference image;

[0074] Otherwise, determine the difference region without a corresponding filtering detection box as the debris region.

[0075] In one embodiment, according to the anomaly recognition result, determining the associated entity includes:

[0076] Determine the channel area corresponding to the camera module that reports the channel debris anomaly recognition result as the target channel area;

[0077] According to the target channel area, determine the corresponding cleaning person in charge;

[0078] Determine the cleaning person in charge corresponding to the target channel area as the associated entity.

[0079] In a second aspect, the present application provides a management device for a micro-nano processing platform. The micro-nano processing platform is provided with a plurality of camera modules. The management device includes:

[0080] A result acquisition module, which uses at least one anomaly recognition mechanism configured for each camera module to perform anomaly recognition on the monitoring area corresponding to each camera module respectively, and obtains the anomaly recognition result;

[0081] A recording module, configured to determine associated entities according to the anomaly recognition results and update the anomaly records of the associated entities;

[0082] A processing level adjustment module, configured to upgrade the processing level of the associated entity if the anomaly record meets the upgrade conditions, otherwise, maintain the processing level of the associated entity;

[0083] A processing module, which uses the processing means corresponding to the current processing level to process the associated entity.

[0084] In a third aspect, the present application provides a computer device, including one or more processors and a memory. Computer-readable instructions are stored in the memory. When the computer-readable instructions are executed by one or more processors, the steps of the management method of the micro-nano processing platform in any of the above embodiments are executed.

[0085] In a fourth aspect, the present application provides a storage medium in which computer-readable instructions are stored. When the computer-readable instructions are executed by one or more processors, one or more processors are caused to execute the steps of the management method of the micro-nano processing platform in any of the above embodiments.

[0086] As can be seen from the above technical solutions, the embodiments of the present application have the following advantages:

[0087] Based on the management method of the micro-nano processing platform in this solution, first, the anomaly recognition mechanism configured by each camera module is used to perform anomaly recognition on the monitoring area to obtain anomaly recognition results. Then, associated entities are determined according to these results, and their anomaly records are updated. The system will evaluate whether the anomaly record meets the upgrade conditions. If it meets, the processing level of the associated entity will be upgraded, otherwise, the current level will be maintained. Finally, the system takes corresponding processing means according to the current processing level to process the associated entity. This solution provides strong support for the efficient unmanned management of the platform, significantly improving the management efficiency and reducing the management cost. Through the automated anomaly detection and processing mechanism, the system can monitor the platform environment all-weather and continuously, discover and respond to various anomalies in a timely manner, and improve the platform operation efficiency. The dynamically adjusted processing level ensures the rationalization of the processing means, reduces the participation of administrators, alleviates the management burden, and also improves the management efficiency. Description of the Drawings

[0088] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0089] Figure 1 Schematic flow chart of the management method for the micro-nano processing platform provided by an embodiment of the present application;

[0090] Figure 2 Schematic flow chart of obtaining the personnel anomaly recognition result by using the personnel anomaly recognition mechanism in an embodiment of the present application;

[0091] Figure 3 Schematic flow chart of obtaining the wearing detection result by using the abnormal wearing recognition in the personnel anomaly recognition mechanism in an embodiment of the present application;

[0092] Figure 4 Schematic diagram of detecting abnormal wearing of personnel without masks in an embodiment of the present application;

[0093] Figure 5 Schematic flow chart of adding abnormal items to the wearing detection result in an embodiment of the present application;

[0094] Figure 6 Schematic diagram of detecting whether a person is facing the camera module directly in an embodiment of the present application;

[0095] Figure 7 Schematic flow chart of obtaining the position detection result by using the abnormal position recognition in the personnel anomaly recognition mechanism in an embodiment of the present application;

[0096] Figure 8 Schematic diagram of recognizing a pattern combination in an embodiment of the present application;

[0097] Figure 9 Schematic flow chart of assigning a unique identifier to the first detection result in an embodiment of the present application;

[0098] Figure 10 Schematic flow chart of assigning a unique identifier to the first detection result in another embodiment of the present application;

[0099] Figure 11 Schematic flow chart of obtaining the desktop anomaly recognition result by using the desktop anomaly recognition mechanism in an embodiment of the present application;

[0100] Figure 12 Schematic flow chart of sending a reminder to the associated entity within the desktop anomaly recognition mechanism in an embodiment of the present application;

[0101] Figure 13 Schematic flow chart of obtaining the passage debris anomaly recognition result by using the passage debris anomaly recognition mechanism in an embodiment of the present application;

[0102] Figure 14Schematic diagram of the effect of obtaining the channel debris anomaly recognition result using the channel debris anomaly recognition mechanism in an embodiment of the present application;

[0103] Figure 15 Schematic diagram of the process for determining whether the difference region is a debris region in an embodiment of the present application;

[0104] Figure 16 Schematic diagram of the process for determining the associated entity within the channel debris anomaly recognition mechanism in an embodiment of the present application;

[0105] Figure 17 Internal structure diagram of a computer device provided in an embodiment of the present application. Detailed implementation manners

[0106] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0107] The present application provides a management method for a micro-nano processing platform. The micro-nano processing platform is provided with a plurality of camera modules, which respectively cover different regions within the micro-nano processing platform and can photograph, identify, and manage abnormal phenomena within their shooting ranges. Specifically, the management method in the present application includes steps S102 to S108.

[0108] S102: Use at least one anomaly recognition mechanism configured for each camera module to respectively perform anomaly recognition on the monitoring area corresponding to each camera module to obtain an anomaly recognition result.

[0109] It can be understood that the anomaly recognition mechanism refers to algorithms and methods for detecting and identifying abnormal states or behaviors based on computer vision technology. In the micro-nano processing platform, strict management is required for personnel and items. Each anomaly recognition mechanism is specifically used to identify specific types of abnormal conditions and obtain anomaly recognition results. The camera module can capture the corresponding monitoring area of the device, and call the configured anomaly recognition mechanism to process the captured image frames, thus automatically completing the anomaly recognition work and obtaining the anomaly recognition results. The management method in this embodiment is applied to the management server. After any camera module obtains the anomaly recognition result, it will report it to the management server. The anomaly recognition mechanism may involve the invocation of a neural network model. Therefore, the camera module in this embodiment needs to have corresponding computing power. In addition, since the monitoring areas corresponding to each camera module are different, the items that need to be detected for abnormal states can also be different. Therefore, the number and types of anomaly recognition mechanisms configured for each camera module can be different, and can be selected according to the installation location, shooting angle, etc. of the camera module. After each camera module obtains the anomaly recognition result

[0110] S104. According to the anomaly recognition result, determine the associated entity and update the anomaly record of the associated entity.

[0111] It can be understood that the associated entity refers to an individual related to the detected anomaly. The anomaly recognition result includes the anomaly type and information related to the associated entity. Some anomaly recognition mechanisms can directly identify the identity information of the associated entity, while some anomaly recognition mechanisms need to call other information in the platform data system to establish the association relationship between the anomaly and the individual. The method of determining the associated entity will vary depending on the anomaly recognition mechanism. For different anomaly recognition mechanisms, more useful information can also be included, such as the location and time of the anomaly occurrence. The anomaly record can be regarded as the file of the associated entity, which records the historical abnormal behaviors of the entity. Updating the anomaly record means that each time an anomaly recognition result is received, according to the information contained therein, the relevant situation of the current anomaly of the associated entity is recorded in the anomaly record corresponding to the associated entity.

[0112] S106. If the anomaly record meets the upgrade condition, then raise the processing level of the associated entity; otherwise, maintain the processing level of the associated entity.

[0113] It can be understood that the upgrade conditions include the rules for triggering the improvement of the processing level. These rules may be based on various factors, including but not limited to: the frequency, severity, duration, scope of influence, etc. of abnormal behaviors. The setting of upgrade conditions needs to balance the sensitivity of the system and the false alarm rate. The processing level reflects the intensity of the processing means adopted for this anomaly and the degree of participation required by the management personnel. It can range from simple automatic reminders to escalating to the management personnel on the reporting platform. After updating the anomaly record, it can be determined whether the anomaly record of the associated entity has triggered the upgrade conditions. If the upgrade conditions are triggered, the processing level is increased; otherwise, the current processing level is maintained. Since the processing level of the entity increases with the accumulation of anomaly records, most anomalies of the entity can be processed by processing means with a low processing level, and this part of the processing does not require the participation of management personnel, greatly reducing the workload of the administrator and improving the efficiency of management work. However, since the higher the processing level, the greater the degree of participation required by the management personnel, if the processing level of each entity keeps increasing, it may ultimately result in the need for the administrator to handle every triggered anomaly personally. Therefore, a processing level reset mechanism can be set up. For example, a restoration period is set, and at the beginning of each restoration period, the processing levels corresponding to all entities are restored to the initial processing levels.

[0114] S108, use the processing means corresponding to the current processing level to process the associated entity.

[0115] It can be understood that there is a corresponding processing means for each processing level. After each judgment on whether the processing level needs to be adjusted, the processing means corresponding to the current processing level can be used to process the associated entity of the anomaly reported this time.

[0116] Based on the management method of the micro-nano processing platform in this embodiment, first, the anomaly recognition mechanism configured by each camera module is used to recognize anomalies in the monitoring area to obtain anomaly recognition results. Then, the associated entities are determined based on these results, and their anomaly records are updated. The system will evaluate whether the anomaly records meet the upgrade conditions. If they do, the processing level for the associated entity is increased; otherwise, the current level is maintained. Finally, the system takes corresponding processing means according to the current processing level to process the associated entity. This solution provides strong support for the efficient unmanned management of the platform, significantly improving management efficiency and reducing management costs. Through the automated anomaly detection and processing mechanism, the system can monitor the platform environment continuously all day long, timely detect and respond to various abnormal situations, and improve the operation efficiency of the platform. The dynamically adjusted processing level ensures the rationalization of the processing means, reduces the degree of administrator participation to reduce the management burden, and also improves management efficiency.

[0117] In one embodiment, the processing levels include a first processing level and a second processing level from low to high. The associated entity is processed using the processing means corresponding to the current processing level, including: if the processing level is the first processing level, a reminder is sent to the associated entity. If the processing level is the second processing level, the anomaly record is reported to the administrator.

[0118] It can be understood that during the platform management process, generally, the platform administrators are teachers, professors, etc. of the research group, and the personnel entering the platform are generally students of the platform and external personnel using the platform. If every anomaly handling requires the participation of the administrator, it will inevitably affect the progress of the administrator's own R & D work. In addition, for each anomaly, the direct connection between the administrator and the associated entity will also cause antagonistic emotions between the two, further increasing the management cost. In this embodiment, a total of two processing levels are set. The first processing level is the most basic processing level. In this processing level, only a reminder will be automatically sent to the associated entity to prompt the associated entity to make rectifications as soon as possible, and the administrator will not participate in this part of the management work. While in the second processing level, the recent behavior of the associated entity triggers the upgrade condition, which requires the administrator's attention and personal participation. Therefore, the anomaly record of the associated entity will be reported to the administrator, and the administrator will participate in handling its recent non-compliant behavior. In a specific embodiment, when updating the anomaly record, the number of anomalies of the associated entity in the current statistical period (such as one day, one week, etc.) will be recorded. If this number is greater than the preset threshold, it means that the associated entity has had anomalies frequently recently, which has a greater impact on the operation of the platform, and will trigger the upgrade of the processing level.

[0119] In one embodiment, the anomaly recognition mechanism includes a personnel anomaly recognition mechanism, which is mainly involved in the management of the personnel appearing in the experiment. The anomaly recognition result includes the personnel anomaly recognition result. Please refer to Figure 2 , and the monitoring area is recognized for anomalies using the personnel anomaly recognition mechanism, including steps S202 to S206.

[0120] S202, input the picture frames captured by the camera module into the first detection model, and obtain the first detection result according to the output of the first detection model. The first detection model is used to generate a personnel detection frame for the personnel appearing in the picture.

[0121] It can be understood that for a camera module configured with a personnel anomaly recognition mechanism, the captured picture frames will be preferentially input into the first detection model. The first detection model is a neural network model obtained by training with a large amount of data, and is at least used to generate a personnel detection box for the personnel appearing in the picture. Each personnel detection box can identify the area corresponding to a personnel in the picture frame. Thus, based on the personnel detection boxes output by the first detection model, it can be further determined whether the personnel corresponding to each personnel detection box has an anomaly, so as to obtain the first detection result corresponding to each personnel one by one. If the first detection result obtained based on a personnel detection box has an anomaly, the first detection result will generate a corresponding anomaly item according to the specific anomaly and add it to the first detection result. Of course, if no personnel appears in the picture captured by the camera module, there will be no personnel detection box in the output of the first detection model, and there will be no first detection result in the current frame.

[0122] S204, assign a unique identifier corresponding to the personnel identity to each first detection result.

[0123] It can be understood that the unique identifier is usually a unique code or ID, which can be a number, a string or other forms of identifiers. The role of this identifier is to uniquely represent a specific person in the whole process. There is a one-to-one correspondence between the unique identifier and the personnel appearing in the platform, so as to facilitate determining the identity of the corresponding associated entity according to the unique identifier of the first detection result. Therefore, in the personnel anomaly recognition mechanism, the way to determine the associated entity is to extract the unique identifier from the first detection result and determine the personnel identity corresponding to the unique identifier according to the preset corresponding relationship to determine the associated entity.

[0124] S206, regard the first detection result containing anomaly items as a personnel anomaly recognition result.

[0125] It can be understood that the personnel anomaly recognition result reflects the anomaly situation of a certain personnel at the moment when the picture frame is captured. There may be multiple personnel in the picture frame, and each personnel has a corresponding first detection result. If an anomaly is recognized in step S202, the corresponding anomaly item will be included in the first detection result. Therefore, for each obtained first detection result, if the first detection result contains an anomaly item, then the first detection result corresponds to the abnormal behavior of a certain personnel, and the first detection result will become a personnel anomaly recognition result. Otherwise, it means that the personnel corresponding to the first detection result behaves normally at the moment when the picture frame is captured and does not need further processing.

[0126] In one of the embodiments, the personnel anomaly recognition may specifically include anomaly wearing recognition. Please refer to Figure 3, in abnormal wearing recognition, according to the output of the first detection model, the first detection result is obtained, including steps S302 to S306. In order to identify the wearing of personnel, the first detection model is also used to generate corresponding abnormal wearing detection frames for multiple abnormal wearing items respectively. That is, the abnormal wearing detection frame is a detection frame that marks the area in the entire human body area that is not worn as required. For example, Figure 4 As shown, the abnormal wearing area includes the area where the mask is not worn. When a person without a mask appears in the picture, the first detection model marks the mouth area of the person with an abnormal wearing detection frame. The first detection result includes the wearing detection result, and the wearing detection result reflects whether there is an abnormal wearing behavior of the person. Specifically, when the first detection model is trained, it will mark rectangular frames for the people and non-compliant wearing areas that appear in the training pictures. By learning these training pictures, the first detection model can master the ability to draw person detection frames and abnormal wearing detection frames.

[0127] S302, when the first detection model outputs at least one person detection frame and at least one abnormal wearing detection frame, match the abnormal wearing detection frame with the person detection frame.

[0128] It can be understood that sometimes there will be no people in the shooting picture of the camera module, then the first detection model will not output person detection frames and abnormal wearing detection frames, and there will be no first detection result. If there are people in the picture frame but all are wearing in compliance, only person detection frames will appear, and no abnormal wearing detection frames will appear. Then these person detection frames can be used as a wearing detection result respectively. If there are people in the picture frame and there are also problems with non-compliant wearing, at least one person detection frame and at least one abnormal wearing detection frame will appear at the same time. At this time, since each person detection frame corresponds to a person, in order to judge whether there is an abnormal wearing problem for this person in the picture frame, it is necessary to judge the association relationship between the abnormal wearing detection frame and the person detection frame by means of detection frame matching, that is, to judge which person detection frame each abnormal wearing detection frame is most relevant to.

[0129] Specifically, the person detection frame frames the entire human body area, while the abnormal wearing detection frame only frames a partial area of a person with unqualified wearing. For example, Figure 4As shown, therefore, the abnormal wearing detection frame of the same person should be located inside the person detection frame corresponding to that person. Based on this, the degree of association between the abnormal wearing detection frame and the person detection frame can be judged according to the coincidence ratio between the wearing detection frame and each person detection frame. That is, for any abnormal wearing detection frame, the first area corresponding to the abnormal wearing detection frame can be determined first. Then, the second area corresponding to the intersection area between the abnormal wearing detection frame and the person detection frame to be matched is calculated respectively. That is, the shapes, sizes, etc. of the intersection areas between the abnormal wearing detection frame and each person detection frame are determined respectively, and then the areas of these intersection areas (i.e., the second areas) are determined. Finally, by calculating the ratio between each second area and the first area, it can be known what share of the abnormal wearing detection frame these intersection areas account for. The larger this share is, the more parts of the abnormal wearing detection frame are located inside the person detection frame, and the greater the degree of association with the person detection frame. Therefore, after obtaining this ratio, a threshold can be set. If the ratio is greater than the corresponding threshold, it can be determined that the abnormal wearing detection frame matches the person detection frame, and this threshold can be set to 0.8 (80%).

[0130] S304, if there is a matching abnormal wearing detection frame for the person detection frame, then take the person detection frame and its matching abnormal wearing detection frame as a wearing detection result.

[0131] It can be understood that if there is a matching abnormal wearing detection frame for the person detection frame, then in the wearing detection result based on this person detection frame, all the abnormal wearing detection frames matched by this person detection frame will be added to this wearing detection result, and each abnormal wearing detection frame represents an abnormal item in the wearing of this person.

[0132] S306, if there is no matching abnormal wearing detection frame for the person detection frame, then directly take the person detection frame as a wearing detection result.

[0133] It can be understood that if there is no matching abnormal wearing detection frame for the person detection frame, it means that the person corresponding to this person detection frame has no abnormal wearing problem, and the person detection frame can be directly used as the wearing detection result of this person in the current frame.

[0134] In one embodiment, in the micro-nano processing platform, the abnormal wearing items can at least include two categories: clothing and work card. Clothing wearing involves the wearing requirements for clean clothes, gloves, protective glasses, masks, etc. Work card wearing requires that personnel must wear work cards. Since the work card is worn on the chest of the person, if the person turns their back to the camera, regardless of whether the person actually wears the work card or not, an abnormal wearing detection frame with the work card as the recognition object will be generated. To avoid misjudgment, it is necessary to distinguish the abnormal judgment logics for ordinary clothing wearing and work card wearing. Specifically, before taking the first detection result containing abnormal items as a person abnormal recognition result, please refer toFigure 5 , further including steps S502 to S506.

[0135] S502, if the abnormal wearing detection frame is included in the wearing detection result, then determine the type of the abnormal wearing detection frame.

[0136] It can be understood that the types here include clothing types and work card types. When the first detection model outputs an abnormal wearing detection frame, it will also output the recognition object of each abnormal wearing detection frame, that is, used to describe what abnormality this abnormal wearing detection frame is used to detect. For example, Figure 4 the recognition object of the abnormal wearing detection frame in [example] is a mask. For each abnormal wearing detection frame in the wearing detection result, classification needs to be carried out first, and the basis for classification is the recognition object of each abnormal wearing detection frame. Which recognition objects are included in each category are preset in advance. By comparing the recognition object of each abnormal wearing detection frame with these preset sets, the categories of each abnormal wearing detection frame can be determined.

[0137] S504, if the type of the abnormal wearing detection frame is a clothing type, then add the recognition object corresponding to the abnormal wearing detection frame to the abnormal item corresponding to the wearing detection result.

[0138] It can be understood that if the type of the abnormal wearing detection frame is a clothing type, then this type of abnormal wearing detection frame will not be misjudged because the person is facing or back to the camera module, and the recognition object corresponding to this abnormal wearing detection frame can be directly added as an abnormal item to the wearing detection result to which it belongs. For example, Figure 4 the abnormal item in the wearing detection result corresponding to the person in [example] includes not wearing a mask.

[0139] S506, if the type of the abnormal wearing detection frame is a work card type, then determine whether the person corresponding to the person detection frame is facing forward. If so, add not wearing a work card to the abnormal item corresponding to the wearing detection result.

[0140] It can be understood that if the type of the abnormal wearing detection frame is a work card type, then this type of abnormal wearing detection frame will be misjudged because the person is facing or back to the camera module, and the orientation of the current person needs to be determined first. If the person is facing forward and there is an abnormal wearing detection frame with the work card as the recognition object, then not wearing a work card can be added as an abnormal item to the wearing detection result to which this abnormal wearing detection frame belongs. If the person is facing back, even if there is an abnormal wearing detection frame with the work card as the recognition object, not wearing a work card will not be used as an abnormal item.

[0141] In one embodiment, determining whether the person corresponding to the person detection box is facing forward includes: when a face is included in the person detection box and the first ratio is greater than the first preset threshold, it is determined that the person corresponding to the person detection box is facing forward. It can be understood that in this embodiment, determining whether a person is facing forward includes two conditions. One is that a face is included, and the other is that the second ratio is greater than the first preset threshold. Only when these two conditions are met simultaneously can it be determined that the person is facing forward. Specifically, when a face is matched, it is considered that the person is basically facing forward. However, since the nameplate is worn on one side of the person's front chest, if the person slightly turns sideways, it may be difficult to determine whether the nameplate is worn, resulting in misjudgment. Therefore, the determination of the first ratio is further added to the judgment conditions. The first ratio is the ratio between the horizontal distance between the key points of the human shoulders and the width of the person detection box. As Figure 6 shown, the first ratio is W shoulder_x and W box The ratio between them. When the pedestrian is completely facing forward (backward), the ratio is the largest. When the pedestrian gradually turns to the side, the ratio gradually decreases. When completely sideways, the shoulders overlap, and at this time the ratio is 0. Based on the changing trend of the first ratio, the first preset threshold can be reasonably set according to experience to find the boundary value that can show the position where the work card is worn, such as 0.45. When it is greater than the first preset threshold, it is considered that the pedestrian is not standing sideways relative to the camera and the position where the work card is worn can be shown. Since the first detection model will output the coordinates of the four vertices of the person detection box when outputting the person detection box, the width of the person detection box can be calculated based on this coordinate. Regarding the method of obtaining the horizontal distance between the key points of the human shoulders, the picture frame needs to be input into the second detection model. The second detection model is used to detect the key points of the human body, which can mark the positions of the key points of the human body in the image or output the coordinates of each group of key points of the human body. The group of key points of the human body includes the coordinates of the positions of each key point of the human body. Based on the coordinates of the group of key points of the human body, the distance between the two shoulder key points can be calculated. There are many ways to divide the key points of the human body, such as 17-point key points of the human body, 26-point key points of the human body, etc. In this embodiment, only the shoulder key points need to be included among them.

[0142] In one embodiment, the abnormal person recognition may specifically include abnormal position recognition, and the first detection result includes a position detection result. In a micro-nano processing platform, in order to ensure safety and the smooth progress of processing, some areas are prohibited for personnel to enter. Therefore, it is necessary to track the position of personnel in each picture frame, and the position detection result in the first detection result reflects the position of the personnel in the picture frame. Please refer to Figure 7 , in the abnormal position recognition, according to the output of the first detection model, the first detection result is obtained, including steps S702 to S706.

[0143] S702, when the first detection model outputs at least one person detection box, input the picture frame into the second detection model to obtain at least one set of human key points.

[0144] It can be understood that the second detection model is used to detect human key points, which can mark the positions of human key points in the image or output the coordinates of the set of human key points. When the first detection model outputs at least one person detection box, it means that there are people in the current picture frame. Inputting this picture frame into the second detection model can obtain a set of human key point coordinates corresponding one by one to the person detection box. There are many ways to divide human key points, such as 17-point human key points, 26-point human key points, etc. In this embodiment, only the ankle key points need to be included among them.

[0145] S704, match the set of human key points with the person detection box.

[0146] It can be understood that after obtaining the set of human key points by using the second detection model, it is necessary to determine which person in the picture frame each set of human key points is used to label. Since each person has a corresponding person detection box, the set of human key points can be matched with the person detection box to judge the attribution of each set of human key points according to the association relationship between the set of human key points and the person detection box. Specifically, if the coordinates of each key point in the set of human key points are located within the person detection box, it can be determined that the set of human key points matches the person detection box, otherwise it does not match. Traverse all sets of human key points and person detection boxes in this way to complete the matching in step S704.

[0147] S706, take the person detection box and the position of the ankle key point in the set of human key points it matches as a position detection result.

[0148] It can be understood that in the traditional technology, the midpoint of the bottom edge of the person detection box is often used to represent the position of the person. When the degree of a person invading the restricted area is not high, such as only one foot entering the restricted area, it is easy to make a misjudgment. In this embodiment, the positions of the left and right ankle key points among the human key points will be selected to represent the position of the person. In the specific scenario of invading the restricted area, this method can better represent whether a person has stepped into the restricted area. Therefore, for the abnormal item in the position detection result, it is based on the position of the obtained ankle key point being within the preset electronic fence area. If any ankle key point in the set of human key points corresponding to a person detection box is within the preset electronic fence area, the position intrusion will be added as an abnormal item to the position detection result corresponding to this person detection box. The preset electronic fence area is a closed polygon that maps the actual environmental fence to the image coordinate system and can be obtained through coordinate transformation.

[0149] In one embodiment, the personnel in the micro-nano processing platform wear a combination of identification patterns composed of a preset number of different identification patterns. The identification patterns can be geometric figures or numbers, letters, Chinese characters, etc. The differences in the identification patterns can be in terms of shape, color, etc. For example Figure 8 As shown, the identification patterns in the figure are circles, squares, stars, and triangles. Each combination of identification patterns corresponds to a unique detection identifier. The unique detection identifier is the identifier obtained using the identification patterns, and there is a one-to-one correspondence between the combination of identification patterns and the unique detection identifier. The total number of combinations of identification patterns obtained by freely combining multiple different identification patterns is the same as the number of available unique detection identifiers. Assuming that the number of optional patterns for each identification pattern is n and the preset number is m, the total number of combinations of identification patterns is the mth power of n. For example Figure 8 As shown, there are 256 combination methods for four different identification patterns, which can correspond to 256 unique detection identifiers. If numbers are directly used as identification patterns and the preset number is 4, a total of 1000 unique detection identifiers can be corresponded. Please refer to Figure 9 , and assigning unique identifiers corresponding to the personnel identities to each first detection result includes steps S902 to S908.

[0150] S902, input the picture frame into the third detection model.

[0151] It can be understood that the third detection model is a detection model specially trained to detect identification patterns, and it is used to generate pattern detection frames for the identification patterns appearing in the picture. The combination of identification patterns can be worn on both the front and back of the personnel at the same time, which is convenient for the camera module to collect.

[0152] S904, in the case where the third detection model outputs multiple pattern detection frames, cluster the pattern detection frames, and determine the cluster with a preset number of pattern detection frames as the target cluster.

[0153] It can be understood that when a person appears in the picture frame, the combination of identification patterns worn by him will be captured by the camera module, and the third detection model can detect the identification patterns and generate corresponding pattern detection frames. When the third detection model outputs multiple pattern detection frames, it is necessary to determine which pattern detection frames come from the same combination of identification patterns. In this embodiment, considering that the identification patterns from the same combination of identification patterns are very close, the clustering method is used to process the obtained pattern detection frames. The obtained clusters need to be further screened, and only those with the same number of pattern detection frames in the cluster as the preset number will be screened out. Therefore, all clusters containing a preset number of pattern detection frames are determined as target clusters. There are many available clustering algorithms, such as the DBSCAN algorithm.

[0154] S906. Determine the central positions of the target clustering clusters, and match each target clustering cluster with a person detection box according to the central positions and the positions of the person detection boxes.

[0155] It can be understood that in order to assign a unique identifier to each person detection box, it is necessary to determine the combination of recognition patterns used by the person corresponding to each person detection box. And a target clustering cluster represents a combination of recognition patterns in the picture frame. The central position of the target clustering cluster and the center of the combination of recognition patterns. If the distance between the central position of the target clustering cluster and the center of a person detection box is closer, the probability that the combination of recognition patterns corresponding to the target clustering cluster is the one used by the person corresponding to the person detection box is greater. Therefore, by calculating the distances between the central positions of the target clustering clusters and each person detection box, a distance matrix can be formed. Using the Hungarian algorithm and this distance matrix, each target clustering cluster can be matched with a person detection box. The central position of the target clustering cluster is usually determined by calculating the geometric center of all the pattern detection boxes in the cluster, generally by averaging the coordinates of the four vertices of the pattern detection box.

[0156] S908. For any person detection box, if there is a matching target clustering cluster, determine a unique detection identifier according to the matching target clustering cluster, and use the unique detection identifier as the unique identifier.

[0157] It can be understood that if there is a matching target clustering cluster for the person detection box, the combination of recognition patterns corresponding to the target clustering cluster can be determined according to the recognition objects corresponding to the pattern detection boxes of the target clustering cluster, so that the unique detection identifier corresponding to the combination of recognition patterns can be obtained, and then this unique detection identifier can be used as the unique identifier corresponding to the person detection box.

[0158] In one of the embodiments, to assign a unique identifier corresponding to the person identity to each first detection result, please refer to Figure 10 , which includes steps S1002 to S1012.

[0159] S1002. Input the picture frame into the third detection model.

[0160] S1004. In the case where the third detection model outputs multiple pattern detection boxes, cluster the pattern detection boxes, and determine the clustering clusters with a preset number of pattern detection boxes as the target clustering clusters.

[0161] S1006. Determine the central positions of the target clustering clusters, and match each target clustering cluster with a person detection box according to the central positions and the positions of the person detection boxes.

[0162] For the descriptions of steps S1002 to S1006, reference can be made to the descriptions of steps S902 to S906.

[0163] S1008, Process the picture frames using a multi-object tracking algorithm to obtain the unique tracking identifier corresponding to each pedestrian detection box.

[0164] It can be understood that the multi-object tracking algorithm is a technology in the field of computer vision used to track multiple objects (pedestrians in this scenario) in consecutive video frames. The main purpose of this algorithm is to maintain consistent recognition of each object at different time points (i.e., different video frames). The identifier assigned by the multi-object tracking algorithm to each pedestrian that appears is called the unique tracking identifier. There are many mature solutions for the multi-object tracking algorithm that can be selected, such as using YOLOv8 combined with DeepSORT, etc. Using these mature solutions, a unique tracking identifier can be assigned to each pedestrian that appears in the video frames continuously captured by the camera module.

[0165] S1010, For any pedestrian detection box, if there is a matching target clustering cluster, determine the unique detection identifier according to the matching target clustering cluster, and use the unique detection identifier as the unique identifier, and bind the unique detection identifier with the unique tracking identifier.

[0166] S1012, If there is no matching target clustering cluster, obtain the unique identifier according to the unique detection identifier bound to the unique tracking identifier of the pedestrian detection box.

[0167] It can be understood that during the continuous shooting process of the camera module, due to the small size of the signs when the distance from the camera is far, and the easy occlusion of pedestrians, etc., it is not easy to correctly detect each recognition pattern. However, the unique tracking identifier obtained by using the multi-object tracking algorithm does not have this problem. Therefore, the unique tracking identifier can be used as a supplement when the unique detection identifier cannot be obtained. Specifically, for any person detection box, if there is a matching target clustering cluster, it means that its unique detection identifier can be obtained, and the unique detection identifier is directly used as the unique identifier of the person detection box. At the same time, a binding relationship needs to be established between the unique detection identifier and the unique tracking identifier of the person detection box for use when the unique detection identifier cannot be obtained. If there is no matching target clustering cluster, it means that its unique detection identifier cannot be obtained. It is necessary to determine the unique detection identifier bound to it from the binding relationships determined by the unique tracking identifier in the past few frames according to the unique tracking identifier of the person detection box, and use the unique detection identifier as the unique identifier. For example, the unique detection identifier obtained for the person detection box corresponding to person A in the x-th frame is 123, and the unique tracking identifier is 008, then the binding {008: 123} can be established. In the (x + 1)-th frame, the unique detection identifier cannot be obtained, but the unique tracking identifier 008 can be obtained. At this time, the unique detection identifier of the person detection box can be determined to be 123 according to the binding relationship obtained in the previous frame, and then the unique identifier is determined to be 123. Of course, if the unique detection identifiers bound in the recent N (a preset positive integer) frames are different, the unique detection identifier can be obtained according to the mode of the unique detection identifiers among them.

[0168] In one embodiment, a sound generating unit is provided on the camera module to give a reminder to the associated subject, including: taking the camera module that reports the first detection result including the abnormal item in the camera module as the first target camera module, and instructing the first target camera module to give a voice broadcast according to the abnormal item and the person identity corresponding to the unique identifier.

[0169] It can be understood that for the person abnormality recognition mechanism, the notification method it can adopt is to use the sound generating unit on the camera module for notification. Specifically, for multiple camera modules existing in the platform, when any one reports the first detection result including the abnormal item, it means that the camera module has detected a person abnormality, including wearing abnormality, position abnormality, etc. At this time, this camera module can be determined as the first target camera module. The associated subject triggering the abnormality is within the shooting range of the first target camera module at this time, and the sound generated by the first target camera module can be heard by this associated subject. Therefore, instructing the first target camera module to generate a sound can give a reminder to the associated subject. The specific content of the reminder can be issued according to the abnormal item and the person identity corresponding to the unique identifier. Including the person identity in the reminder content can remind the people near the first target camera module who triggered the abnormality. Including the abnormal item in the reminder content can remind the person what abnormalities have been triggered. For exampleFigure 4 In the situation shown, assuming that the unique identifier of the person is 123, information such as the name of the person corresponding to the unique identifier 123 (such as Cxx) can be found in the database. The content of the voice broadcast can be: Cxx (personnel identity), please pay attention to wearing a mask (abnormal item). In this type of mechanism, the method of prompting the associated entity also includes sending the abnormal item to the associated entity by means of email, text message, social media message, etc.

[0170] In one embodiment, in the requirements of platform 6s management and lean management, requirements are put forward for the placement of tools on the experimental bench desktop, etc. The desktop anomaly recognition mechanism in the anomaly recognition mechanism is used to identify whether there is an anomaly in the placement of tools on the experimental bench desktop. The desktop anomaly recognition result in the anomaly recognition result reflects the anomaly in the placement of desktop tools. The camera module corresponding to the desktop anomaly recognition mechanism is used to photograph the experimental bench desktop. Use the desktop anomaly recognition mechanism to perform anomaly recognition on the monitoring area, please refer to Figure 11 , including step S1102 to step S1106.

[0171] S1102, when the desktop inspection condition is met, input the picture frame captured by the camera module into the fourth detection model to obtain multiple desktop object detection frames.

[0172] It can be understood that the desktop inspection condition refers to the condition for triggering the camera module to start inspecting the placement situation of the experimental bench desktop. In some embodiments, the experimental bench needs to be reserved for use. During the reserved usage period, since the tools on the experimental bench need to be used, there is no need for inspection. The desktop inspection can be triggered after the reserved time of the experimental bench corresponding to the camera module ends. In some embodiments, it can also be triggered when a certain period of time has passed after the person leaves the shooting range of the camera module. The inspection of the experimental bench desktop is also based on computer vision. Specifically, the picture frame captured by the camera module needs to be input into the fourth detection model. The fourth detection model is used to generate desktop object detection frames for various desktop objects that appear in the picture. That is, during the training process, the objects that often appear on the experimental bench desktop will be used as training images, and the desktop objects that appear in the training pictures will be boxed and labeled, so that the fourth detection model gradually learns to use various common desktop objects as the recognition objects and generate corresponding desktop object detection frames.

[0173] S1104, match the images within each desktop object detection frame with the reference image library.

[0174] It can be understood that the reference image library is a pre-established database that stores the reference images of each desktop tool, and the reference images reflect the standard form of the desktop tools. After obtaining the desktop object detection frames, screenshots are taken with each desktop object detection frame as the boundary, and the obtained screenshots are compared with each reference image to obtain the similarity between the images within each desktop object detection frame and each reference image. Based on whether the similarity is greater than the third set threshold, it is determined whether there is a matching reference image for each desktop detection frame.

[0175] S1106. If there is a desktop object detection frame that cannot be matched, the information of the experimental bench corresponding to the camera module and the information of the person who recently reserved the experimental bench are used as the desktop anomaly recognition result.

[0176] It can be understood that if there is a desktop object detection frame that cannot be matched with the reference image among the recognized desktop object detection frames, it means that the desktop object corresponding to the detection frame is not a desktop tool, that is, it is detected that there are sundries on the desktop of the experimental bench, which does not meet the requirements of platform management and belongs to desktop anomalies, and a desktop anomaly recognition result needs to be generated. The desktop anomaly recognition result needs to include the information of the experimental bench and the relevant information of the associated entity. Specifically, the camera module configured with the desktop anomaly recognition mechanism is used to photograph the desktop of the experimental bench, so it has a corresponding experimental bench, and the information of its corresponding experimental bench can be stored in advance and directly extracted for use when reporting. In addition, after the camera module recognizes an anomaly, it can find the information of the person who recently reserved the experimental bench from the historical records of the reservation platform according to the information of the experimental bench. These people are the associated entities causing the desktop anomaly. Their information is also added to the desktop anomaly recognition result. Therefore, in the desktop anomaly recognition mechanism, the way to determine the associated entity is to extract the personnel information of the person who reserved the relevant experimental bench from the desktop anomaly recognition result to determine the associated entity.

[0177] In one embodiment, a sound-emitting unit is provided on the camera module to give a reminder to the associated entity. Please refer to Figure 12 , including steps S1202 to S1210.

[0178] S1202. Select the camera module that reports the desktop anomaly recognition result as the starting camera module.

[0179] S1204. According to the time when the desktop anomaly recognition result is reported and the reservation end time of the experimental bench corresponding to the starting camera module, obtain the departure time difference.

[0180] S1206. According to the departure time difference and the preset walking speed, obtain the departure distance.

[0181] S1208. Determine the second target camera module as the camera module whose distance from the starting camera module is within the departure distance.

[0182] S1210, control the second target camera module to issue a voice broadcast according to the test bench information and personnel information.

[0183] It can be understood that the person who triggers the desktop anomaly leaves the test bench after the experiment ends, but it is difficult to predict the direction and current location of their departure. The camera module that reports the desktop anomaly recognition result is the camera module that initially detected the desktop anomaly, which will serve as the starting point for the person who triggers the desktop anomaly to leave. It is also possible to determine how long the person with the desktop anomaly has been away based on the time when the anomaly was recognized by this camera module and the scheduled end time of the corresponding test bench. Then, multiply the time difference of departure by the preset walking speed. Based on this, it is possible to predict how far the person who triggered the desktop anomaly has left, that is, obtain the departure distance. Taking the starting-point camera module as the center and the departure distance as the radius, all the camera modules included therein are determined as the second target camera modules. It is very likely that the person who triggers the desktop anomaly is within the broadcast coverage range of the second target camera modules. Therefore, instruct the second target camera modules to conduct a voice broadcast. The broadcast content needs to include the personnel information, which can remind who triggered the desktop anomaly. The broadcast content also needs to include the test bench information, which can remind which test bench needs to be tidied up. For example, the personnel information retrieved includes a name (such as Cxx), and the test bench number is 006. The content of the voice broadcast can be: Cxx (personnel information), please pay attention to cleaning up test bench 006 (test bench information). In this type of mechanism, the prompting methods for associated entities also include sending the test bench information to the associated entities by means of emails, text messages, social media messages, etc.

[0184] In one of the embodiments, in the requirements of platform 6s management and lean management, requirements are put forward for the cleanliness of the test bench passageways. The passageway clutter anomaly recognition mechanism in the anomaly recognition mechanism is used to identify whether there is clutter in the platform passageways. The passageway clutter anomaly recognition result in the anomaly recognition result reflects the anomaly of the existence of clutter in the passageway. The camera module corresponding to the passageway clutter anomaly recognition mechanism is used to photograph the platform passageways. Use the passageway clutter anomaly recognition mechanism to conduct anomaly recognition on the monitoring area. Please refer to Figure 13 , including steps S1302 to S1308.

[0185] S1302, respectively use the picture frames captured by the camera module to update the long-term background model and the short-term background model. Both the long-term background model and the short-term background model are used to extract the background images in the pictures, and the learning rate of the long-term background model is lower than that of the short-term background model.

[0186] It can be understood that the background model can learn from images of the same location to separate the background part and the foreground part in the images. Therefore, the background model can process the images to obtain background images that reflect the background of the captured scene. Due to the characteristics of the background model algorithm itself, a moving foreground will gradually blend into the background after staying stationary for a period of time, and the speed of blending into the background can be adjusted through the learning rate. The long-term background model has a lower learning rate and is not sensitive to short-term changes. While the short-term background model has a higher learning rate and can quickly adapt to scene changes and capture the recent background state.

[0187] S1304, input the picture frame into the long-term background model to obtain the first image, and input the picture frame into the short-term background model to obtain the second image.

[0188] It can be understood that in this step, the currently captured picture frame is respectively input into the long-term background model and the short-term background model to obtain the first image and the second image. As Figure 14 shown, the first row in the second column in the figure represents the first image, and the second row in the second column represents the second image. The black part in the first image and the second image represents the background, and the white part represents the foreground. The cardboard box in the original picture frame has been stationary for a certain time. Due to the low learning rate of the long-term background model, the cardboard box has not yet blended into the background and remains as the foreground part in the first image. While in the short-term background model, the cardboard box has basically blended into the background and becomes the background part in the second image.

[0189] S1306, compare the first image and the second image, and determine whether the difference region in the obtained difference image is a clutter region.

[0190] It can be understood that based on the characteristics of the first image and the second image, objects placed in the channel for a long time have become the background and will not cause differences in the image. However, for objects that appear later, they will blend into the background at different rates. The difference between the first image and the second image reflects the existence of these recently appeared objects. Therefore, by comparing the first image with the second image and highlighting the different parts, a difference image can be obtained. As Figure 14 visible in the third column, compare the first image and the second image, and set the same regions to black and the different regions to white to obtain the difference image. The white part in the difference image is the object that recently appeared in the channel. Since the recently appeared objects may be passing pedestrians or other non-clutter, for the objects in these difference regions, it is necessary to further determine whether they are really clutter regions.

[0191] Furthermore, before determining whether it is a clutter region, the difference image can be post-processed to remove the noise in the difference image. The means that can be used for post-processing include morphological opening operation, morphological closing operation, width-height-area filtering, etc., to obtain Figure 14The effect in the fourth column.

[0192] S1308. Obtain the abnormal recognition result of the passage sundries according to the determined sundries area.

[0193] It can be understood that for the picture frame where there is a determined sundries area, a sundries annotation box can be generated in the picture frame according to the sundries area to obtain the abnormal recognition result of the passage sundries.

[0194] In one embodiment, to determine whether the difference area is a sundries area, please refer to Figure 15 , including steps S1502 to S1506.

[0195] S1502. Input the difference image into the fifth detection model.

[0196] It can be understood that the target filtered image refers to the image of non-sundries objects that often appear in the platform. During the training process of the fifth detection model, by annotating the non-sundries objects that appear in the platform passage, the fifth detection model gradually learns to mark the non-sundries objects in it with filtered detection boxes, so that the fifth detection model can be used to generate filtered detection boxes for the target filtered image in the image.

[0197] S1504. If there is a corresponding filtered detection box for each difference area in the difference image, it is determined that there is no sundries area in the difference image.

[0198] S1506. Otherwise, determine the difference area without a corresponding filtered detection box as the sundries area.

[0199] It can be understood that the filtered detection box can identify the non-sundries area. After inputting the difference image into the fifth detection model, if a difference area is framed by the filtered detection box, it means that the object corresponding to the difference area is non-sundries. Therefore, if each difference area in the difference image is marked as non-sundries, it can be determined that there is no sundries area in the difference image. Otherwise, as long as there is one difference area not marked as non-sundries, it will be determined as the sundries area, generate a circumscribed rectangle for it, and mark the sundries area in the original difference image.

[0200] In one embodiment, for the abnormal recognition mechanism of the passage sundries, according to the abnormal recognition result, determine the associated entity, please refer to Figure 16 , including steps S1602 to S1606.

[0201] S1602. Determine the target passage area as the passage area corresponding to the camera module that reports the abnormal recognition result of the passage sundries.

[0202] S1604. Determine the corresponding cleaning person in charge according to the target passage area.

[0203] S1606, determine the cleaning person in charge corresponding to the target channel area as the associated entity.

[0204] It can be understood that a camera module configured with a channel debris abnormality recognition mechanism will have a channel area that it is responsible for monitoring. When any such camera module reports a channel debris abnormality recognition result, it can be determined that there is debris in the channel area it is responsible for, and this channel area can be determined as the target channel area. Each channel area has a corresponding cleaning person in charge, and the cleaning person in charge should do a good job in the cleaning work of the channel area they are responsible for and clean up the debris in a timely manner. Therefore, according to the corresponding relationship between each channel area and the cleaning person in charge, the cleaning person in charge corresponding to the target channel area can be found as the associated entity. In this type of mechanism, the prompting method for the associated entity also includes sending the location information of the target channel area to the associated entity by means of emails, text messages, social media messages, etc.

[0205] In one embodiment, a sounding unit and / or a vibration unit are arranged on the work card worn by the personnel in the micro-nano processing platform to give a reminder to the associated entity, including: selecting the work card corresponding to the associated entity as the target work card, and instructing the target work card to give a voice broadcast according to the abnormality item, and / or give a vibration.

[0206] It can be understood that for personnel abnormality recognition mechanisms, channel debris abnormality recognition mechanisms, desktop abnormality recognition mechanisms, etc., as long as the associated entity can be determined, communication can be carried out with the work card of the associated entity, and the work card method can be used for notification. A sounding unit and / or a vibration unit can be arranged on the work card. When a sounding unit is provided, a voice broadcast can be given according to the reported abnormality item. When a vibration unit is provided, the associated entity can be reminded of the triggered abnormality through vibration and needs to be rectified.

[0207] In one embodiment, the management method further includes: in the case where any camera module obtains an abnormality recognition result, taking the picture frame corresponding to the obtained abnormality recognition result as the midpoint, obtaining a plurality of picture frames on its front and back sides to form a recorded video stream, and the recorded video stream will be reported together when reporting the abnormality recognition result.

[0208] It can be understood that the recorded video stream can be used as a basis for manually determining whether there is a misjudgment. By consulting the recorded video stream, the process of the associated entity triggering the abnormality can be reviewed to cover for misjudgment situations. The associated entity can send an appeal message to the management server, and the management server can feedback the recorded video stream corresponding to the abnormality triggered by it to the associated entity. After the associated entity watches it and understands the specific situation of triggering the abnormality, then confirm whether to continue the appeal.

[0209] The present application provides a management device for a micro-nano processing platform. The micro-nano processing platform is provided with a plurality of camera modules. The management device includes a result acquisition module, a recording module, a processing level adjustment module, and a processing module. The result acquisition module uses at least one anomaly recognition mechanism configured for each camera module to respectively perform anomaly recognition on the monitoring area corresponding to each camera module, and obtains an anomaly recognition result. The recording module is used to determine an associated entity according to the anomaly recognition result and update the anomaly record of the associated entity. The processing level adjustment module is used to upgrade the processing level of the associated entity if the anomaly record meets the upgrade condition; otherwise, maintain the processing level of the associated entity. The processing module uses the processing means corresponding to the current processing level to process the associated entity.

[0210] In one embodiment, the processing levels include a first processing level and a second processing level from low to high. The processing module is used to send a reminder to the associated entity if the processing level is the first processing level; if the processing level is the second processing level, report the anomaly record to the administrator.

[0211] In one embodiment, the anomaly record includes the number of anomalies occurring within the current statistical period. The upgrade condition includes: if the number is greater than a preset threshold, upgrade from the first processing level to the second processing level.

[0212] In one embodiment, the anomaly recognition mechanism includes a personnel anomaly recognition mechanism, and the anomaly recognition result includes a personnel anomaly recognition result. Using the personnel anomaly recognition mechanism to perform anomaly recognition on the monitoring area includes: inputting the picture frames captured by the camera module into a first detection model, and obtaining a first detection result according to the output of the first detection model; the first detection model is used to generate a personnel detection frame for the personnel appearing in the picture; assigning a unique identifier corresponding to the personnel identity to each first detection result; taking the first detection result containing an anomaly item as a personnel anomaly recognition result.

[0213] In one embodiment, the first detection model is further used to respectively generate corresponding anomaly wearing detection frames for a plurality of anomaly wearing items, and the first detection result includes a wearing detection result; obtaining the first detection result according to the output of the first detection model includes: when the first detection model outputs at least one personnel detection frame and at least one anomaly wearing detection frame, matching the anomaly wearing detection frame with the personnel detection frame; if there is a matching anomaly wearing detection frame for the personnel detection frame, taking the personnel detection frame and its matching anomaly wearing detection frame as a wearing detection result; if there is no matching anomaly wearing detection frame for the personnel detection frame, directly taking the personnel detection frame as a wearing detection result.

[0214] In one embodiment, before taking the first detection result including abnormal items as a personnel anomaly recognition result, it further includes: if the wearable detection result includes an abnormal wearable detection box, determining the type of the abnormal wearable detection box; if the type of the abnormal wearable detection box is a clothing type, adding the recognition object corresponding to the abnormal wearable detection box to the abnormal item corresponding to the wearable detection result; if the type of the abnormal wearable detection box is a work permit type, determining whether the person corresponding to the personnel detection box is facing forward, and if so, adding not wearing a work permit to the abnormal item corresponding to the wearable detection result.

[0215] In one embodiment, determining whether the person corresponding to the personnel detection box is facing forward includes: determining that the person corresponding to the personnel detection box is facing forward when a face is included in the personnel detection box and a first ratio is greater than a first preset threshold; the first ratio is the ratio between the horizontal distance between the human shoulder key points in the personnel detection box and the width of the personnel detection box.

[0216] In one embodiment, if the type of the abnormal wearable detection box is a clothing type, the recognition object corresponding to the abnormal wearable detection box includes at least one of not wearing a clean suit, not wearing gloves, not wearing safety glasses, and not wearing a mask.

[0217] In one embodiment, matching the abnormal wearable detection box with the personnel detection box includes: for any abnormal wearable detection box, determining a first area corresponding to the abnormal wearable detection box; respectively calculating a second area corresponding to the intersection area between the abnormal wearable detection box and the personnel detection box to be matched; if the ratio between the second area and the first area is greater than a second preset threshold, determining that the abnormal wearable detection box is matched with the personnel detection box.

[0218] In one embodiment, the first detection result includes a position detection result; obtaining the first detection result according to the output of the first detection model includes: when the first detection model outputs at least one personnel detection box, inputting the picture frame into the second detection model to obtain at least one group of human key points; matching the group of human key points with the personnel detection box; using the ankle key point positions in the personnel detection box and its matched group of human key points as a position detection result.

[0219] In one embodiment, before taking the first detection result including abnormal items as an anomaly recognition result, it further includes: if the ankle key point position is within a preset electronic fence area, adding position intrusion to the abnormal item corresponding to the position detection result.

[0220] In one embodiment, the personnel in the micro-nano processing platform wear an identification pattern combination composed of a preset number of different identification patterns. The identification pattern combination corresponds to a unique detection identifier. Assigning a unique identifier corresponding to the personnel identity to each first detection result includes: inputting a picture frame into a third detection model; the third detection model is used to generate pattern detection frames for the identification patterns appearing in the picture; in the case where the third detection model outputs multiple pattern detection frames, clustering the pattern detection frames, and determining the clustering cluster with a preset number of pattern detection frames as the target clustering cluster; determining the central positions of the target clustering clusters, and matching the target clustering clusters with the personnel detection frames according to the positions of the central positions and the personnel detection frames; for any personnel detection frame, if there is a matching target clustering cluster, determining the unique detection identifier according to the matching target clustering cluster, and using the unique detection identifier as the unique identifier.

[0221] In one embodiment, assigning a unique identifier corresponding to the personnel identity to each first detection result further includes: processing the picture frame using a multi-target tracking algorithm to obtain a unique tracking identifier corresponding to each pedestrian detection frame. For any personnel detection frame, if there is a matching target clustering cluster, determining the unique detection identifier according to the matching target clustering cluster, and using the unique detection identifier as the unique identifier, further includes: for any personnel detection frame, if there is a matching target clustering cluster, determining the unique detection identifier according to the matching target clustering cluster, and using the unique detection identifier as the unique identifier, and binding the unique detection identifier with the unique tracking identifier; if there is no matching target clustering cluster, obtaining the unique identifier according to the unique detection identifier bound to the unique tracking identifier of the personnel detection frame.

[0222] In one embodiment, matching the target clustering clusters with the personnel detection frames includes: using the Hungarian algorithm to match the target clustering clusters with the personnel detection frames.

[0223] In one embodiment, a sound generating unit is provided on the camera module. The processing module uses the camera module that reports the first detection result including the abnormal item as the first target camera module, and instructs the first target camera module to issue a voice broadcast according to the abnormal item and the personnel identity corresponding to the unique identifier.

[0224] In one embodiment, a sound generating unit and / or a vibration unit are provided on the work card worn by the personnel in the micro-nano processing platform. The processing module is used to select the work card corresponding to the associated subject as the target work card, and instruct the target work card to issue a voice broadcast according to the abnormal item, and / or generate a vibration.

[0225] In one embodiment, the anomaly recognition mechanism includes a desktop anomaly recognition mechanism, and the anomaly recognition result includes a desktop anomaly recognition result. The camera module corresponding to the desktop anomaly recognition mechanism is used to capture the desktop of the experimental bench. The desktop anomaly recognition mechanism is used to recognize anomalies in the monitoring area, including: when the desktop inspection conditions are met, inputting the picture frames captured by the camera module into a fourth detection model to obtain multiple desktop object detection frames; the fourth detection model is used to generate desktop object detection frames for various desktop objects appearing in the picture; matching the images within each desktop object detection frame with a reference image library; the reference image library stores the reference images of each desktop tool; if there is a desktop object detection frame that cannot be matched, the information of the experimental bench corresponding to the camera module and the information of the person who reserved the experimental bench most recently are used as the desktop anomaly recognition result.

[0226] In one embodiment, a sound - emitting unit is provided on the camera module. The processing module is used to select the camera module that reports the desktop anomaly recognition result as the starting camera module; according to the time when the desktop anomaly recognition result is reported and the reservation end time of the experimental bench corresponding to the starting camera module, obtain the leaving time difference; according to the leaving time difference and the preset walking speed, obtain the leaving distance; determine the camera modules whose distances from the starting camera module are within the leaving distance as the second target camera modules; control the second target camera modules to issue voice broadcasts according to the experimental bench information and the person information.

[0227] In one embodiment, the desktop detection condition includes that the time reaches after the reservation time of the experimental bench corresponding to the camera module ends.

[0228] In one embodiment, the anomaly recognition mechanism includes a passage clutter anomaly recognition mechanism, and the anomaly recognition result includes a passage clutter anomaly recognition result. The camera module corresponding to the passage clutter anomaly recognition mechanism is used to capture the platform passage. The passage clutter anomaly recognition mechanism is used to recognize anomalies in the monitoring area, including: using the picture frames captured by the camera module to update the long - term background model and the short - term background model respectively; both the long - term background model and the short - term background model are used to extract the background image in the picture, and the learning rate of the long - term background model is lower than that of the short - term background model; inputting the picture frame into the long - term background model to obtain a first image, and inputting the picture frame into the short - term background model to obtain a second image; comparing the first image and the second image to determine whether the difference area in the obtained difference image is a clutter area; obtaining the passage clutter anomaly recognition result according to the determined clutter area.

[0229] In one embodiment, determining whether the difference region is a sundry region includes: inputting the difference image into a fifth detection model; the fifth detection model is used to generate a filtering detection frame for the target filtering image in the image; if there is a corresponding filtering detection frame for each difference region in the difference image, it is determined that there is no sundry region in the difference image; otherwise, the difference region without a corresponding filtering detection frame is determined as the sundry region.

[0230] In one embodiment, the recording module is used to determine the target channel region as the channel region corresponding to the camera module that reports the sundry abnormality recognition result of the reporting channel; according to the target channel region, determine the corresponding cleaning person in charge; and determine the cleaning person in charge corresponding to the target channel region as the associated entity.

[0231] For the specific limitations of the management device of the micro-nano processing platform, reference can be made to the limitations of the management method of the micro-nano processing platform in the above text, which will not be elaborated here. Each module in the above management device of the micro-nano processing platform can be implemented in whole or in part by software, hardware and their combination. The above modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules. It should be noted that the division of modules in the embodiments of the present application is illustrative, only a logical function division, and there may be other division methods in actual implementation.

[0232] The present application provides a computer device, including one or more processors and a memory. Computer-readable instructions are stored in the memory. When the computer-readable instructions are executed by one or more processors, the following operations are performed: using at least one abnormality recognition mechanism configured for each camera module, respectively performing abnormality recognition on the monitoring region corresponding to each camera module to obtain an abnormality recognition result; according to the abnormality recognition result, determining an associated entity and updating the abnormality record of the associated entity; if the abnormality record meets the upgrade condition, raising the processing level for the associated entity, otherwise, maintaining the processing level for the associated entity; and using the processing means corresponding to the current processing level to process the associated entity.

[0233] In one embodiment, when the computer-readable instructions are executed by one or more processors, the management method of the micro-nano processing platform described in any of the above embodiments is executed.

[0234] Schematically, as Figure 17 shown, Figure 17 is a schematic internal structure diagram of a computer device provided by an embodiment of the present application. Referring to Figure 17, the computer device 1700 includes a processing component 1702, which further includes one or more processors, and memory resources represented by a memory 1701 for storing instructions executable by the processing component 1702, such as application programs. The application programs stored in the memory 1701 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1702 is configured to execute instructions to perform the steps of the management method of the micro-nano processing platform in any of the above embodiments.

[0235] The computer device 1700 may further include a power supply component 1703 configured to perform power management of the computer device 1700, a wired or wireless model interface 1704 configured to connect the computer device 1700 to a model, and an input / output (I / O) interface 1705.

[0236] This application provides a storage medium storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform: determining an associated entity according to an anomaly recognition result and updating the anomaly record of the associated entity; if the anomaly record meets the upgrade condition, upgrading the processing level of the associated entity, otherwise, maintaining the processing level of the associated entity; and processing the associated entity using the processing means corresponding to the current processing level.

[0237] In one of the embodiments, when the computer-readable instructions are executed by one or more processors, the one or more processors are caused to perform the management method of the micro-nano processing platform described in any of the above embodiments.

[0238] Finally, it should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0239] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0240] The foregoing description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A management method for a micro-nano processing platform, characterized in that: The micro-nano processing platform is provided with a plurality of camera modules, and personnel in the micro-nano processing platform wear an identification pattern combination consisting of a preset number of different identification patterns, and the identification pattern combination corresponds to a unique detection identifier, and the management method includes: Using at least one abnormality recognition mechanism configured by each camera module, abnormality recognition is performed on the monitoring area corresponding to each camera module to obtain abnormality recognition results, wherein the abnormality recognition results include personnel abnormality recognition results; specifically comprising: inputting the picture frame taken by the camera module into a first detection model, and obtaining a first detection result according to the output of the first detection model; the first detection model is used to generate a personnel detection frame for the person appearing in the picture; a unique identifier corresponding to the identity of the person is assigned to each of the first detection results; and the first detection result containing the abnormal item is used as a personnel abnormality recognition result; The assigning of a unique identifier corresponding to a person's identity to each of the first detection results includes: inputting the picture frame into a third detection model; the third detection model is used to generate a pattern detection frame for the recognition pattern appearing in the picture; when the third detection model outputs a plurality of the pattern detection frames, clustering the pattern detection frames, and determining a clustering cluster having the preset number of the pattern detection frames as a target clustering cluster; determining a center position of each of the target clustering clusters, and matching each of the target clustering clusters with the person detection frame according to the center position and the position of each of the person detection frames; processing the picture frame using a multi-target tracking algorithm to obtain a unique tracking identifier corresponding to each of the person detection frames; for any of the person detection frames, if there is a matching target clustering cluster, determining the unique detection identifier according to the matching target clustering cluster, taking the unique detection identifier as the unique identifier, and binding the unique detection identifier to the unique tracking identifier; if there is no matching target clustering cluster, obtaining the unique identifier according to the unique detection identifier bound to the unique tracking identifier of the person detection frame; Determine the associated subject according to the abnormality identification result, and update the abnormality record of the associated subject; If the abnormal record meets the upgrade conditions, the processing level of the associated subject is upgraded; otherwise, the processing level of the associated subject is maintained; the processing level reflects the degree of management personnel's participation and the strength of the processing measures; The associated subject is processed using the processing means corresponding to the current processing level.

2. The management method according to claim 1, characterized in that: The processing level includes a first processing level and a second processing level from low to high, and the processing means corresponding to the current processing level is used to process the associated subject, including: If the processing level is the first processing level, a reminder is issued to the associated subject; If the processing level is the second processing level, the abnormal record is reported to an administrator.

3. The management method according to claim 2, characterized in that: The abnormal record includes the number of abnormalities that occurred in the current statistical period, and the upgrade conditions include: If the number is greater than a preset threshold, the first processing level is upgraded to the second processing level.

4. The management method according to claim 1, characterized in that: The first detection model is further used to generate corresponding abnormal wear detection frames for the multiple abnormal wear items respectively, and the first detection result includes a wear detection result; the first detection result is obtained according to the output of the first detection model, including: In a case where the first detection model outputs at least one of the person detection frames and at least one of the abnormal wear detection frames, matching the abnormal wear detection frames with the person detection frames; If the person detection frame has a matching abnormal wear detection frame, the person detection frame and the matching abnormal wear detection frame are taken as one wear detection result; If the person detection frame does not have a matching abnormal wear detection frame, the person detection frame is directly used as a wear detection result.

5. The management method according to claim 4, characterized in that: Before taking the first detection result containing the abnormal item as a result of identifying abnormal personnel, the method further includes: If the wear detection result includes the abnormal wear detection frame, determining the type of the abnormal wear detection frame; If the type of the abnormal wear detection frame is clothing, adding the identification object corresponding to the abnormal wear detection frame to the abnormal item corresponding to the wear detection result; If the type of the abnormal wear detection frame is the work badge type, determine whether the person corresponding to the person detection frame is facing forward. If so, add the person not wearing the work badge to the wear detection result corresponding to the abnormal item.

6. The management method according to claim 5, characterized in that: The determining whether the person corresponding to the person detection frame is facing forward includes: When a face is included in the person detection frame and a first ratio is greater than a first preset threshold, it is determined that the person corresponding to the person detection frame is front-facing; the first ratio is the ratio between the horizontal distance between the key points of the human shoulders in the person detection frame and the width of the person detection frame.

7. The management method according to claim 5, characterized in that: If the type of the abnormal wear detection frame is clothing, the identification object corresponding to the abnormal wear detection frame includes at least one of not wearing clean clothes, not wearing gloves, not wearing protective glasses, and not wearing a mask.

8. The management method according to claim 4, characterized in that: The matching of the abnormal wear detection frame with the person detection frame includes: For any of the abnormal wear detection frames, determining a first area corresponding to the abnormal wear detection frame; Respectively calculating the second area corresponding to the intersection area of ​​the abnormal wear detection frame and the person detection frame to be matched; If the ratio between the second area and the first area is greater than a second preset threshold, it is determined that the abnormal wear detection frame matches the person detection frame.

9. The management method according to claim 1, characterized in that: The first detection result includes a position detection result; the first detection result is obtained according to the output of the first detection model, including: In the case where the first detection model outputs at least one of the person detection frames, the picture frame is input into a second detection model to obtain at least one human key point group; Matching the human body key point group with the person detection frame; The person detection frame and the ankle key point position in the matched human body key point group are taken as a position detection result.

10. The management method according to claim 9, characterized in that: Before taking the first detection result containing the abnormal item as a result of identifying abnormal personnel, the method further includes: If the ankle key point position is located within the preset electronic fence area, the position intrusion is added to the abnormal item corresponding to the position detection result.

11. The management method according to claim 1, characterized in that: The matching of each of the target clusters with the person detection frame includes: The Hungarian algorithm is used to match each of the target clusters with the person detection frame.

12. The management method according to claim 1, characterized in that: The camera module is provided with a sound unit, and the reminding the associated subject includes: The camera module that reports the first detection result including the abnormal item is used as the first target camera module, and the first target camera module is instructed to issue a voice broadcast according to the abnormal item and the identity of the person corresponding to the unique identifier.

13. The management method according to claim 1, characterized in that: A sound unit and / or a vibration unit is provided on the work badge worn by the personnel in the micro-nano processing platform, and the reminder to the associated subject includes: The work badge corresponding to the associated entity is selected as the target work badge, and the target work badge is instructed to issue a voice broadcast and / or vibrate according to the abnormal item.

14. The management method according to claim 2, characterized in that: The abnormality recognition mechanism includes a desktop abnormality recognition mechanism, the abnormality recognition result includes a desktop abnormality recognition result, the camera module corresponding to the desktop abnormality recognition mechanism is used to shoot the desktop of the experimental table, and the desktop abnormality recognition mechanism is used to perform abnormality recognition on the monitoring area, including: When the desktop inspection condition is met, the picture frame captured by the camera module is input into the fourth detection model to obtain multiple desktop object detection frames; the fourth detection model is used to generate the desktop object detection frames for multiple desktop objects appearing in the picture; Matching the image in each of the desktop object detection frames with a reference image library; the reference image library stores the reference image of each desktop tool; If there is an unmatched desktop object detection frame, the laboratory table information corresponding to the camera module and the information of the person who recently reserved the laboratory table are used as the desktop abnormality recognition result.

15. The management method according to claim 14, characterized in that: The camera module is provided with a sound unit, and the reminding the associated subject includes: Selecting the camera module that reports the desktop abnormality recognition result as the starting camera module; Obtaining a departure time difference according to the time of reporting the desktop abnormality recognition result and the reservation end time of the experimental table corresponding to the starting camera module; Obtaining a departure distance according to the departure time difference and a preset walking speed; Determine the camera module whose distance from the starting camera module is within the departure distance as the second target camera module; Control the second target camera module to issue a voice broadcast according to the experimental platform information and the personnel information.

16. The management method according to claim 14, characterized in that: The desktop detection condition includes the time when the reservation time of the experimental table corresponding to the camera module ends.

17. The management method according to claim 2, characterized in that: The abnormality recognition mechanism includes a channel debris abnormality recognition mechanism, the abnormality recognition result includes a channel debris abnormality recognition result, the camera module corresponding to the channel debris abnormality recognition mechanism is used to shoot the platform channel, and the channel debris abnormality recognition mechanism is used to perform abnormality recognition on the monitoring area, including: The picture frames captured by the camera module are used to update the long-term background model and the short-term background model respectively; the long-term background model and the short-term background model are both used to extract the background image in the picture, and the learning rate of the long-term background model is lower than that of the short-term background model; Input the picture frame into the long-term background model to obtain a first image, and input the picture frame into the short-term background model to obtain a second image; Comparing the first image with the second image, and determining whether a difference region in the obtained difference image is a debris region; The channel debris abnormality recognition result is obtained according to the determined debris area.

18. The management method according to claim 17, characterized in that: The determining whether the difference area is a debris area includes: Inputting the difference image into a fifth detection model; the fifth detection model is used to generate a filter detection frame for the target filter image in the image; If each of the difference regions in the difference image has a corresponding filtering detection frame, determining that the debris region does not exist in the difference image; Otherwise, the difference area for which there is no corresponding filtering detection frame is determined as the debris area.

19. The management method according to claim 17, characterized in that: The determining of the associated subject according to the abnormal identification result includes: Determine the channel area corresponding to the camera module that reports the channel debris abnormality recognition result as the target channel area; According to the target channel area, determine the corresponding cleaning person in charge; The cleaning person in charge corresponding to the target channel area is determined as the associated subject.

20. A management device for a micro-nano processing platform, characterized in that: The micro-nano processing platform is provided with a plurality of camera modules, and personnel in the micro-nano processing platform wear an identification pattern combination consisting of a preset number of different identification patterns, and the identification pattern combination corresponds to a unique detection mark, and the management device includes: The result acquisition module uses at least one abnormality recognition mechanism configured by each camera module to perform abnormality recognition on the monitoring area corresponding to each camera module to obtain abnormality recognition results, wherein the abnormality recognition results include personnel abnormality recognition results; specifically including: inputting the picture frame taken by the camera module into the first detection model, and obtaining the first detection result according to the output of the first detection model; the first detection model is used to generate a personnel detection frame for the personnel appearing in the picture; assigning a unique identifier corresponding to the identity of the person to each of the first detection results; and taking the first detection result containing the abnormal item as a personnel abnormality recognition result; The assigning of a unique identifier corresponding to a person's identity to each of the first detection results comprises: inputting the picture frame into a third detection model; the third detection model is used to generate a pattern detection frame for the recognition pattern appearing in the picture; when the third detection model outputs a plurality of pattern detection frames, clustering the pattern detection frames, and determining a clustering cluster having the preset number of pattern detection frames as a target clustering cluster; determining a center position of each of the target clustering clusters, and matching each of the target clustering clusters with the person detection frame according to the center position and the position of each of the person detection frames; and using a multi-target tracking algorithm to track the The picture frame is processed to obtain a unique tracking identifier corresponding to each of the personnel detection frames; for any of the personnel detection frames, if there is a matching target cluster, the unique detection identifier is determined according to the matched target cluster, and the unique detection identifier is used as the unique identifier, and the unique detection identifier is bound to the unique tracking identifier; if there is no matching target cluster, the unique identifier is obtained according to the unique detection identifier bound to the unique tracking identifier of the personnel detection frame; a recording module is used to determine an associated subject according to the abnormality recognition result, and update the abnormality record of the associated subject; A processing level adjustment module is used to upgrade the processing level of the associated subject if the abnormal record meets the upgrade conditions, otherwise, maintain the processing level of the associated subject; the processing level reflects the degree of management personnel's participation and the strength of the processing means; The processing module processes the associated subject using a processing method corresponding to the current processing level.

21. A computer device, characterized in that: It includes one or more processors and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the management method of the micro-nano processing platform as described in any one of claims 1-19 are executed.

22. A storage medium, characterized in that: The storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the management method of the micro-nano processing platform as described in any one of claims 1-19.

Images