A distributed streaming narrowband transmission optimization method

Abstract

This invention provides a distributed streaming media narrowband transmission optimization method, involving data processing technology. The method retrieves the transmission bandwidth of the narrowband transmission and the monitoring streaming media collected by a distributed acquisition device within a target area, and obtains the current bitrate of each monitoring frame in the monitoring streaming media. When the current bitrate is determined to be greater than the transmission bandwidth, the corresponding monitoring frame is used as an encoded frame. The encoded frame is then partitioned according to a partitioning strategy and a transmission bandwidth control encoding model to obtain a display frame. Based on time sequence, the monitoring frame and the display frame are combined to obtain the transmission streaming media, which is then sent to the monitoring terminal.

Description

Technical Field

[0001] This invention relates to data processing technology, and more particularly to a method for optimizing distributed streaming media narrowband transmission. Background Technology

[0002] Currently, due to the limited bandwidth of narrowband networks, the transmission of streaming media data (such as video streams) often faces the problem of bitrate mismatch with bandwidth, which can easily lead to stuttering, latency, or even interruption. To solve this problem, the industry generally adopts the method of reducing video bitrate to adapt to narrowband bandwidth. Common methods include compressing video frame size, reducing frame rate, or using efficient encoding algorithms. The core goal is to achieve stable transmission of streaming media within limited bandwidth, while preserving as much key information in the video as possible to meet users' basic viewing needs in scenarios such as monitoring and communication.

[0003] However, most existing technologies for reducing video bitrate employ a method of compressing the entire video frame. This approach fails to differentiate the importance of different regions within the video frame, applying a uniform compression strategy to both critical areas of focus (such as workers in a factory monitoring scenario or speakers in a meeting) and irrelevant areas (such as static backgrounds or empty areas). This directly leads to two problems: First, to meet narrowband bandwidth limitations, overall compression often requires a significant reduction in bitrate, excessively sacrificing the clarity of critical areas and failing to meet users' needs for identifying core information. Second, irrelevant areas are either undercompressed or compressed to the same degree as critical areas, resulting in wasted bandwidth resources on unnecessary information and hindering precise bitrate control.

[0004] Therefore, how to partition and customize the bitrate of each region within an image frame in streaming media according to the actual situation, so as to adapt to narrow bandwidth while maintaining the required clarity, thus ensuring the satisfaction of the user's core needs and achieving more efficient use of bitrate, has become an urgent problem to be solved. Summary of the Invention

[0005] This invention provides a distributed streaming media narrowband transmission optimization method, which can partition and customize each region within an image frame in the streaming media according to the actual situation to reduce the bit rate. This adapts to the narrowband bandwidth while maintaining the clarity of the required parts as much as possible, ensuring that the core needs of users are met and achieving relatively efficient use of bit rate.

[0006] A first aspect of this invention provides a method for optimizing narrowband transmission of distributed streaming media, comprising: The transmission bandwidth of the narrowband transmission and the monitoring streaming media collected by the distributed acquisition device in the target area are retrieved, and the current bit rate of each monitoring frame in the monitoring streaming media is obtained. When it is determined that the current bit rate is greater than the transmission bandwidth, the corresponding monitoring frame is used as the encoding frame. The encoding frame is partitioned according to the partitioning processing strategy and the transmission bandwidth control encoding model to obtain the display frame. The monitoring frames and the display frames are combined in chronological order to obtain a transmission streaming media, which is then sent to the monitoring terminal.

[0007] Optionally, in one possible implementation of the first aspect, the step of partitioning the encoded frame according to the partitioning strategy and the transmission bandwidth control coding model to obtain the display frame includes: Identify the outlines of people and devices in the encoded frame, and perform customized cropping of the encoded frame based on the positional relationship between the outlines of people and devices to obtain the main body area and background area in the encoded frame; The main area and the background area are partitioned according to the partitioning strategy and the transmission bandwidth control coding model to obtain the display frame.

[0008] Optionally, in one possible implementation of the first aspect, the identification of the person's outline and the device's outline in the encoded frame, and the customized cropping of the encoded frame based on the positional relationship between the person's outline and the device's outline to obtain the main body region and the background region in the encoded frame, includes: Identify the outlines of people and devices in the encoded frames and perform coordinate processing on the encoded frames; When it is determined that the outline of the person and the outline of the device intersect, the main body region in the encoded frame is determined based on the coordinate extreme values ​​of the corresponding outline of the person. When it is determined that the figure outline and the device outline do not intersect, the figure region corresponding to the figure outline is taken as the main figure region in the encoded frame; Based on the selected main area and the main character area, the main area in the encoded frame is obtained, and the remaining areas in the encoded frame are used as the background area.

[0009] Optionally, in one possible implementation of the first aspect, the step of partitioning the main region and the background region according to a partitioning strategy and a transmission bandwidth control coding model to obtain a display frame includes: The main body region and the background region are partitioned and compressed according to the partition compression strategy control coding model to obtain an initial compressed frame, and the initial bit rate of the initial compressed frame is obtained. When the initial bit rate is determined to be less than or equal to the transmission bandwidth, the initial compressed frame is used as the display frame; When the initial bit rate is determined to be greater than the transmission bandwidth, the encoding model is controlled according to the main color block strategy to perform partition coarsening processing on the main body region and the background region to obtain an intermediate compressed frame, and the intermediate bit rate of the intermediate compressed frame is obtained. When the intermediate bit rate is determined to be less than or equal to the transmission bandwidth, the intermediate compressed frame is used as the display frame; When the intermediate bit rate is determined to be greater than the transmission bandwidth, the encoding model is controlled according to the contour preservation strategy to perform partitioning and stripping processing on the main body region and the background region, and the final compressed frame is used as the display frame.

[0010] Optionally, in one possible implementation of the first aspect, the step of performing partitioned compression processing on the main region and the background region according to the partitioned compression strategy control coding model to obtain an initial compressed frame includes: Retrieve the preset display area and preset display bitrate corresponding to the preset display area, obtain the main area area of ​​each main area, and obtain the main body proportion based on the ratio of the main body area to the preset display area; The initial subject bitrate of each subject area is obtained by multiplying the subject proportion and the preset display bitrate. Retrieve the preset background area and preset background bitrate corresponding to the preset background area, obtain the background area of ​​the background area, and obtain the background ratio based on the ratio of the background area to the preset background area; The initial background bitrate of the background region is obtained by multiplying the background ratio and the preset background bitrate. The main body region is compressed according to the initial main body bitrate control coding model, and the background region is compressed according to the initial background bitrate control coding model to obtain the initial compressed frame.

[0011] Optionally, in one possible implementation of the first aspect, when determining that the initial bit rate is greater than the transmission bandwidth, performing partitioning and coarsening processing on the main body region and the background region according to the main color block strategy control coding model to obtain an intermediate compressed frame includes: When the initial bit rate is greater than the transmission bandwidth, the character region corresponding to the character outline in the main body region is obtained, and the associated region is obtained based on the difference between the main body region and the character region. The outline of the character is retrieved, and the character area is segmented according to the outline to obtain the fill area corresponding to each outline. The outline has a corresponding preset compression rate. A weighted average is calculated on the pixel values ​​of the pixels in each of the color-filled regions to obtain the fill pixel value of each color-filled region. Based on the fill pixel value, the coding model is used to perform color-filling processing on the color-filled regions. Obtain preset key points corresponding to the contour parts, and perform horizontal segmentation on the associated areas based on the preset key points to obtain the upper torso segmentation area located in the upper part, the middle torso segmentation area located in the middle part, and the lower torso segmentation area located in the lower part. Based on the upper torso segmentation area, the middle torso segmentation area, and the lower torso segmentation area, the torso segmentation areas corresponding to each direction are obtained; The preset compression ratios of the contour parts adjacent to the torso segmentation area within the main body region are calculated by weighted average to obtain the orientation compression ratio corresponding to the torso segmentation area. Based on the orientation compression ratio control coding model, the corresponding torso segmentation area is subjected to regional compression processing, and the background area is removed to obtain the intermediate compressed frame.

[0012] Optionally, in one possible implementation of the first aspect, when determining that the intermediate bit rate is greater than the transmission bandwidth, the encoding model is controlled according to the contour preservation strategy to perform partitioning and stripping processing on the main body region and the background region to obtain a final compressed frame as a display frame, including: When the intermediate bit rate is determined to be greater than the transmission bandwidth, the control encoding model strips the background area and retains only the outline of the character in the main area, thus obtaining the final compressed frame as the display frame.

[0013] Optionally, in one possible implementation of the first aspect, it also includes: The acquisition position of the distributed acquisition device corresponding to the display frame is retrieved, and the acquisition position includes a constraint area and a working area. When the collection location is determined to be a constraint area, the constraint part of the corresponding constraint area is retrieved. Based on the constraint part, the outline of the person in the corresponding display frame is processed to obtain the verification outline corresponding to the constraint area. The verification outline is then verified to obtain the verification result corresponding to the constraint area. When the collection location is determined to be a work area, the corresponding work part of the work area is retrieved. Based on the work part, the outline of the person in the corresponding display frame is processed to obtain the associated outline corresponding to the work area. The associated outline is then verified to obtain the verification result corresponding to the work area.

[0014] Optionally, in one possible implementation of the first aspect, when the determined acquisition location is a constraint area, the constraint part of the corresponding constraint area is retrieved, and the outline of the person in the corresponding display frame is subjected to part detail processing based on the constraint part to obtain a verification outline corresponding to the constraint area. The verification outline is then verified to obtain a verification result corresponding to the constraint area, including: When the data collection location is determined to be a constrained area, retrieve the constrained portion of the corresponding constrained area; Retrieve the encoded frame corresponding to the corresponding display frame as a refinement frame, and identify the outline of the wearing details at the constrained part in the refinement frame; Based on the center point of the refined frame, the refined frame is processed into coordinates, and the coordinates of the wear detail contour are counted to obtain the set of wear coordinates corresponding to the wear detail contour. Based on the center point of the display frame, the display frame is processed into coordinates. Based on the set of wear coordinates, the outline pixels corresponding to the wear detail outline are moved to the figure outline in the corresponding display frame to obtain the verification outline corresponding to the constraint area. By statistically analyzing the pixel values ​​corresponding to the contour pixels in the verification contour, a set of verification pixel values ​​corresponding to the verification contour is obtained. Retrieve the standard pixel value set corresponding to the corresponding constraint area, and obtain the first verification set based on the difference between the standard pixel value set and the verification pixel value set; If it is determined that the first verification set is not an empty set, a pixel value failure result is generated. When the first verification set is determined to be an empty set, the number of contour pixels in the verification contour is counted based on each pixel value in the verification pixel value set to obtain the second verification number corresponding to each pixel value. The second verification count is counted to obtain the total verification count. The second verification percentage for each pixel value is obtained based on the ratio of the second verification count for each pixel value to the total verification count. Retrieve the standard pixel value ratio corresponding to the corresponding constraint area. Based on the absolute value of the difference between the standard pixel value ratio and the second verification ratio, obtain the deviation value of each pixel value. Calculate the deviation value of each pixel value to obtain the total deviation value. When the total deviation value is determined to be greater than or equal to a preset deviation value, a percentage failure result is generated; If the total deviation value is less than the preset deviation value, a verification pass result is generated.

[0015] Optionally, in one possible implementation of the first aspect, when the determination that the acquisition location is a work area, the corresponding work part of the work area is retrieved, and the outline of the person in the corresponding display frame is processed by part association based on the work part to obtain the associated outline corresponding to the work area. The associated outline is then verified to obtain a verification result corresponding to the work area, including: When the data collection location is determined to be a work area, retrieve the corresponding work area location; Retrieve the encoded frame corresponding to the corresponding display frame as the associated frame, identify the contact detail contour at the working part in the associated frame, and the outline of the operating object that intersects with the contact detail contour; Based on the center point of the associated frame, the associated frame is processed into coordinates, and the coordinates of the contact detail contour and the operation object contour are counted to obtain the contact coordinate set corresponding to the contact detail contour and the object coordinate set corresponding to the operation object contour. The display frame is coordinate-based according to the center point of the display frame. Based on the contact coordinate set, the contour pixels corresponding to the contact detail contour are moved to the figure contour in the corresponding display frame. Based on the object coordinate set, the contour pixels corresponding to the operation object contour are moved to the figure contour in the corresponding display frame, thus obtaining the contour associated with the work area. The coordinates of the area corresponding to the standard work area in the associated frame are counted to obtain the set of standard work coordinates corresponding to the standard work area, and the contour coordinates of the associated contour are counted to obtain the set of associated contour coordinates. When all coordinates in the associated contour coordinate set are within the standard operation coordinate set, a verification result for the operation area is generated.

[0016] A second aspect of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program, wherein the computer program is stored in the memory, and the processor executes the computer program to perform the methods described in the first aspect of the present invention and various possible methods related to the first aspect.

[0017] A third aspect of the present invention provides a storage medium storing a computer program, which, when executed by a processor, is used to implement the first aspect of the present invention and various methods possibly involved in the first aspect.

[0018] The beneficial effects of this invention are as follows: 1. This invention, by retrieving the transmission bandwidth and the current bitrate of the monitoring frame, only processes the encoded frames whose bitrate exceeds the bandwidth, rather than uniformly compressing all frames. This avoids the waste of computing power caused by indiscriminate processing and focuses on key areas (such as people and equipment) through the partitioning strategy, maximizing the preservation of core monitoring information while adapting to narrowband bandwidth. Compared to traditional overall compression methods, its core advantages lie in on-demand processing and partitioned differentiated optimization. This solves the stuttering and interruption problems caused by the mismatch between bitrate and bandwidth in narrowband transmission and avoids the loss of information in key areas due to over-compression, providing stable streaming media data for the monitoring end.

[0019] 2. This invention identifies the outlines of people and devices in the encoded frame and, based on their positional relationship (whether they intersect), custom-extracts the main body region (such as an area containing interactive people and devices, or an independent person region), using the remaining area as the background region. A progressive processing flow is designed for the encoded frame, involving partitioned compression, partitioned coarsening, and partitioned stripping of the main and background regions: First, a partitioned compression strategy allocates differentiated bitrates to the main body and background. If the bitrate still exceeds the bandwidth, a main body color block strategy is used to fill in the person region, compress related regions, and remove the background; in extreme cases, only the person outline is retained. This dynamic adaptation mechanism can flexibly adjust the processing intensity according to real-time bandwidth, retaining more details when bandwidth is sufficient and prioritizing the retention of core outlines when bandwidth is limited. This avoids the problem of a single compression method being unable to adapt to bandwidth fluctuations and ensures monitoring information under different bandwidth conditions, achieving better allocation of narrowband resources.

[0020] 3. This invention refines the data collection locations (constraint areas, work areas) to supplement and verify key details for different scenarios: in constraint areas (such as high-voltage areas), compliance verification is performed by restoring wearing details (such as safety helmets); in work areas (such as assembly areas), operational procedures are ensured by supplementing the interaction details between the operating parts and objects. This makes narrowband streaming media not only stable but also provides the management end with effective information that can be used for safety verification and operational procedure judgment. Attached Figure Description

[0021] Figure 1 This is a schematic diagram illustrating the application scenario of the technical solution provided by the present invention; Figure 2 A flowchart of a distributed streaming media narrowband transmission optimization method provided by the present invention; Figure 3 A schematic diagram of a torso segmentation region provided by the present invention; Figure 4 This is a schematic diagram of the hardware structure of an electronic device provided by the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein.

[0024] It should be understood that in the various embodiments of the present invention, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0025] It should be understood that in this invention, "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or device.

[0026] It should be understood that in this invention, "multiple" refers to two or more. "And / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "Contains A, B, and C", "Contains A, B, and C" means that all three A, B, and C are contained; "Contains A, B, or C" means that one of A, B, and C is contained; "Contains A, B, and / or C" means that any one, two, or three of A, B, and C are contained.

[0027] It should be understood that in this invention, "B corresponding to A", "B corresponding to A", "A and B correspond", or "B and A correspond" means that B is associated with A, and B can be determined based on A. Determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information. Matching A and B is defined as a similarity between A and B that is greater than or equal to a preset threshold.

[0028] Depending on the context, "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection."

[0029] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0030] like Figure 1 The diagram shows a scenario illustrating the technical solution provided by this invention. The application scenario includes a server, a distributed acquisition device, and a monitoring terminal. The server is communicatively connected to the distributed acquisition device and the monitoring terminal. When the transmission bandwidth does not match the monitoring streaming media acquired by the distributed acquisition device, the server performs partitioned compression processing on the corresponding monitoring frames in the monitoring streaming media using an encoding model. Finally, the compressed and compliant streaming media is sent to the monitoring terminal. The distributed acquisition device can be a surveillance camera (C1-C8) used to monitor the working status of personnel in various areas of the factory. The monitoring terminal can be a mobile terminal corresponding to the administrator, such as a tablet, mobile phone, or computer, etc., without limitation.

[0031] This invention provides a distributed streaming media narrowband transmission optimization method, such as... Figure 2 As shown, steps S1-S3 are included: S1, retrieve the transmission bandwidth of the narrowband transmission and the monitoring streaming media collected by the distributed acquisition device in the target area, and obtain the current bit rate of each monitoring frame in the monitoring streaming media.

[0032] It should be noted that in narrowband environments, transmission bandwidth is limited and fluctuates greatly, while the frame bitrate of monitoring streaming media changes dynamically with the scene (e.g., when the scene is simple and there are few staff, the corresponding image bitrate is low; when the scene is complex and the number of staff increases, the corresponding bitrate increases). Currently, the method adapted to narrowband transmission is to compress the entire streaming media image using a lightweight encoding model. The higher the compression rate, the lower the corresponding bitrate (data volume), resulting in poorer image quality (more blurry, fewer details), which can lead to the loss of content in key areas and is unavoidable.

[0033] Narrowband transmission refers to a network environment with low transmission bandwidth; transmission bandwidth refers to the maximum data transmission rate that the current network can support (e.g., 100kbps); distributed acquisition devices refer to monitoring equipment (e.g., high-definition cameras, mobile monitoring terminals) distributed in different locations in the target area (e.g., factory area, park); monitoring streaming media refers to the video stream (composed of multiple monitoring frames) acquired and generated by the acquisition device; monitoring frame refers to a single frame image in the streaming media; current bitrate refers to the data volume (kbps) of a single monitoring frame, reflecting the transmission resource requirements of that frame.

[0034] Understandably, the server determines the current network's transmission capacity by accessing the real-time transmission bandwidth of narrowband transmission; simultaneously, it acquires the monitoring streaming media collected by all distributed acquisition devices (such as multiple cameras distributed in various locations within the factory area) within the target area; subsequently, it performs frame parsing on the monitoring streaming media, extracts the current bitrate of each monitoring frame, completes the data matching of bandwidth and bitrate before transmission, and provides a basis for subsequent determination of which frames need to be processed.

[0035] Through the above implementation method, we will subsequently select some video frames in the monitoring streaming media for partitioned compression, and not compress video frames that meet the transmission conditions, thereby improving processing efficiency while ensuring the overall quality of the picture.

[0036] S2, when it is determined that the current bit rate is greater than the transmission bandwidth, the corresponding monitoring frame is used as the encoding frame, and the encoding frame is partitioned according to the partitioning processing strategy and the transmission bandwidth control encoding model to obtain the display frame.

[0037] Here, "encoded frame" refers to a monitoring frame whose current bit rate exceeds the transmission bandwidth and requires processing before transmission; "partition processing strategy" refers to the rules for dividing and differentiating encoded frames according to the importance of the monitored content (e.g., people are the core in a factory, and the background is secondary); "encoding model" refers to an algorithm model that can compress and process the images in the video frame. This is existing technology. By using the encoding model to compress or process the image, a higher compression rate results in a lower bit rate and thus a worse image quality, while a lower compression rate results in a higher bit rate and thus a better image quality; "display frame" refers to a frame whose bit rate meets the transmission bandwidth requirements and retains key information after partition processing of the encoded frame.

[0038] In some embodiments, step S2 (partitioning the encoded frame according to the partitioning strategy and the transmission bandwidth control coding model to obtain the display frame) includes S21-S22: S21, identify the outlines of the person and the device in the encoded frame, and perform customized cropping of the encoded frame based on the positional relationship between the outlines of the person and the device to obtain the main area and background area in the encoded frame.

[0039] Among them, the human figure outline refers to the external outline of the human body extracted through image recognition; the equipment outline refers to the edge outline of the equipment in the factory in the encoded frame, which can be extracted by OpenCV or other methods without limitation; the positional relationship refers to the spatial relationship between the human figure outline and the equipment outline (such as whether they overlap or have intersection); the main body region refers to the key area of ​​interest containing the human figure; the background region refers to the area of ​​interest in the encoded frame other than the main body region (such as walls and the ground).

[0040] It is understandable that OpenCV can be used to identify the outlines of people and equipment in the encoded frame; then the positional relationship is analyzed: if there is an intersection (such as a person operating equipment), the smallest rectangular area containing both is extracted; if they are independent (such as a person standing next to equipment), the person area and the equipment area are extracted separately and merged into the main area; finally, the part of the encoded frame not covered by the main area is determined as the background area.

[0041] In some embodiments, step S21 (identifying the outlines of a person and a device in the encoded frame, and performing customized cropping of the encoded frame based on the positional relationship between the outlines of the person and the device to obtain the main body region and the background region in the encoded frame) includes S211-S214: S211, identify the outlines of people and devices in the encoded frame, and perform coordinate processing on the encoded frame.

[0042] S212, when it is determined that the outline of the person and the outline of the device have an intersection, the main body region in the encoded frame is determined based on the coordinate extreme value of the corresponding outline of the person.

[0043] It should be noted that when the outlines of a person and the outlines of equipment overlap (such as when a person is operating equipment), the smallest rectangular area containing both is selected. This is mainly to ensure the integrity of the information while taking into account transmission efficiency and subsequent analysis needs. This area can fully include the interaction details of the person's actions and the equipment status (such as the correspondence between hand operations and equipment buttons, and the linkage between body posture and equipment operation). This avoids the logical coherence of the operation process being disrupted due to the separation of the two related aspects caused by separate selection. For example, in factory assembly line monitoring, if only the worker's hand or only the button is selected when a worker presses the equipment start button, it is impossible to determine whether the pressing action successfully triggered the equipment to start. However, the smallest rectangular area containing the hand and the button in contact can clearly show the pressing action and also be associated with the feedback of the equipment's start status.

[0044] Understandably, by comparing the coordinate sets of the figure's outline and the device's outline, it is determined whether there are overlapping coordinates (i.e., intersection); if there is an intersection, the extreme values ​​of the horizontal and vertical coordinates of the figure's outline are extracted, and a rectangular area that encloses the figure's outline is constructed based on the extreme values ​​of the horizontal and vertical coordinates as the main selection area.

[0045] S213, when it is determined that the character outline and the device outline do not intersect, the character region corresponding to the character outline is taken as the main character region in the encoded frame.

[0046] It should be noted that when there is no overlap between personnel and equipment (such as personnel patrolling or equipment being stationary), the dynamic behavior of personnel is the focus of monitoring, unrelated to the equipment; the equipment can be considered secondary information. Traditional methods still include equipment in the main monitoring area, which increases the amount of invalid data.

[0047] Therefore, when it is determined that the outline of the person and the outline of the device do not intersect, the person may be in a moving state. In this case, the person area corresponding to the outline of the person will be directly used as the main body area of ​​the person in the encoded frame. That is, only the actual state of the person is recorded, without considering the surrounding related areas.

[0048] S214, based on the selected main body area and the character main body area, the main body area in the encoded frame is obtained, and the remaining areas in the encoded frame are used as the background area.

[0049] It's easy to understand that if both the selected main area and the main subject area are considered the main area in the encoded frame, then the remaining areas in the encoded frame are considered the background area. In other words, the main area is the focus of attention in the encoded frame, while the background area is the area that is not of primary concern.

[0050] S22, the main area and the background area are partitioned according to the partitioning strategy and the transmission bandwidth control coding model to obtain the display frame.

[0051] It should be noted that the information importance of the subject area and the background area differs (e.g., the actions of the characters in the subject area need to be clear, while the details in the background area can be simplified). Traditional uniform compression processing can lead to blurry subjects or redundant background data. By using differentiated partitioning processing, the clarity of the subject area can be better preserved while ensuring that the total bitrate meets the bandwidth requirements.

[0052] In some embodiments, step S22 (partitioning the main region and the background region according to the partitioning strategy and the transmission bandwidth control coding model to obtain a display frame) includes S221-S225: S221, the main region and the background region are partitioned and compressed according to the partitioned compression strategy control coding model to obtain an initial compressed frame, and the initial bit rate of the initial compressed frame is obtained.

[0053] It should be noted that the information importance of the main body area and the background area is different. In narrowband transmission, the bit rate needs to be allocated differently (such as retaining more details in the main body area and appropriately simplifying the background area). However, traditional uniform compression of the whole frame will lead to blurring of the main body or redundancy of background data.

[0054] Therefore, my solution will perform targeted partitioning and step-by-step compression (progressive): it is more suitable for scenarios with frequent bandwidth fluctuations, and can dynamically degrade according to real-time bandwidth to retain as much information as possible (without sacrificing key details unless absolutely necessary).

[0055] In some embodiments, step S221 (performing partitioned compression of the main region and the background region according to the partitioned compression strategy control coding model to obtain an initial compressed frame) includes S2211-S2215: S2211, retrieve the preset display area and preset display bitrate corresponding to the preset display area, obtain the main area area of ​​each main area, and obtain the main proportion based on the ratio of the main area area to the preset display area.

[0056] It should be noted that the area of ​​the subject area varies for different subjects (e.g., the area of ​​a person is larger when shot up close, and smaller when shot from a distance). If a fixed bitrate compression is used, it will lead to over-compression (blurring) of small subjects or waste of bitrate for large subjects. This is because, for example, the larger the area occupied by a person, the more details are displayed, and a higher bitrate will be allocated. On the other hand, the smaller the outline of a person, the smaller the area occupied, and the fewer details need to be displayed, so a lower bitrate will be allocated.

[0057] Among them, the preset display area is a pre-set display area, the preset display area is the area corresponding to the preset display area, and the preset display bitrate is the bitrate for display in the preset display area. All of these are set manually based on the actual situation.

[0058] It is easy to understand that for a given preset display area, the preset display bitrate can be set in advance, so that details can be displayed while a certain degree of compression is performed. The larger the area, the higher the corresponding bitrate. Based on the relationship between the preset display area and the area, the actual bitrate is confirmed to facilitate subsequent compression. That is, we will determine different bitrates based on different area sizes to facilitate subsequent compression processing.

[0059] Specifically, the preset display area and preset display bitrate corresponding to the preset display area are retrieved to obtain the main area area of ​​each main area. The main proportion is obtained based on the ratio of the main area area to the preset display area. The main proportion and the preset display bitrate are then calculated.

[0060] S2212, Based on the product of the main body proportion and the preset display bitrate, the initial main body bitrate of each main body region is obtained.

[0061] It should be noted that traditional fixed bitrates cannot adapt to changes in area, leading to unreasonable resource allocation.

[0062] Therefore, the server will obtain the initial bitrate of each subject area based on the product of the subject proportion and the preset display bitrate, ensuring that we can dynamically determine the bitrate of different areas for display according to the different area sizes, which facilitates targeted compression in the future.

[0063] S2213, retrieve the preset background area and preset background bitrate corresponding to the preset background area, obtain the background area of ​​the background area, and obtain the background ratio based on the ratio of the background area to the preset background area.

[0064] Similarly, since the background area is a non-critical area, we will also pre-set a corresponding preset background area and configure a corresponding preset background area and preset background bitrate for the preset background area, so as to facilitate subsequent calculation of the display bitrate of the background area corresponding to different area sizes.

[0065] Therefore, the background percentage is obtained based on the ratio of the background area to the preset background area.

[0066] S2214, Based on the product of the background ratio and the preset background bitrate, the initial background bitrate of the background region is obtained.

[0067] Similarly, the initial background bitrate of the background region is obtained by multiplying the background ratio and the preset background bitrate.

[0068] S2215, the main body region is compressed according to the initial main body bitrate control coding model, and the background region is compressed according to the initial background bitrate control coding model to obtain an initial compressed frame.

[0069] Understandably, the server controls the encoding model to compress the main region based on the initial main bitrate and the background region based on the initial background bitrate to obtain the initial compressed frame.

[0070] S222, when it is determined that the initial bit rate is less than or equal to the transmission bandwidth, the initial compressed frame is used as the display frame.

[0071] Understandably, the initial bitrate is compared with the transmission bandwidth. If the initial bitrate is less than or equal to the transmission bandwidth, it means that the initial compressed frame has met the requirements of narrowband transmission, and it is directly determined as the display frame.

[0072] S223, when it is determined that the initial bit rate is greater than the transmission bandwidth, the main body region and the background region are partitioned and coarsened according to the main color block strategy control encoding model to obtain an intermediate compressed frame, and the intermediate bit rate of the intermediate compressed frame is obtained.

[0073] It should be noted that when the initial compressed frame bitrate still exceeds the bandwidth, the bitrate needs to be further reduced, but the outline and color block features of the main area must be preserved. For people, it is only necessary to identify which person is whom by the color block filled in by the person. For example, if person B is wearing a red ornament around their neck, the person can be identified by the different color features of the person. However, the device that the person is in contact with needs to show relevant details, such as whether a button has been pressed. Therefore, we will divide the outline of the person into sections and fill them in according to the color blocks. However, the corresponding related areas will be compressed to reduce the clarity while retaining some details.

[0074] In some embodiments, step S223 (when it is determined that the initial bit rate is greater than the transmission bandwidth, the encoding model is used to perform partitioning and coarsening processing on the main body region and the background region according to the main color block strategy to obtain an intermediate compressed frame) includes S2231-S2237: S2231, when the initial bit rate is greater than the transmission bandwidth, obtain the character region corresponding to the character outline in the main area, and obtain the associated region based on the difference between the main area and the character region.

[0075] The character area refers to the main area containing only the outline of the character; the related area refers to the part of the main area excluding the character area (which will display the part of the person touching the device).

[0076] Understandably, when the initial bit rate exceeds the transmission bandwidth, the first step is to extract the person region corresponding to the person's outline from the main body region (by determining the boundary using the person's outline coordinates); then, the pixel difference between the main body region and the person region is calculated to obtain the associated region (i.e., the device or object region in the main body region that is associated with the person); by separating the two, it is ensured that the subsequent coarsening process can focus on the core features of the person and the associated device respectively.

[0077] S2232, retrieve the outline parts corresponding to the character outline, segment the character area according to the outline parts, and obtain the fill area corresponding to each outline part. The outline parts have a corresponding preset compression rate.

[0078] It should be noted that the monitoring value of different parts of a person is different (for example, in a factory assembly line, the focus is generally on the hand movements of the personnel, because the compression rate of the area near the hands is relatively low, thereby improving the image display quality of the area near the hands). In addition, for the details of the person, when the transmission bandwidth still does not meet the requirements, it is only necessary to show the person through the color area, without showing the details of the person.

[0079] Among them, the outline refers to the subdivided structure of the character's outline (such as the head, hands, legs, etc.); the fill area refers to the independent area obtained after dividing according to the outline (used for subsequent color block filling); the preset compression rate refers to the coarsening parameter set for different outline parts (such as a larger compression rate for the head and a smaller compression rate for the hands).

[0080] Therefore, the standard part division rules of the figure outline are retrieved (such as the hands, neck and head to divide the figure area), and the figure area is divided into color-filled areas such as head, torso, upper limbs and lower limbs according to the rules; at the same time, a corresponding preset compression rate is configured for each outline part to facilitate subsequent compression of the area associated with that part.

[0081] S2233, perform a weighted average calculation on the pixel values ​​of the pixels in each of the color-filled regions to obtain the fill pixel value of each color-filled region, and perform color-filling processing on the color-filled regions based on the control coding model of the fill pixel value.

[0082] In this context, it can be understood that for all pixels in each color-filled area (such as the neck), a weighted average of the RGB values ​​is calculated to obtain the fill pixel value. For example, the weighted average of the R, G, and B values ​​of all pixels in the neck can be calculated separately to obtain the fill pixel value. The control coding model uses this fill pixel value to fill the entire color-filled area, making the area appear as a uniform color block (such as the head being filled with "black" to correspond to hair color, and the neck being yellow to correspond to a person wearing yellow accessories), eliminating texture details to reduce the bitrate, and displaying only color blocks.

[0083] S2234, Obtain preset key points corresponding to the contour parts, and perform horizontal segmentation processing on the associated areas based on the preset key points to obtain the upper torso segmentation area located in the upper part, the middle torso segmentation area located in the middle part, and the lower torso segmentation area located in the lower part.

[0084] Among them, preset key points refer to preset reference points corresponding to the outline of the figure (such as the height point corresponding to the shoulder, the height point corresponding to the waist), which can be points that are set in advance by humans.

[0085] It should be noted that in factory settings, the main focus is on the three positions of the personnel: upper, middle, and lower. For example, whether the head is wearing a safety helmet, whether the arms in the middle are in contact with the equipment and how they are operated, and whether the lower limbs are in the correct area, such as whether they are standing on an electrostatic isolation mat when operating equipment.

[0086] Therefore, key points can be pre-set by humans, for example, see Figure 3 The key areas are divided by the height points corresponding to the shoulders and the waist, thus obtaining the upper torso segmentation area in the upper part, the middle torso segmentation area in the middle part, and the lower torso segmentation area in the lower part.

[0087] S2235, based on the upper torso segmentation area, the middle torso segmentation area, and the lower torso segmentation area, the torso segmentation areas corresponding to each direction are obtained.

[0088] S2236, calculate the weighted average of the preset compression rates of the contour parts adjacent to the torso segmentation area within the main body area to obtain the orientation compression rate corresponding to the torso segmentation area.

[0089] Understandably, the process involves identifying the human figure outlines adjacent to each torso segmentation area (e.g., the upper torso segmentation area is close to the head and neck, while the middle area is the hands and torso). The preset compression rates of these parts are extracted and a weighted average is calculated to obtain the orientation compression rate of the corresponding torso segmentation area. For example, the upper limb compression rate is 0.3, ensuring that the coarsening level of the associated area matches that of the human figure area. Generally, the compression rate of the middle torso segmentation area is lower due to the presence of the hands, and more details are preserved after compression, thus better displaying the details within the middle torso segmentation area.

[0090] S2237, Based on the orientation compression ratio control coding model, perform regional compression processing on the corresponding torso segmentation area and remove the background area to obtain the intermediate compressed frame.

[0091] Understandably, the control coding model processes each torso segmentation region according to the orientation compression rate. The coding model can compress the region to a fixed bit rate or compress it according to a fixed compression rate, while completely removing the background region (not transmitting any background pixels). Finally, the filled-in character region is merged with the compressed associated region to generate an intermediate compressed frame.

[0092] S224, when it is determined that the intermediate bit rate is less than or equal to the transmission bandwidth, the intermediate compressed frame is used as the display frame.

[0093] Understandably, by comparing the intermediate bitrate with the transmission bandwidth, if the intermediate bitrate is less than or equal to the transmission bandwidth, it means that the coarsened intermediate compressed frame has met the transmission requirements, and the intermediate compressed frame is determined as the display frame.

[0094] S225, when it is determined that the intermediate bit rate is greater than the transmission bandwidth, the encoding model is controlled to perform partitioning and stripping processing on the main body region and the background region according to the contour preservation strategy, so as to obtain the final compressed frame as the display frame.

[0095] It should be noted that in extremely narrow bandwidth conditions (such as when the bandwidth is much lower than the intermediate bit rate), the bit rate needs to be reduced to the maximum. At this time, the outline information of the subject (such as the outline of a person) still has monitoring value (such as determining whether there is a person and the movement of the person).

[0096] Therefore, we will only record personnel outlines and remove the rest.

[0097] In some embodiments, step S225 (when it is determined that the intermediate bitrate is greater than the transmission bandwidth, controlling the encoding model to perform partitioning and stripping processing on the main body region and the background region according to the contour preservation strategy to obtain the final compressed frame as the display frame) includes: When the intermediate bit rate is determined to be greater than the transmission bandwidth, the control encoding model strips the background area and retains only the outline of the character in the main area, thus obtaining the final compressed frame as the display frame.

[0098] Understandably, when the intermediate bitrate still exceeds the transmission bandwidth, the outline preservation strategy is activated: first, the encoding model is controlled to completely strip the background area (without retaining any background pixels); then, edge extraction processing is performed on the main body area, retaining only the line information of the figure's outline (removing all color blocks inside the outline); finally, a final compressed frame containing only the black and white figure outline is generated and transmitted as a display frame, retaining core information such as whether there are people and their shapes at the lowest bitrate.

[0099] S3, Based on the time sequence, the monitoring frame and the display frame are combined to obtain the transmission streaming media and sent to the monitoring terminal.

[0100] Understandably, the timestamps of the monitoring frames and display frames (recording the time of collection or generation) are extracted; then, the two types of frames are sorted in ascending order of timestamps, and a streaming media that meets the transmission bandwidth is generated and sent to the monitoring terminal, which can be the mobile terminal corresponding to the administrator.

[0101] Based on the above embodiments, A1-A3 are also included: A1, retrieve the acquisition position of the distributed acquisition device corresponding to the display frame, based on the acquisition position including the constraint area and the work area.

[0102] It should be noted that the monitoring focus differs significantly between different data collection locations (e.g., in areas with potential hazards, attention should be paid to whether personnel are wearing safety helmets correctly, and in work areas, attention should be paid to whether operations are standardized, such as whether work gloves are worn in the transfer area, and whether welding guns are held at the gloves, etc.), and different areas have different focuses.

[0103] Therefore, the server retrieves the acquisition device ID corresponding to each display frame and associates it with its preset acquisition location information; based on the location attributes, the acquisition location is divided into a constraint area (such as an area with safety hazards in the production line) and a work area (such as a welding area, assembly area, etc.), so that different parts of different areas can be focused on in the future.

[0104] A2, when the collection location is determined to be a constraint area, the constraint part of the corresponding constraint area is retrieved, and the outline of the person in the corresponding display frame is processed for detail based on the constraint part to obtain the verification outline corresponding to the constraint area. The verification outline is then verified to obtain the verification result corresponding to the constraint area.

[0105] In some embodiments, step A2 (when the acquisition location is determined to be a constraint area, the constraint part of the corresponding constraint area is retrieved, and the outline of the person in the corresponding display frame is processed for detail based on the constraint part to obtain a verification outline corresponding to the constraint area, and the verification outline is verified to obtain a verification result corresponding to the constraint area) includes A201-A212: A201, when the collection location is determined to be a constraint area, retrieve the constraint part of the corresponding constraint area.

[0106] It should be noted that the safety management requirements for different constraint areas are different (e.g., safety helmets and heads need to be verified in high-voltage areas), and the corresponding key constraint parts are different. Therefore, different constraint areas will have corresponding preset constraint parts.

[0107] A202, retrieve the encoded frame corresponding to the corresponding display frame as the refinement frame, and identify the wear detail contours at the constrained parts in the refinement frame.

[0108] It should be noted that the display frame may have been compressed or coarsened (e.g., color blocking), and details of the constrained parts (e.g., helmet color, markings, etc.) may be lost. Directly using it for verification would lead to insufficient accuracy. Therefore, this step extracts details from the original encoded frame.

[0109] Understandably, this step retrieves the original encoded frame corresponding to the display frame as a refinement frame, extracts the wearing details of constrained parts (such as safety helmets), and retains key information such as texture and markings.

[0110] A203, based on the center point of the refined frame, perform coordinate processing on the refined frame, count the coordinates of the wear detail contour, and obtain the wear coordinate set corresponding to the wear detail contour.

[0111] A two-dimensional coordinate system is established with the center point of the refined frame as the origin. The pixel coordinates of the detailed outline of the clothing are counted point by point to form a set of clothing coordinates, which provides a position reference for subsequent mapping to the display frame.

[0112] A204. Based on the center point of the display frame, the display frame is processed into coordinates. Based on the set of wear coordinates, the outline pixels corresponding to the wear detail outline are moved to the figure outline in the corresponding display frame to obtain the verification outline corresponding to the constraint area.

[0113] It is understandable that a coordinate system is established with the center point of the display frame as the origin, and the set of clothing coordinates is then converted into the corresponding coordinates in the display frame coordinate system. Based on the converted coordinates, the pixels of the clothing detail outline are superimposed onto the character outline in the display frame to form the verification outline. Both of these coordinate systems are established with the center point as the origin, so the coordinates of the same pixels are exactly the same and can be directly mapped.

[0114] A205, count the pixel values ​​corresponding to the contour pixels in the verification contour to obtain the set of verification pixel values ​​corresponding to the verification contour.

[0115] Understandably, we will first analyze the verification contours, for example, by statistically analyzing the color of the pixels corresponding to the pixels within the contours (e.g., yellow and black), to obtain a set of verification pixel values. Subsequently, we will retrieve the colors from the actual contours of the safety helmet corresponding to the constraint area. For example, if the helmet is yellow and the straps are black, we will perform initial color verification. If a person is violating regulations and not wearing a safety helmet, the corresponding contour colors will be inconsistent. Even if a person is wearing a non-safety helmet of the same color, we will still perform subsequent proportion recognition for judgment.

[0116] A206, retrieve the standard pixel value set corresponding to the corresponding constraint area, and obtain the first verification set based on the difference between the standard pixel value set and the verification pixel value set.

[0117] The standard pixel value set refers to the set of pixel values ​​of compliant wearables specified in the constraint area (e.g., the standard pixel value set for a safety helmet is yellow and black); the first verification set refers to the set of colors in the verification pixel value set that exceed the standard range.

[0118] A207, when it is determined that the first verification set is not an empty set, a pixel value failure result is generated.

[0119] Understandably, if this step determines that the first verification set is not empty (there are pixel values ​​that do not meet the standard), it will directly generate a verification result that the pixel value has failed, such as wearing a hat of a different color (e.g., the color of the safety helmet is not right, which is a violation).

[0120] A208, when it is determined that the first verification set is an empty set, the number of contour pixels in the verification contour is counted based on the pixel values ​​in the verification pixel value set, and the second verification number corresponding to each pixel value is obtained.

[0121] Understandably, this step occurs when the first verification set is empty (all pixel values ​​are compliant), meaning the colors match. However, to prevent people from wearing hats of similar colors, this solution will also count the number of outline pixels in the verification outline by verifying each pixel value in the verification pixel value set, and obtain the second verification quantity corresponding to each pixel value, such as the number of yellow pixels and the number of black pixels. For the color of the safety helmet, the color ratio is relatively fixed, for example, the black ratio is 1:10.

[0122] A209, count the second verification count to obtain the total verification count, and obtain the second verification percentage corresponding to each pixel value based on the ratio of the second verification count of each pixel value to the total verification count.

[0123] Understandably, this step sums up all the second verification counts to get the total number of verifications, and then calculates the ratio of the number of each pixel value to the total number, thereby obtaining the proportion of second verifications corresponding to each pixel value.

[0124] A210: Retrieve the standard pixel value ratio corresponding to the pixel value of the corresponding constraint area. Based on the absolute value of the difference between the standard pixel value ratio and the second verification ratio of each pixel value, obtain the deviation value of each pixel value. Calculate the deviation value of each pixel value to obtain the total deviation value.

[0125] Understandably, the standard pixel value ratio (e.g., black ratio 1:10) is retrieved, the deviation value of each pixel value is calculated, and the total deviation value is obtained by summing them.

[0126] A211, when the total deviation value is determined to be greater than or equal to the preset deviation value, a percentage failure result is generated.

[0127] Understandably, if the total deviation is greater than or equal to the preset value (e.g., 20%), it means that the corresponding safety equipment was not worn, and the percentage will be generated as a failure result.

[0128] A212, if the total deviation value is less than the preset deviation value, a verification pass result is generated.

[0129] Understandably, if the total deviation is less than the preset value (e.g., 20%), a verification result is generated (e.g., the wearer complies with the constraint area regulations).

[0130] A3, when the collection location is determined to be a work area, the corresponding work part of the work area is retrieved, and the outline of the person in the corresponding display frame is processed by part association based on the work part to obtain the associated outline corresponding to the work area. The associated outline is then verified to obtain the verification result corresponding to the work area.

[0131] In some embodiments, step A3 (when the collection location is determined to be a work area, the corresponding work part of the work area is retrieved, and the outline of the person in the corresponding display frame is processed by part association based on the work part to obtain the associated outline corresponding to the work area, and the associated outline is verified to obtain the verification result corresponding to the work area) includes A31-A3: A31, when the collection location is determined to be a work area, retrieve the corresponding work part of the work area.

[0132] It should be noted that the core operating parts differ in different work areas (for example, in the assembly area, the focus is on the interaction between the hands and the parts and whether it is carried out within the assembly area; when operating the electrical cabinet, whether the personnel wear work shoes and stand on the anti-static mat). Understandably, after determining the collection location as the work area, the system database retrieves the preset work parts for that area (e.g., hands for assembly lines, feet for electrical cabinet operations) to provide clear objects for subsequent extraction of contact details.

[0133] A32, retrieve the encoded frame corresponding to the corresponding display frame as the associated frame, identify the contact detail contour at the working part in the associated frame, and the contour of the operating object that intersects with the contact detail contour.

[0134] Among them, the contact detail contour refers to the contour corresponding to the clothing worn at the work site, and the operation object contour refers to the contour of the item that intersects with the contact detail contour.

[0135] Understandably, the original encoded frame corresponding to the display frame is retrieved as the associated frame, and the contact detail contour of the work area is extracted using OpenCV, such as the detail contour corresponding to the gloves worn by the hand. At the same time, the contour of the operation object that has pixel intersection with the contour (such as the item being assembled by the gloves) is identified.

[0136] A33, based on the center point of the associated frame, the associated frame is processed into coordinates, and the coordinates of the contact detail contour and the operation object contour are counted to obtain the contact coordinate set corresponding to the contact detail contour and the object coordinate set corresponding to the operation object contour.

[0137] Similarly, after processing the associated frames into coordinates, the set of contact coordinates corresponding to the contact detail outline and the set of object coordinates corresponding to the operation object outline are obtained. Subsequently, the details of the corresponding interested parts and the details of the contacted objects are directly updated to the display frames.

[0138] A34. Based on the center point of the display frame, the display frame is processed into coordinates. Based on the contact coordinate set, the contour pixels corresponding to the contact detail contour are moved to the character contour in the corresponding display frame. Based on the object coordinate set, the contour pixels corresponding to the operation object contour are moved to the character contour in the corresponding display frame, thus obtaining the contour associated with the work area.

[0139] It is understandable that a coordinate system is established with the center point of the display frame as the origin, and both coordinate systems are constructed with the center point as the center. The set of contact coordinates and the set of object coordinates are converted into corresponding coordinates in the display frame coordinate system. According to the converted coordinates, the pixels of the contact detail outline and the operation object outline are superimposed on the character outline of the display frame to form the associated outline (i.e., the details of the items worn on the display focus area and the details of the items touched).

[0140] A35, count the area coordinates corresponding to the standard work area in the associated frame to obtain the standard work coordinate set corresponding to the standard work area, and count the contour coordinates of the associated contour to obtain the associated contour coordinate set.

[0141] Understandably, the coordinate range of the standard work area corresponding to the associated frame is retrieved from the system (e.g., all coordinates within the area where the worker is allowed to assemble), and all coordinates within this range are counted to form a standard work coordinate set; at the same time, all pixel coordinates of the associated contour are extracted to form an associated contour coordinate set for subsequent range comparison.

[0142] A36, when it is determined that all coordinates in the associated contour coordinate set are within the standard operation coordinate set, a verification result for the operation area is generated.

[0143] Understandably, this step compares the associated contour coordinate set with the standard operation coordinate set. If all coordinates of the former belong to the latter (no pixels outside the range), then a verification result is generated, meaning that the personnel wearing gloves are performing the transfer action within the area, or the personnel wearing work shoes are at the electrostatic mat.

[0144] It is easy to understand that during welding, assembly and other operations, it is necessary to ensure that the hands are wearing gloves and that the outlines of the gloves intersect with the corresponding welding equipment or the parts to be assembled are within the corresponding assembly and welding areas, i.e., within the standard work area. However, some work areas require specific location requirements for the objects that personnel come into contact with. For example, in high-altitude work areas where objects need to be retrieved from heights, there are specific location requirements for the personnel's work boots and steps.

[0145] Based on the above embodiments, it also includes: When the work area is determined to be a high-altitude work area, obtain the adjacent tangent segments in the contact detail contour that are tangent to the contour of the object being operated.

[0146] The distance between the heel protrusion in the contact detail contour and the adjacent tangent segment is obtained. When it is determined that the distance is not within the preset distance range, an alarm message is generated.

[0147] See Figure 4 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. The electronic device 40 includes: a processor 41, a memory 42, and a computer program; wherein... The memory 42 is used to store the computer program, and the memory may also be flash memory. The computer program is, for example, an application program or functional module that implements the above method.

[0148] The processor 41 is configured to execute the computer program stored in the memory to implement the various steps performed by the device in the above method. For details, please refer to the relevant descriptions in the preceding method embodiments.

[0149] Alternatively, the memory 42 can be either standalone or integrated with the processor 41.

[0150] When the memory 42 is a device independent of the processor 41, the device may further include: Bus 43 is used to connect the memory 42 and the processor 41.

[0151] The present invention also provides a readable storage medium storing a computer program, which, when executed by a processor, is used to implement the methods provided in the various embodiments described above.

[0152] The readable storage medium can be a computer storage medium or a communication medium. A communication medium includes any medium that facilitates the transfer of computer programs from one location to another. A computer storage medium can be any available medium accessible to a general-purpose or special-purpose computer. For example, a readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application-Specific Integrated Circuit (ASIC). Alternatively, the ASIC can be located in a user equipment. Of course, the processor and the readable storage medium can also exist as discrete components in a communication device. The readable storage medium can be a read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0153] The present invention also provides a program product including executable instructions stored in a readable storage medium. At least one processor of the device can read the executable instructions from the readable storage medium, and the at least one processor executes the executable instructions to cause the device to implement the methods provided in the various embodiments described above.

[0154] In the embodiments of the above-described device, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0155] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A distributed streaming media narrowband transmission optimization method, characterized in that, include: The transmission bandwidth of the narrowband transmission and the monitoring streaming media collected by the distributed acquisition device in the target area are retrieved, and the current bit rate of each monitoring frame in the monitoring streaming media is obtained. When it is determined that the current bit rate is greater than the transmission bandwidth, the corresponding monitoring frame is used as the encoded frame. The encoded frame is then subjected to partition compression processing according to the partition processing strategy and the transmission bandwidth control encoding model to obtain the display frame. The acquisition position of the distributed acquisition device corresponding to the display frame is retrieved, and the acquisition position includes a constraint area and a working area; When the acquisition location is determined to be a constraint area, the constraint parts of the corresponding constraint area are retrieved. Based on the constraint parts, the outline of the person in the corresponding display frame is processed to obtain a verification outline corresponding to the constraint area. The verification outline is then verified to obtain a verification result corresponding to the constraint area, including: When the data collection location is determined to be a constrained area, retrieve the constrained portion of the corresponding constrained area; Retrieve the encoded frame corresponding to the corresponding display frame as a refinement frame, and identify the outline of the wearing details at the constrained part in the refinement frame; Based on the center point of the refined frame, the refined frame is processed into coordinates, and the coordinates of the wear detail contour are counted to obtain the set of wear coordinates corresponding to the wear detail contour. The display frame is coordinated based on the center point of the display frame. Based on the set of wear coordinates, the outline pixels corresponding to the wear detail outline are updated to the character outline in the corresponding display frame to obtain the verification outline corresponding to the constraint area. By statistically analyzing the pixel values ​​corresponding to the contour pixels in the verification contour, a set of verification pixel values ​​corresponding to the verification contour is obtained. Retrieve the standard pixel value set corresponding to the corresponding constraint area, and obtain the first verification set based on the difference between the standard pixel value set and the verification pixel value set; If it is determined that the first verification set is not an empty set, a pixel value failure result is generated. When the first verification set is determined to be an empty set, the number of contour pixels in the verification contour is counted based on each pixel value in the verification pixel value set to obtain the second verification number corresponding to each pixel value. The second verification count is counted to obtain the total verification count. The second verification percentage for each pixel value is obtained based on the ratio of the second verification count for each pixel value to the total verification count. Retrieve the standard pixel value ratio corresponding to the corresponding constraint area. Based on the absolute value of the difference between the standard pixel value ratio and the second verification ratio, obtain the deviation value of each pixel value. Calculate the deviation value of each pixel value to obtain the total deviation value. When the total deviation value is determined to be greater than or equal to a preset deviation value, a percentage failure result is generated; When the total deviation value is less than the preset deviation value, a verification pass result is generated; When the collection location is determined to be a work area, the work part of the corresponding work area is retrieved, and the outline of the person in the corresponding display frame is processed by part association based on the work part to obtain the associated outline corresponding to the work area. The associated outline is then verified to obtain the verification result corresponding to the work area. Based on the time sequence, the monitoring frames that have not undergone partition compression and the display frames are combined to obtain the transmission streaming media, which is then sent to the monitoring terminal.

2. The method according to claim 1, characterized in that, The step of performing partitioned compression processing on the encoded frame according to the partitioning strategy and the transmission bandwidth control coding model to obtain the display frame includes: Identify the outlines of people and devices in the encoded frame, and perform customized cropping of the encoded frame based on the positional relationship between the outlines of people and devices to obtain the main body area and background area in the encoded frame; The main body region and the background region are partitioned and compressed according to the partitioning strategy and the transmission bandwidth control coding model to obtain the display frame.

3. The method according to claim 2, characterized in that, The identification of the person's silhouette and the device's silhouette in the encoded frame involves customized cropping of the encoded frame based on the positional relationship between the person's silhouette and the device's silhouette, resulting in the main body region and background region of the encoded frame, including: Identify the outlines of people and devices in the encoded frames and perform coordinate processing on the encoded frames; When it is determined that the outline of the person and the outline of the device intersect, the main body region in the encoded frame is determined based on the coordinate extreme values ​​of the corresponding outline of the person. When it is determined that the figure outline and the device outline do not intersect, the figure region corresponding to the figure outline is taken as the main figure region in the encoded frame; Based on the selected main area and the main character area, the main area in the encoded frame is obtained, and the remaining areas in the encoded frame are used as the background area.

4. The method according to claim 2, characterized in that, The step of performing partitioned compression processing on the main body region and the background region according to the partitioning processing strategy and the transmission bandwidth control coding model to obtain the display frame includes: The main body region and the background region are partitioned and compressed according to the partitioning processing strategy control coding model to obtain an initial compressed frame and the initial bitrate of the initial compressed frame. When the initial bit rate is determined to be less than or equal to the transmission bandwidth, the initial compressed frame is used as the display frame; When the initial bit rate is determined to be greater than the transmission bandwidth, the encoding model is controlled according to the main color block strategy to perform partition coarsening processing on the main body region and the background region to obtain an intermediate compressed frame, and the intermediate bit rate of the intermediate compressed frame is obtained. When the intermediate bit rate is determined to be less than or equal to the transmission bandwidth, the intermediate compressed frame is used as the display frame; When the intermediate bit rate is determined to be greater than the transmission bandwidth, the encoding model is controlled according to the contour preservation strategy to perform partitioning and stripping processing on the main body region and the background region, and the final compressed frame is used as the display frame.

5. The method according to claim 4, characterized in that, The step of controlling the encoding model according to the partitioning processing strategy to perform partitioned compression processing on the main region and the background region to obtain an initial compressed frame includes: Retrieve the preset display area and preset display bitrate corresponding to the preset display area, obtain the main area area of ​​each main area, and obtain the main body proportion based on the ratio of the main body area to the preset display area; The initial subject bitrate of each subject area is obtained by multiplying the subject proportion and the preset display bitrate. Retrieve the preset background area and preset background bitrate corresponding to the preset background area, obtain the background area of ​​the background area, and obtain the background ratio based on the ratio of the background area to the preset background area; The initial background bitrate of the background region is obtained by multiplying the background ratio and the preset background bitrate. The main body region is compressed according to the initial main body bitrate control coding model, and the background region is compressed according to the initial background bitrate control coding model to obtain the initial compressed frame.

6. The method according to claim 4, characterized in that, When the initial bit rate is determined to be greater than the transmission bandwidth, the encoding model is used to perform partitioning and coarsening processing on the main body region and the background region according to the main color block strategy to obtain an intermediate compressed frame, including: When the initial bit rate is greater than the transmission bandwidth, the character region corresponding to the character outline in the main body region is obtained, and the associated region is obtained based on the difference between the main body region and the character region. The outline of the character is retrieved, and the character area is segmented according to the outline to obtain the fill area corresponding to each outline. The outline has a corresponding preset compression rate. A weighted average is calculated on the pixel values ​​of the pixels in each of the color-filled regions to obtain the fill pixel value of each color-filled region. Based on the fill pixel value, the coding model is used to perform color-filling processing on the color-filled regions. Obtain preset key points corresponding to the contour parts, and perform horizontal segmentation on the associated areas based on the preset key points to obtain the upper torso segmentation area located in the upper part, the middle torso segmentation area located in the middle part, and the lower torso segmentation area located in the lower part. Based on the upper torso segmentation area, the middle torso segmentation area, and the lower torso segmentation area, the torso segmentation areas corresponding to each direction are obtained; The preset compression ratios of the contour parts adjacent to the torso segmentation area within the main body region are calculated by weighted average to obtain the orientation compression ratio corresponding to the torso segmentation area. Based on the orientation compression ratio control coding model, the corresponding torso segmentation area is subjected to regional compression processing, and the background area is removed to obtain the intermediate compressed frame.

7. The method according to claim 4, characterized in that, When the intermediate bit rate is greater than the transmission bandwidth, the encoding model is controlled according to the contour preservation strategy to perform partitioning and stripping processing on the main body region and the background region, resulting in a final compressed frame as the display frame, including: When the intermediate bit rate is determined to be greater than the transmission bandwidth, the control encoding model strips the background area and retains only the outline of the character in the main area, thus obtaining the final compressed frame as the display frame.

8. The method according to claim 1, characterized in that, When the determined acquisition location is a work area, the corresponding work area is retrieved, and the outline of the person in the corresponding display frame is associated with the work area based on the work area to obtain an associated outline corresponding to the work area. The associated outline is then verified to obtain a verification result corresponding to the work area, including: When the data collection location is determined to be a work area, retrieve the corresponding work area location; Retrieve the encoded frame corresponding to the corresponding display frame as the associated frame, identify the contact detail contour at the working part in the associated frame, and the outline of the operating object that intersects with the contact detail contour; Based on the center point of the associated frame, the associated frame is processed into coordinates, and the coordinates of the contact detail contour and the operation object contour are counted to obtain the contact coordinate set corresponding to the contact detail contour and the object coordinate set corresponding to the operation object contour. The display frame is coordinate-based according to the center point of the display frame. Based on the contact coordinate set, the contour pixels corresponding to the contact detail contour are moved to the figure contour in the corresponding display frame. Based on the object coordinate set, the contour pixels corresponding to the operation object contour are moved to the figure contour in the corresponding display frame, thus obtaining the contour associated with the work area. The coordinates of the area corresponding to the standard work area in the associated frame are counted to obtain the set of standard work coordinates corresponding to the standard work area, and the contour coordinates of the associated contour are counted to obtain the set of associated contour coordinates. When all coordinates in the associated contour coordinate set are within the standard operation coordinate set, a verification result for the operation area is generated.

Images