Camera-based area configuration analysis method and apparatus, electronic device, and medium

Abstract

The application provides a camera-based region configuration analysis method and device, electronic equipment and medium, and is applied to the field of camera AI analysis. When a user draws a region, the system performs intersection and concave polygon detection on the polygon vertices. If the polygons intersect, the system rejects updating. If the polygons are concave polygons, the system constructs their convex hulls to obtain polygons to be detected. The vertex coordinates of the polygons to be detected are stored as normalized coordinates based on a fixed reference size. When rendering, the normalized coordinates are converted into target canvas coordinates for display. The polygons to be detected are encoded into a colon-separated ordered string and delivered to the camera. The camera performs AI intelligent analysis on the collected video frames to obtain AI detection results and simultaneously sets a first timestamp and a second timestamp for the AI detection results. The AI results are enqueued in the cache according to the timestamps that are homologous to the video rendering engine. When rendering the video frames to be rendered, the AI detection results that match the time in the cache are superimposed and displayed with the video frames to be rendered.

Description

Technical Field

[0001] This application relates to the field of camera AI analysis, and in particular to a camera-based regional configuration analysis method, apparatus, electronic device, and medium. Background Technology

[0002] In AI-powered intelligent analysis scenarios for network cameras (such as boundary crossing detection, area intrusion, passenger flow statistics, parking space detection, etc.), users need to draw the detection area on the video screen and send it to the device. The device then performs inference and overlays the detection results back onto the screen. However, in related technologies, area coordinates are mostly stored as absolute pixel values. Switching device resolutions or display modes (fisheye, zoom) can easily lead to area misalignment, resulting in poor display quality. Summary of the Invention

[0003] The purpose of this application is to provide a camera-based region configuration analysis method, apparatus, electronic device, and medium to avoid region misalignment after switching device resolution or display mode. The specific technical solution is as follows: In a first aspect of this application, a camera-based region configuration analysis method is provided, the method comprising: If it is detected that the user is drawing a region on the screen captured by the camera, for the polygon vertices edited by the user, the new edge formed by the edited vertex and the adjacent vertices after the vertex is moved is subjected to intersection detection with each edge that is not adjacent to the new edge. If there is an intersection, the edited vertex is not updated. Concave polygon detection is performed on the polygons formed by the vertices of each polygon. When the polygon is a concave polygon, the convex hull corresponding to the concave polygon is constructed to obtain the polygon to be detected. Based on a fixed reference size, the vertex coordinates of the polygon to be detected are stored as normalized coordinates; during rendering, the normalized coordinates are converted into display coordinates under the target canvas according to the target canvas size for rendering. The polygon to be detected is encoded into a colon-separated ordered string, and then organized into a two-dimensional structure according to the basic configuration and the pan-tilt preset point configuration before being sent to the camera. Based on the received configuration, the camera performs AI intelligent analysis on the acquired video frames and outputs AI detection results for the polygon to be detected; and the AI ​​detection results for the polygon to be detected also carry a first timestamp and a second timestamp, the first timestamp corresponding to the bitstream reference and the second timestamp corresponding to the plug-in hardware reference. The AI ​​detection results are queued and cached according to the target timestamp, which is a timestamp that is from the same source as the current video rendering engine, selected from the first timestamp and the second timestamp based on the current video rendering engine type. When rendering a video frame to be rendered, a query is triggered by the timestamp of the video frame to be rendered, and the AI ​​detection results that match the time in the cache are retrieved and superimposed on the video frame to be rendered. The video frame to be rendered is a video frame of the video containing the polygon to be detected, captured by the camera.

[0004] In one possible implementation, encoding the polygon to be detected into a colon-separated ordered string, organizing it according to a two-dimensional structure of the basic configuration and the pan-tilt preset point configuration, and then sending it to the camera includes: The identifier, type, and normalized vertex coordinates of the polygon to be detected are concatenated into a sequence string by separating them with colons according to a fixed field order, and the key name is omitted in the sequence string; Construct a one-dimensional basic configuration array and a two-dimensional preset point configuration array. The one-dimensional basic configuration array is used to store the sequence string corresponding to the detection area that is not associated with any gimbal preset point. The rows of the two-dimensional preset point configuration array correspond to different gimbal preset points, and the columns correspond to different area numbers. It is used to store the sequence string corresponding to each detection area under different gimbal preset points. Unconfigured positions are filled with preset placeholder strings. The one-dimensional basic configuration array and the two-dimensional preset point configuration array are merged into a two-dimensional structure of [basic configuration, preset point 1, preset point 2, …, preset point N], and sent to the camera along with the enable bits of each detection area.

[0005] In one possible implementation, before storing the vertex coordinates of the polygon to be detected as normalized coordinates based on a fixed reference size, the method further includes: Using the longest side of the polygon to be detected as a reference, rotate the text label of the polygon to be detected until the text label is parallel to the longest side of the polygon to be detected; The inner and outer sides of the text label relative to the polygon to be detected are determined by ray casting. Based on the inner and outer results, the offset direction and offset distance of the text label are determined so that the text label does not obscure the polygon to be detected.

[0006] In one possible implementation, the step of overlaying the time-matched AI detection result retrieved from the cache with the video frame to be rendered includes: Based on the timestamp corresponding to the video frame to be rendered, the AI ​​detection result that is closest to the timestamp corresponding to the video frame to be rendered within the preset symmetric tolerance window is determined as the target AI detection result. The target AI detection results are overlaid on the video frames to be rendered.

[0007] In one possible implementation, before overlaying the target AI detection result onto the video frame to be rendered, the method further includes: Within the preset symmetrical tolerance window, if no AI detection result with the closest timestamp to the video frame to be rendered is identified, the AI ​​detection result in the cache that has an earlier timestamp than the video frame to be rendered and is closest in time will be used as the target AI detection result.

[0008] In one possible implementation, the method further includes: Eliminate AI detection results whose timestamps are earlier than the target AI detection result.

[0009] In one possible implementation, the preset symmetric tolerance window is ±200ms.

[0010] In a second aspect of this application, a camera-based area configuration analysis device is provided, the device comprising: The line segment intersection detection module is used to detect the intersection of the new edge formed by the user and the adjacent vertices after the edited vertex is moved with the polygon vertex in the image captured by the camera. If the new edge is not adjacent to the new edge, the edited vertex is not updated. The convex hull construction module is used to perform concave polygon detection on the polygons formed by the vertices of each polygon. When the polygon is a concave polygon, the convex hull corresponding to the concave polygon is constructed to obtain the polygon to be detected. The coordinate normalization module is used to store the vertex coordinates of the polygon to be detected as normalized coordinates based on a fixed reference size; during rendering, the normalized coordinates are converted into display coordinates under the target canvas according to the target canvas size for rendering. The character encoding module is used to encode the polygon to be detected into a colon-separated ordered string, and then send it to the camera after organizing it into a two-dimensional structure according to the basic configuration and the preset point configuration of the pan-tilt unit. The timestamp setting module is used by the camera to perform AI intelligent analysis on the acquired video frames based on the received configuration, and output the AI ​​detection result for the polygon to be detected; and the AI ​​detection result of the polygon to be detected carries a first timestamp and a second timestamp, the first timestamp corresponding to the bitstream reference, and the second timestamp corresponding to the plug-in hardware reference. The timestamp matching module is used to queue and cache the AI ​​detection results according to the target timestamp, which is a timestamp that is from the same source as the current video rendering engine, selected from the first timestamp and the second timestamp according to the type of the current video rendering engine. When rendering the video frame to be rendered, the timestamp of the video frame to be rendered triggers a query, and the time-matched AI detection result is retrieved from the cache and superimposed on the video frame to be rendered. The video frame to be rendered is a video frame of the video containing the polygon to be detected, captured by the camera.

[0011] In one possible implementation, the character encoding module includes: The character concatenation submodule is specifically used to concatenate the identifier, type, and normalized vertex coordinates of the polygon to be detected into a fixed-order string separated by colons according to a fixed field order, wherein the key name is omitted from the fixed-order string; The configuration array construction submodule is specifically used to construct a one-dimensional basic configuration array and a two-dimensional preset point configuration array. The one-dimensional basic configuration array is used to store the sequence string corresponding to the detection area that is not associated with any PTZ preset point. The rows of the two-dimensional preset point configuration array correspond to different PTZ preset points, and the columns correspond to different area numbers. It is used to store the sequence string corresponding to each detection area under different PTZ preset points. Unconfigured positions are filled with preset placeholder strings. The array merging submodule is specifically used to merge the one-dimensional basic configuration array and the two-dimensional preset point configuration array into a two-dimensional structure of [basic configuration, preset point 1, preset point 2, …, preset point N], and send it to the camera along with the enable bits of each detection area.

[0012] In one possible implementation, the device further includes: A text label rotation module is used to rotate the text label of the polygon to be detected, based on the longest side of the polygon to be detected, until the text label is parallel to the longest side of the polygon to be detected. The inner / outer side determination module is used to determine the inner / outer side of the text label relative to the polygon to be detected using the ray casting method; The text label offset module is used to determine the offset direction and offset distance of the text label based on the inner and outer results, so that the text label does not obscure the polygon to be detected.

[0013] In one possible implementation, when the time-matched AI detection result is retrieved from the cache and overlaid on the video frame to be rendered, the timestamp matching module includes: The first sub-module for AI detection result query is specifically used to query based on the timestamp corresponding to the video frame to be rendered, and determine the AI ​​detection result that is closest to the timestamp corresponding to the video frame to be rendered within a preset symmetrical tolerance window as the target AI detection result. The overlay display submodule is specifically used to overlay the target AI detection results with the video frames to be rendered.

[0014] In one possible implementation, the device further includes: The second sub-module for querying AI detection results is specifically used to, within the preset symmetrical tolerance window, if no AI detection result with the closest timestamp to the video frame to be rendered is identified, use the AI ​​detection result in the cache that is earlier than the timestamp to the video frame to be rendered and has the closest time as the target AI detection result.

[0015] In one possible implementation, the device further includes: The AI ​​detection result elimination module is used to eliminate AI detection results whose timestamps are earlier than the target AI detection result.

[0016] In one possible implementation, the preset symmetric tolerance window is ±200ms.

[0017] In a third aspect of this application, an electronic device is provided, comprising: Memory, used to store computer programs; When a processor executes a program stored in memory, it implements the method described in the first aspect of the embodiments of this application.

[0018] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein a computer program is stored therein, and the computer program, when executed by a processor, implements the method described in the first aspect of the present application.

[0019] Compared with the prior art, the embodiments of this application have at least the following technical effects: This application provides a camera-based region configuration analysis method, apparatus, electronic device, and medium. When it is detected that a user is drawing a region on the image captured by the camera, for the user-edited polygon vertices, the new edges formed by the moved edited vertices and adjacent vertices are subjected to intersection detection with edges not adjacent to the new edges. If an intersection is found, the edited vertex is not updated. Concave polygon detection is performed on the polygons formed by the polygon vertices. If a concave polygon is found, the corresponding convex hull is constructed to obtain the polygon to be detected. Based on a fixed reference size, the vertex coordinates of the polygon to be detected are stored as normalized coordinates. During rendering, the normalized coordinates are converted to display coordinates under the target canvas size for rendering. The polygon to be detected is encoded as a colon-separated ordered string and configured according to the basic configuration and the preset points of the pan-tilt unit. After the two-dimensional structure is organized, it is sent to the camera; based on the received configuration, the camera performs AI intelligent analysis on the acquired video frames and outputs AI detection results for the polygon to be detected; the AI ​​detection results for the polygon to be detected also carry a first timestamp and a second timestamp, the first timestamp corresponding to the bitstream reference and the second timestamp corresponding to the plug-in hardware reference; the AI ​​detection results are queued and cached according to the target timestamp, the target timestamp being a timestamp from the first timestamp and the second timestamp that is of the same origin as the current video rendering engine type; when rendering the video frame to be rendered, the timestamp of the video frame to be rendered triggers a query, and the time-matching AI detection result is retrieved from the cache and superimposed on the video frame to be rendered, the video frame to be rendered being the video frame of the video containing the polygon to be detected acquired by the camera.

[0020] The method described in this application normalizes the vertex coordinates of the polygon to be detected by fixing a reference size. This decouples the normalized coordinates from the pixels and the canvas, making the normalized coordinates applicable to different sizes, resolutions, and canvases, ensuring pixel clarity and avoiding pixel misalignment. By encoding the polygon to be detected as a colon-separated ordered string, compact encoding is achieved, reducing system cache pressure. By simultaneously setting dual-reference timestamps for the AI ​​detection results, timestamps from the same source can be selected based on the type of the current video rendering engine, ensuring that the same set of cache and synchronization logic can be aligned with video frames under both heterogeneous rendering engines.

[0021] Of course, implementing any method of the embodiments of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0023] Figure 1 A flowchart of a camera-based region configuration analysis method provided in an embodiment of this application; Figure 2 A schematic diagram of a camera-based area configuration analysis device provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0025] In AI-powered intelligent analysis scenarios for network cameras (such as boundary crossing detection, area intrusion detection, passenger flow statistics, parking space detection, etc.), users need to draw detection areas on the video screen and send them to the device. The device then performs inference and overlays the detection results back onto the screen. During the front-end implementation of this process, the applicant discovered deficiencies in three key areas: area drawing, configuration encoding distribution, and result overlay synchronization. Step 1 (Region Drawing): 1. The legality (convexity, self-intersection) of the detection area and warning line is mostly verified by the backend after drawing. Users cannot know the illegal shape in time, resulting in poor interactive feedback.

[0026] 2. Region coordinates are mostly stored as absolute pixel values, which can easily lead to region misalignment after switching device resolution or display mode (fisheye, zoom).

[0027] 3. The text in the area labels is drawn horizontally in a fixed manner, which makes them easy to be obscured and unreadable in slanted or irregular areas.

[0028] Step Two (Configuration Code Distribution): 4. Embedded cameras have limited configuration interface bandwidth and parsing capabilities. Using structured text such as JSON to transmit area configurations results in high redundancy and high parsing overhead.

[0029] 5. The detection area of ​​the gimbal camera needs to switch different configurations depending on the preset point of the gimbal. In related technologies, the complete configuration is stored independently for each preset point, which is redundant and not conducive to reuse.

[0030] Step 3 (Result Overlay and Synchronization) 6. The AI ​​detection results and video frames belong to two independent real-time data channels, each carrying a timestamp from the camera's hardware clock (microsecond level, subject to jitter). However, in scenarios with two real-time streams, jittery hardware clocks, and an uncontrollable arrival order, the jitter from both channels can cause the detection box to misalign with the moving target in the frame, resulting in shaking and inability to keep up.

[0031] To address at least one of the aforementioned problems, a first aspect of this application provides a camera-based region configuration analysis method that can run on a browser client of a camera AI configuration / preview page. The user draws detection regions on the screen for different pan-tilt-zoom (PTZ) preset points, and the system applies geometric constraints in real time and sends them to the device in a compact coded two-dimensional structure. During real-time preview / playback, the system automatically overlays the most matching device detection results according to the video frame timestamps. The method includes, as follows: Figure 1 The steps shown are as follows: Step S101: When it is detected that the user is drawing a region on the screen captured by the camera, for the polygon vertices edited by the user, the new edge formed by the edited vertex and the adjacent vertex after the vertex is moved is intersected with each edge that is not adjacent to the new edge. If there is an intersection, the edited vertex is not updated.

[0032] In practical applications, user-generated region drawing on camera-captured images can be used in scenarios such as boundary crossing detection, area intrusion, passenger flow statistics, and parking space detection. The vertices of the polygon edited by the user can be vertices selected directly from the camera's image, or vertices identified based on the user-drawn shape. Moving the edited vertex can refer to adding a new vertex or moving an existing vertex, thus creating new edges between the edited vertex and its adjacent vertices.

[0033] For each newly generated edge, an intersection check is performed between it and all edges that are not adjacent to it. In one possible implementation, the intersection check can be performed using a fast rejection test (rectangular bounding box pre-screening) and a cross-product test (cross product judgment). If an intersection is found, the edited vertices are not updated, thus ensuring that the generated polygon does not have self-intersections.

[0034] Step S102: Perform concave polygon detection on the polygon formed by each polygon vertex. When it is a concave polygon, construct the convex hull corresponding to the concave polygon to obtain the polygon to be detected.

[0035] When detecting concave polygons, we can traverse each vertex of the polygon, take its preceding and following edges to form two vectors, and calculate their two-dimensional cross product. We then determine whether the signs of the cross products are consistent. If they are inconsistent, the polygon is considered concave. If they are consistent, the polygon is convex, and this convex polygon is identified as the polygon to be detected.

[0036] For concave polygons, a convex hull can be constructed using Graham scan, and the convex hull can be identified as the polygon to be detected.

[0037] Step S103: Based on a fixed reference size, store the vertex coordinates of the polygon to be detected as normalized coordinates; during rendering, convert the normalized coordinates into display coordinates under the target canvas according to the target canvas size for rendering.

[0038] This fixed reference size can be set according to actual business needs. In one example, the fixed reference size is set to 1024. When normalizing the vertex coordinates of the polygon to be detected based on the fixed reference size, you can first read the displayed coordinates of each vertex under the current canvas, and then calculate the fixed reference size / the displayed coordinates of each vertex under the current canvas to obtain the normalized coordinates. In one example, assuming the displayed coordinates of a vertex under the current canvas are (x0, y0), its normalized coordinates are (1024 / x0, 1024 / y0).

[0039] During rendering, if the canvas remains the current canvas, the normalized coordinates can be converted back to the display coordinates under the current canvas according to a fixed reference size. If the canvas size changes, the display coordinates of each vertex under the current canvas are converted to the display coordinates under the target canvas based on the sizes of the target canvas and the current canvas before rendering.

[0040] Step S104: Encode the polygon to be detected into a colon-separated ordered string, organize it into a two-dimensional structure according to the basic configuration and the pan-tilt preset point configuration, and then send it to the camera.

[0041] In one possible implementation, this step can be achieved through the following steps: Step 1: Concatenate the identifier, type, and normalized vertex coordinates of the polygon to be detected into a fixed string, separated by colons, according to a fixed field order.

[0042] In this context, the key name is omitted from the ordered string. The area to be detected can be divided into multiple polygons. For example, in a monitoring scenario, each polygon can represent a different sub-area or a monitored area from a different angle within the area to be detected. The identifier of each polygon can be its ID (identification) or number, or other information that identifies it. The type of polygon can correspond to the type of the actual business scenario, such as a lobby, corridor, or entrance.

[0043] In one example, the above information of the polygon to be detected can be concatenated in the order of "id:type:startX:startY:stopX:stopY:" to obtain a sequence string. Here, id represents the identifier of the polygon to be detected; type represents the type of the polygon to be detected; startX represents the X-axis coordinate of the starting vertex of the polygon to be detected; startY represents the Y-axis coordinate of the starting vertex of the polygon to be detected; stopX represents the X-axis coordinate of the ending vertex of the polygon to be detected; and stopY represents the Y-axis coordinate of the ending vertex of the polygon to be detected. Since all positions (including unconfigured bits) are aligned with a fixed field structure, the parser can parse the fields according to a fixed offset, without the need for keys, achieving a compact and parsable message. Therefore, the sequence string in this application can be separated only by colons and does not need to include keys.

[0044] Step 2: Construct a one-dimensional basic configuration array and a two-dimensional preset point configuration array. The one-dimensional basic configuration array is used to store the sequence strings corresponding to the detection areas that are not associated with any PTZ preset points. The rows of the two-dimensional preset point configuration array correspond to different PTZ preset points, and the columns correspond to different area numbers. It is used to store the sequence strings corresponding to each detection area under different PTZ preset points. Unconfigured positions are filled with preset placeholder strings.

[0045] Preset points for the PTZ camera refer to pre-defined camera angles / positions (such as looking down into the lobby or facing the entrance / exit). When configuring detection areas, the coordinates of the same physical area will be completely different from different preset point perspectives, thus requiring different storage logic. If a detection area is not associated with any PTZ preset point, it can be considered a "basic configuration" (not associated with any preset point). The information of this area does not depend on the camera perspective. Writing it into a one-dimensional basic configuration array according to the area number facilitates the reuse of basic configurations later.

[0046] If the region is associated with a PTZ preset point, then the [preset point number][region number] is written into the two-dimensional preset point configuration array to achieve independent configuration and reuse of the same physical region under different preset points. For unconfigured positions, a preset placeholder string can be pre-set according to actual business needs, for example, the preset placeholder string is -1.

[0047] Step 3: Merge the one-dimensional basic configuration array and the two-dimensional preset point configuration array into a two-dimensional structure of [basic configuration, preset point 1, preset point 2, …, preset point N], and send it to the camera along with the enable bits of each detection area.

[0048] Step S105: Based on the received configuration, the camera performs AI intelligent analysis on the acquired video frames and outputs the AI ​​detection result for the polygon to be detected; and the AI ​​detection result for the polygon to be detected carries a first timestamp and a second timestamp, the first timestamp corresponding to the bitstream reference and the second timestamp corresponding to the plug-in hardware reference.

[0049] When performing AI intelligent analysis on the captured video frames, the AI ​​intelligent analysis can be performed through the intelligent analysis module embedded in the camera, or the camera can send the video frames to the cloud or server, where the cloud or server will perform AI intelligent analysis and return the AI ​​detection results to the camera.

[0050] This AI intelligent analysis is an intelligent analysis that corresponds to actual business needs. For example, if the actual business scenario is a train station, then the AI ​​intelligent analysis can be passenger flow analysis; if the actual business scenario is an examination scenario, then the AI ​​intelligent analysis can be cheating detection.

[0051] For AI detection results, both a first and a second timestamp are set. The first timestamp corresponds to the bitstream baseline, and the second timestamp corresponds to the plugin hardware baseline. In plugin-free mode, when decoding is performed by the browser's MSE (Media Source Extensions) / WASM (WebAssembly), the first timestamp corresponding to the bitstream baseline needs to be selected. In native plugin mode, when decoding is performed internally by the plugin, the second timestamp corresponding to the plugin hardware baseline needs to be selected.

[0052] Step S106: Enqueue and cache the AI ​​detection results according to the target timestamp. The target timestamp is a timestamp that is from the first timestamp and the second timestamp and is of the same origin as the current video rendering engine. When rendering the video frame to be rendered, the query is triggered by the timestamp of the video frame to be rendered. The AI ​​detection results that match the time in the cache are retrieved and superimposed on the video frame to be rendered. The video frame to be rendered is a video frame of the video containing the polygon to be detected captured by the camera.

[0053] The AI ​​detection results can be queued and cached according to the target timestamp. When a query is triggered based on the timestamp of the video frame to be rendered, the time-matching AI detection results are retrieved from the queue cache and overlaid on the video frame to be rendered for display.

[0054] In one example, when determining the target timestamp, you can first determine whether the current video rendering engine uses a non-plugin (browser MSE / WASM decoding) bitstream base path. If so, the first timestamp is used as the target timestamp; otherwise, the second timestamp is used as the target timestamp.

[0055] The target timestamp is selected as the cache key based on the rendering engine to ensure that the enqueue key is comparable to the timestamp of the video frame that triggers the query in the future. The selected target timestamp is used as the enqueue key. Multiple packets arrive under the same reference timestamp (such as the detection box and the EPTZ (Electronic Pan-Tilt-Zoom) trajectory belonging to different packets), and are merged by type through appendData (append data module).

[0056] When overlaying the time-matched AI detection results retrieved from the cache onto the video frame to be rendered, the following steps can be used for overlay display: Step 1: Query based on the timestamp corresponding to the video frame to be rendered, and determine the AI ​​detection result that is closest to the timestamp corresponding to the video frame to be rendered within the preset symmetric tolerance window as the target AI detection result.

[0057] Step 2: Overlay the target AI detection results with the video frames to be rendered.

[0058] The preset symmetric tolerance window is a timestamp interpolation range set according to actual application requirements. For example, if the preset symmetric tolerance window is ±200ms, then the AI ​​detection result within the preset symmetric tolerance window is the AI ​​detection result within the range of 200ms before to 200ms after the timestamp of the video frame to be rendered.

[0059] The AI ​​detection result that is closest to the timestamp of the video frame to be rendered within the preset symmetric tolerance window can be the AI ​​detection result that has the smallest absolute interpolation value between the timestamp and the timestamp of the video frame to be rendered within the preset symmetric tolerance window.

[0060] In one possible implementation, within a preset symmetrical tolerance window, if no AI detection result with the closest timestamp to the video frame to be rendered is identified, the AI ​​detection result in the cache that has an earlier timestamp than the video frame to be rendered and is closest in time is taken as the target AI detection result.

[0061] If there is no corresponding AI detection result within the preset symmetrical tolerance window, the AI ​​detection result whose timestamp is closest to the timestamp of the video frame to be rendered is determined from the AI ​​detection results stored in the queue cache and used as the target AI detection result.

[0062] After determining the target AI detection result, AI detection results with timestamps earlier than the corresponding timestamps of the target AI detection result are eliminated, thereby reducing cache pressure.

[0063] The method described in this application normalizes the vertex coordinates of the polygon to be detected by fixing a reference size. This decouples the normalized coordinates from the pixels and the canvas, making the normalized coordinates applicable to different sizes, resolutions, and canvases, ensuring pixel clarity and avoiding pixel misalignment. By encoding the polygon to be detected as a colon-separated ordered string, compact encoding is achieved, reducing system cache pressure. By simultaneously setting dual-reference timestamps for the AI ​​detection results, timestamps from the same source can be selected based on the type of the current video rendering engine, ensuring that the same set of cache and synchronization logic can be aligned with video frames under both heterogeneous rendering engines.

[0064] In one possible implementation, the user may need to annotate the area to be detected, thereby generating corresponding text labels. The method of this embodiment may further include the following steps: Step A: Using the longest side of the polygon to be detected as a reference, rotate the text label of the polygon to be detected until the text label is parallel to the longest side of the polygon to be detected.

[0065] Step B: Determine the inner and outer sides of the text label relative to the polygon to be detected using the ray casting method.

[0066] Step C: Based on the inner and outer results, determine the offset direction and offset distance of the text label so that the text label does not obscure the polygon to be detected.

[0067] In one example, we can first iterate through each side of the polygon to be detected, compare the side lengths by the sum of the squares of the differences between the coordinates of the two endpoints, and take the longest side as the label reference side. Then, we calculate the slope ratio based on the difference in coordinates between the two endpoints of the longest side, determine the text rotation angle according to the slope ratio, and perform quadrant correction according to the sign of the difference in coordinates between the two endpoints (for example, when dy>0 and dx<0, angle = 180). Angle, where dy is the y-value of the coordinate difference, dx is the x-value of the coordinate difference, and angle is the text rotation angle, makes the label parallel to the label's reference edge.

[0068] Then, a ray is emitted from the label anchor point along the coordinate axis, and the number of intersections between the ray and each side of the polygon to be detected (skipping the side where the anchor point is located) is counted; the anchor point is determined to be inside or outside the polygon according to the odd-even rule of the ray method - if the number of intersections is odd, it is inside, and if the number is even, it is outside.

[0069] Based on the inner and outer results, the offset direction and offset distance of the text label are determined so that the label falls on one side of the detection area without obstructing it and is always readable in the positive direction. In practical applications, the font size in the text label can be adaptively determined according to the canvas width. For example, based on "ontSize = 12 + parseInt((clientWidth 456) / 120) (upper limit 24)" determines the font size in the question tag, where ontSize represents the font size in the text tag, parseInt is the rounding function, and clientWidth represents the width of the browser window, thus ensuring that the tag is clear under different window sizes.

[0070] By applying the method of the embodiments of this application, and by rotating and offsetting the text label, it can be ensured that the text label does not obscure the polygon to be detected and is readable in the forward direction.

[0071] In a second aspect of this application, a camera-based area configuration analysis apparatus is provided, the apparatus comprising: Figure 2 The structure shown is as follows: The line segment intersection detection module 201 is used to detect the intersection of the new edge formed by the user and the adjacent vertices after the edited vertex is moved with the polygon vertex in the image captured by the camera. If there is an intersection, the edited vertex is not updated.

[0072] The convex hull construction module 202 is used to perform concave polygon detection on the polygon formed by each polygon vertex. When it is a concave polygon, the convex hull corresponding to the concave polygon is constructed to obtain the polygon to be detected.

[0073] The coordinate normalization module 203 is used to store the vertex coordinates of the polygon to be detected as normalized coordinates based on a fixed reference size; during rendering, the normalized coordinates are converted into display coordinates under the target canvas according to the target canvas size for rendering.

[0074] The character encoding module 204 is used to encode the polygon to be detected into a colon-separated ordered string, and then send it to the camera after organizing it into a two-dimensional structure according to the basic configuration and the preset point configuration of the pan-tilt unit.

[0075] The timestamp setting module 205 is used by the camera to perform AI intelligent analysis on the acquired video frames based on the received configuration, and output the AI ​​detection result for the polygon to be detected; and the AI ​​detection result of the polygon to be detected carries a first timestamp and a second timestamp at the same time, the first timestamp corresponds to the bitstream reference, and the second timestamp corresponds to the plug-in hardware reference.

[0076] The timestamp matching module 206 is used to queue and cache the AI ​​detection results according to the target timestamp. The target timestamp is a timestamp that is from the same source as the engine, selected from the first timestamp and the second timestamp according to the type of the current video rendering engine. When rendering the video frame to be rendered, the query is triggered by the timestamp of the video frame to be rendered. The time-matched AI detection results are retrieved from the cache and superimposed on the video frame to be rendered. The video frame to be rendered is a video frame of the video containing the polygon to be detected, captured by the camera.

[0077] In one possible implementation, the character encoding module includes: The character concatenation submodule is specifically used to concatenate the identifier, type, and normalized vertex coordinates of the polygon to be detected into a ordered string separated by colons according to a fixed field order. The key name is omitted from the ordered string. The configuration array construction submodule is specifically used to construct a one-dimensional basic configuration array and a two-dimensional preset point configuration array. The one-dimensional basic configuration array is used to store the sequence strings corresponding to the detection areas that are not associated with any PTZ preset points. The rows of the two-dimensional preset point configuration array correspond to different PTZ preset points, and the columns correspond to different area numbers. It is used to store the sequence strings corresponding to each detection area under different PTZ preset points. Unconfigured positions are filled with preset placeholder strings. The array merging submodule is specifically used to merge the one-dimensional basic configuration array and the two-dimensional preset point configuration array into a two-dimensional structure of [basic configuration, preset point 1, preset point 2, …, preset point N], and send it to the camera along with the enable bits of each detection area.

[0078] In one possible implementation, the device further includes: The text label rotation module is used to rotate the text label of the polygon to be detected, based on the longest side of the polygon to be detected, until the text label is parallel to the longest side of the polygon to be detected. The inner / outer side determination module is used to determine the inner / outer side of the text label relative to the polygon to be detected using the ray casting method. The text label offset module is used to determine the offset direction and offset distance of the text label based on the inner and outer results, so that the text label does not obscure the polygon to be detected.

[0079] In one possible implementation, when displaying the time-matched AI detection result retrieved from the cache overlaid with the video frame to be rendered, the timestamp matching module includes: The first submodule of AI detection result query is specifically used to query based on the timestamp corresponding to the video frame to be rendered, and determine the AI ​​detection result that is closest to the timestamp corresponding to the video frame to be rendered within the preset symmetrical tolerance window as the target AI detection result. The overlay display submodule is specifically used to overlay the target AI detection results with the video frames to be rendered.

[0080] In one possible implementation, the device further includes: The second sub-module for querying AI detection results is specifically used to, within a preset symmetrical tolerance window, if no AI detection result with the closest timestamp to the video frame to be rendered is identified, take the AI ​​detection result with the closest timestamp to the video frame to be rendered as the target AI detection result.

[0081] In one possible implementation, the device further includes: The AI ​​detection result elimination module is used to eliminate AI detection results whose timestamps are earlier than the target AI detection result.

[0082] In one possible implementation, the preset symmetric tolerance window is ±200ms.

[0083] In a third aspect of the embodiments of this application, an electronic device is provided, see [link to relevant documentation]. Figure 3 ,include: Memory 301 is used to store computer programs; When processor 302 executes a program stored in memory, it implements the following: If it is detected that the user is drawing a region on the screen captured by the camera, for the polygon vertices edited by the user, the new edge formed by the edited vertex and its adjacent vertices after the vertex is moved is checked for intersection with each edge that is not adjacent to the new edge. If there is an intersection, the edited vertex is not updated. Concave polygon detection is performed on the polygons formed by the vertices of each polygon. When a polygon is concave, the convex hull corresponding to the concave polygon is constructed to obtain the polygon to be detected. Based on a fixed reference size, the vertex coordinates of the polygon to be detected are stored as normalized coordinates; during rendering, the normalized coordinates are converted into display coordinates under the target canvas according to the target canvas size for rendering. The polygon to be detected is encoded into a colon-separated ordered string, and then organized into a two-dimensional structure according to the basic configuration and the preset point configuration of the pan-tilt unit before being sent to the camera. Based on the received configuration, the camera performs AI intelligent analysis on the captured video frames and outputs AI detection results for the polygon to be detected. The AI ​​detection results for the polygon to be detected also carry a first timestamp and a second timestamp. The first timestamp corresponds to the bitstream reference, and the second timestamp corresponds to the plug-in hardware reference. The AI ​​detection results are queued and cached according to the target timestamp, which is a timestamp that is from the same source as the engine, selected from the first timestamp and the second timestamp based on the current video rendering engine type. When rendering the video frame to be rendered, the query is triggered by the timestamp of the video frame to be rendered. The AI ​​detection results that match the time in the cache are retrieved and overlaid on the video frame to be rendered. The video frame to be rendered is a video frame of the video containing the polygon to be detected, captured by the camera.

[0084] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0085] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0086] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0087] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0088] In a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements the method of the first aspect of the embodiments of this application.

[0089] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to implement the method of the first aspect of the embodiments of this application.

[0090] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a solid-state drive (SSD), etc.

[0091] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0092] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. Related parts can be found in the descriptions of the system embodiments.

[0093] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A camera-based region configuration analysis method, characterized in that, The method includes: If it is detected that the user is drawing a region on the screen captured by the camera, for the polygon vertices edited by the user, the new edge formed by the edited vertex and the adjacent vertices after the vertex is moved is subjected to intersection detection with each edge that is not adjacent to the new edge. If there is an intersection, the edited vertex is not updated. Concave polygon detection is performed on the polygons formed by the vertices of each polygon. When the polygon is a concave polygon, the convex hull corresponding to the concave polygon is constructed to obtain the polygon to be detected. Based on a fixed reference size, the vertex coordinates of the polygon to be detected are stored as normalized coordinates; during rendering, the normalized coordinates are converted into display coordinates under the target canvas according to the target canvas size for rendering. The polygon to be detected is encoded into a colon-separated ordered string, and then organized into a two-dimensional structure according to the basic configuration and the pan-tilt preset point configuration before being sent to the camera. Based on the received configuration, the camera performs AI intelligent analysis on the acquired video frames and outputs AI detection results for the polygon to be detected; and the AI ​​detection results for the polygon to be detected also carry a first timestamp and a second timestamp, the first timestamp corresponding to the bitstream reference and the second timestamp corresponding to the plug-in hardware reference. The AI ​​detection results are queued and cached according to the target timestamp, which is a timestamp that is from the same source as the current video rendering engine, selected from the first timestamp and the second timestamp based on the current video rendering engine type. When rendering a video frame to be rendered, a query is triggered by the timestamp of the video frame to be rendered, and the AI ​​detection results that match the time in the cache are retrieved and superimposed on the video frame to be rendered. The video frame to be rendered is a video frame of the video containing the polygon to be detected, captured by the camera.

2. The method according to claim 1, characterized in that, The step of encoding the polygon to be detected into a colon-separated ordered string, organizing it according to the two-dimensional structure of the basic configuration and the pan-tilt preset point configuration, and then sending it to the camera includes: The identifier, type, and normalized vertex coordinates of the polygon to be detected are concatenated into a sequence string by separating them with colons according to a fixed field order, and the key name is omitted in the sequence string; Construct a one-dimensional basic configuration array and a two-dimensional preset point configuration array. The one-dimensional basic configuration array is used to store the sequence string corresponding to the detection area that is not associated with any gimbal preset point. The rows of the two-dimensional preset point configuration array correspond to different gimbal preset points, and the columns correspond to different area numbers. It is used to store the sequence string corresponding to each detection area under different gimbal preset points. Unconfigured positions are filled with preset placeholder strings. The one-dimensional basic configuration array and the two-dimensional preset point configuration array are merged into a two-dimensional structure of [basic configuration, preset point 1, preset point 2, …, preset point N], and sent to the camera along with the enable bits of each detection area.

3. The method according to claim 1, characterized in that, Before storing the vertex coordinates of the polygon to be detected as normalized coordinates based on a fixed reference size, the method further includes: Using the longest side of the polygon to be detected as a reference, rotate the text label of the polygon to be detected until the text label is parallel to the longest side of the polygon to be detected; The inner and outer sides of the text label relative to the polygon to be detected are determined by ray casting. Based on the inner and outer results, the offset direction and offset distance of the text label are determined so that the text label does not obscure the polygon to be detected.

4. The method according to claim 1, characterized in that, The process of retrieving time-matched AI detection results from the cache and overlaying them onto the video frames to be rendered includes: Based on the timestamp corresponding to the video frame to be rendered, the AI ​​detection result that is closest to the timestamp corresponding to the video frame to be rendered within the preset symmetric tolerance window is determined as the target AI detection result. The target AI detection results are overlaid on the video frames to be rendered.

5. The method according to claim 4, characterized in that, Before overlaying the target AI detection result with the video frame to be rendered, the method further includes: Within the preset symmetrical tolerance window, if no AI detection result with the closest timestamp to the video frame to be rendered is identified, the AI ​​detection result in the cache that has an earlier timestamp than the video frame to be rendered and is closest in time will be used as the target AI detection result.

6. The method according to claim 4 or 5, characterized in that, The method further includes: Eliminate AI detection results whose timestamps are earlier than the target AI detection result.

7. The method according to claim 4 or 5, characterized in that, The preset symmetrical tolerance window is ±200ms.

8. A camera-based area configuration analysis device, characterized in that, The device includes: The line segment intersection detection module is used to detect the intersection of the new edge formed by the user and the adjacent vertices after the edited vertex is moved with the polygon vertex in the image captured by the camera. If the new edge is not adjacent to the new edge, the edited vertex is not updated. The convex hull construction module is used to perform concave polygon detection on the polygons formed by the vertices of each polygon. When the polygon is a concave polygon, the convex hull corresponding to the concave polygon is constructed to obtain the polygon to be detected. The coordinate normalization module is used to store the vertex coordinates of the polygon to be detected as normalized coordinates based on a fixed reference size; during rendering, the normalized coordinates are converted into display coordinates under the target canvas according to the target canvas size for rendering. The character encoding module is used to encode the polygon to be detected into a colon-separated ordered string, and then send it to the camera after organizing it into a two-dimensional structure according to the basic configuration and the preset point configuration of the pan-tilt unit. The timestamp setting module is used by the camera to perform AI intelligent analysis on the acquired video frames based on the received configuration, and output the AI ​​detection result for the polygon to be detected; and the AI ​​detection result of the polygon to be detected carries a first timestamp and a second timestamp, the first timestamp corresponding to the bitstream reference, and the second timestamp corresponding to the plug-in hardware reference. The timestamp matching module is used to queue and cache the AI ​​detection results according to the target timestamp, which is a timestamp that is from the same source as the current video rendering engine, selected from the first timestamp and the second timestamp according to the type of the current video rendering engine. When rendering the video frame to be rendered, the timestamp of the video frame to be rendered triggers a query, and the time-matched AI detection result is retrieved from the cache and superimposed on the video frame to be rendered. The video frame to be rendered is a video frame of the video containing the polygon to be detected, captured by the camera.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains a computer program that, when executed by a processor, implements the method described in claims 1-7.

Images