Region tagging method, information processing method, system, apparatus, device and medium
By performing area calibration in the image acquisition device, determining and correlating the calibration area, the problem of invalid information interference in the image is solved, and the accuracy and efficiency of the analysis process are improved.
Patent Information
- Application Number
- PCT/CN2024/088440
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-31
- Filing Date
- 2024-04-17
- Publication Date
- 2024-12-05
AI Technical Summary
In the prior art, there is invalid information in the images collected by the image acquisition device that is independent of service, resulting in a decrease in the analysis processing efficiency and accuracy.
By performing the area calibration method in the target scene, the calibration area of each image acquisition device is determined and associated with the image acquisition device, only the calibration area is identified and processed, and invalid information is excluded.
It improves the accuracy and efficiency of analysis and processing, reduces the amount of calculation, and avoids interference with the processing of invalid information.
Smart Images

Figure CN2024088440_05122024_PF_FP_ABST
Abstract
Description
Area calibration method, information processing method and system, device, equipment and medium
[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on May 31, 2023, with application number 202310637609.9 and invention name “Region calibration method, information processing method and system, device, equipment and medium”, the entire contents of which are incorporated by reference into this application. Technical Field
[0002] The present disclosure relates to the field of information processing technology, and in particular to a region calibration method, an information processing method and system, a device, equipment, and a medium. Background Art
[0003] With the development of computer technology, image acquisition devices are used to capture images of target scenes. Based on the results of analyzing and processing the captured images, services such as pedestrian flow in the target area and motion tracking of target objects are performed. For example, consider a shopping mall, where multiple cameras are deployed. The images captured by the cameras include shopping guides and customers entering the mall. By identifying these customers, the mall's customer flow can be analyzed.
[0004] In the related art, invalid information irrelevant to the business may exist in the images collected by the image acquisition device, causing interference to the analysis and processing, thereby reducing the efficiency and accuracy of the analysis and processing.
[0005] Summary of the Invention
[0006] In a first aspect, the present disclosure provides a method for region calibration, the method comprising:
[0007] Displaying scene images captured by at least one first image capture device located in the target scene, wherein the first image capture device is used to capture images of a portion or all of the target scene;
[0008] In response to a region drawing operation performed on the scene image, displaying a candidate region selected by the region drawing operation in a simulation image simulating the target scene;
[0009] Determine, based on the candidate areas corresponding to the respective first image acquisition devices, calibration areas corresponding to the plurality of first image acquisition devices; wherein at least two adjacent calibration areas do not overlap;
[0010] The plurality of calibration areas are respectively associated with the plurality of first image acquisition devices to instruct to identify the image areas framed by the calibration areas in the images acquired by the first image acquisition devices.
[0011] A second aspect of the present disclosure provides an information processing method, the method comprising:
[0012] Respectively acquiring scene images captured by a plurality of image acquisition devices of a target scene, and calibration regions corresponding to the plurality of image acquisition devices; wherein the calibration regions are determined according to the region calibration method described in the first aspect;
[0013] Identify the object to be identified in the target area of the scene image to obtain target information; wherein the target area is the area selected by the calibration area;
[0014] For each of the target information, information processing corresponding to a scene task is performed, wherein the scene task is a task corresponding to the target scene.
[0015] A third aspect of the present disclosure provides an information processing system, comprising: a first module, a second module, and a plurality of image acquisition devices located in a target scene; wherein the first module and the second module are both connected to the plurality of image acquisition devices;
[0016] Wherein, the first module is used to execute the area calibration method described in the first aspect;
[0017] The second module is used to execute the information processing method described in the second aspect based on the calibration area corresponding to each of the image acquisition devices.
[0018] A fourth aspect of the present disclosure provides a region calibration device, comprising:
[0019] a display module, configured to display scene images captured by at least one first image capture device located in a target scene, wherein the first image capture device is configured to capture images of a portion or all of the target scene;
[0020] a drawing module, responsive to a region drawing operation performed on the scene image, displaying a candidate region selected by the region drawing operation in a simulation image simulating the target scene;
[0021] a determination module, configured to determine, based on the candidate regions corresponding to the respective first image acquisition devices, calibration regions corresponding to the plurality of first image acquisition devices, wherein at least two adjacent calibration regions do not overlap;
[0022] The associating module associates the plurality of calibration areas with the plurality of first image acquisition devices respectively, so as to instruct to identify the image areas framed by the calibration areas in the images acquired by the first image acquisition devices.
[0023] According to a fifth aspect of the present disclosure, an information processing device is provided, comprising:
[0024] An image acquisition module, configured to respectively acquire scene images captured by a plurality of image acquisition devices of a target scene, and calibration areas corresponding to the plurality of image acquisition devices; wherein the calibration areas are determined by the area calibration method;
[0025] an identification module, configured to identify an object to be identified in a target area of the scene image to obtain target information; wherein the target area is the area framed by the calibration area;
[0026] The information processing module is used to perform information processing corresponding to the scene task on each of the target information, wherein the scene task is a task corresponding to the target scene.
[0027] An embodiment of the present disclosure also discloses an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the computer program implements the area calibration method as described in the first aspect, or the information processing method as described in the embodiment of the second aspect.
[0028] The fourth aspect of the embodiments of the present disclosure further discloses a computer-readable storage medium, which stores a computer program that enables a processor to execute the area calibration method as described in the first aspect, or the information processing method as described in the embodiment of the second aspect of the present disclosure.
[0029] The region calibration method according to the first aspect of the present disclosure can be applied to a target scene. Specifically, scene images captured by each of multiple first image acquisition devices located in the target scene can be displayed, and in response to a region drawing operation performed on the scene image, a candidate region selected by the region drawing operation is displayed in a simulated image simulating the target scene; then, based on the candidate regions corresponding to each of the first image acquisition devices, calibration regions corresponding to each of the multiple first image acquisition devices are determined; wherein, there is no overlap between at least two adjacent calibration regions; thereafter, the multiple calibration regions are respectively associated with the multiple first image acquisition devices to indicate that an object to be identified in the image region framed by the calibration region in the image of the first image acquisition device is to be identified.
[0030] Because a candidate region corresponding to a region drawing operation performed on a scene image captured by the first image acquisition device can be determined based on the region drawing operation, that is, a candidate region on the scene image can be selected. Subsequently, the candidate region is displayed in a simulated image of the target scene, and thus, based on the candidate region displayed in the simulated image, a calibration region corresponding to each first image acquisition device can be determined. After the calibration region is determined, the first image acquisition device can be bound to the calibration region. Thus, when analyzing and processing the image captured by the first image acquisition device, the calibration region in the image can be analyzed and processed, while analyzing and processing objects to be identified in uncalibrated regions of the image can be avoided. Thus, even if invalid information irrelevant to the business is present in the image captured by the image acquisition device, the invalid information can be excluded during analysis based on the calibration region, and only valid information in the image can be processed. This eliminates interference caused by the invalid information in the analysis and processing, improves analysis accuracy, and reduces computational complexity by reducing the processing of invalid information, while also improving analysis and processing efficiency.
[0031] Using the image processing method of the second aspect of the present disclosure, the calibration area corresponding to the image acquisition device in the target scene can be used to identify the object to be identified in the target area framed by the calibration area in the scene image captured by the image acquisition device, thereby obtaining target information, and then performing information processing on the obtained target information. Because each image acquisition device has a corresponding calibration area, when identifying the object to be identified in the image captured by the image acquisition device, it is possible to avoid identifying the object to be identified in invalid areas outside the target area, thereby improving the targetedness of the image processing, avoiding the processing of invalid information, and further reducing the amount of computation and improving the accuracy of the information processing.
[0032] The above description is only an overview of the technical solution of the present disclosure. In order to more clearly understand the technical means of the present disclosure, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present disclosure more obvious and easy to understand, the specific implementation methods of the present disclosure are listed below.
[0033] BRIEF DESCRIPTION OF THE DRAWINGS
[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or related technologies, the following is a brief introduction to the drawings required for the description of the embodiments or related technologies. Obviously, the drawings described below are some embodiments of the present disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without inventive efforts. It should be noted that the scales in the drawings are for illustration only and do not represent the actual scale.
[0035] FIG1 is a schematic diagram showing an application scenario in which the region calibration method and information processing method disclosed herein are applied;
[0036] FIG2 shows a flowchart of the steps of the region calibration method according to an embodiment of the present disclosure;
[0037] FIG3 is a schematic diagram showing a flow chart of the region calibration method according to an embodiment of the present disclosure applied in the implementation environment shown in FIG1 ;
[0038] FIG4 is a schematic diagram showing interface changes before and after performing a region adjustment operation on a candidate region on a simulated image in an embodiment of the present disclosure;
[0039] FIG5a shows a schematic diagram of an interface for adaptively adjusting a candidate region in an embodiment of the present disclosure;
[0040] FIG5b shows another interface diagram for adaptively adjusting the candidate area in an embodiment of the present disclosure.
[0041] FIG6 shows a schematic diagram of a process for determining an installation posture, taking the application scenario of FIG1 as an example, in an embodiment of the present disclosure;
[0042] FIG7 shows a schematic flow chart of steps for determining coordinate conversion parameters in an embodiment of the present disclosure;
[0043] FIG8 is a schematic diagram showing a flow chart of steps for adjusting the calibration area after the position of the image acquisition device changes in an embodiment of the present disclosure;
[0044] FIG9 shows a schematic flow chart of the steps of the information processing method in the second embodiment of the present disclosure;
[0045] FIG10 shows a complete flow diagram of the present disclosure combining the area calibration method and the information processing method, taking the furniture store shown in FIG1 as an example;
[0046] FIG11 shows a schematic structural diagram of a region calibration device according to an embodiment of the present disclosure;
[0047] FIG12 shows a schematic structural diagram of an information processing device according to an embodiment of the present disclosure;
[0048] FIG13 shows a schematic structural diagram of an information processing system in an embodiment of the present disclosure.
[0049] Detailed description
[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.
[0051] In response to the problems in related technologies, some common solutions are: first identify invalid information in the image, for example, through the content correlation between images captured by multiple image acquisition devices, identify invalid information, and then remove the invalid information from the image and process the remaining valid information; this method will consume a large amount of computing power due to the identification of invalid information, and the recognition accuracy is not necessarily reliable, and may not match the analysis business to be targeted.
[0052] In view of this, the present disclosure proposes an improved idea for solving the above-mentioned technical problems: in the target scene, the image captured by the image acquisition device can determine its image acquisition range, and a calibration area is determined within the image acquisition range. Thereafter, the calibration area is associated with the image acquisition device. Thus, the calibration area can be used to indicate the area of the image captured by the image acquisition device that needs to be identified. In this way, invalid information in the image is eliminated from the source, thereby avoiding the identification and elimination of invalid information, reducing the amount of calculation and improving efficiency, and also improving the targeted analysis and processing of valid information, thereby improving the accuracy of the analysis and processing.
[0053] 1 , a schematic diagram of an application scenario in which the area calibration method and information processing method of the present disclosure are applied is shown. As shown in FIG1 , taking a shopping mall as an example of a target scene, a plurality of image acquisition devices 101 are deployed in the shopping mall, wherein the plurality of image acquisition devices 101 may be existing equipment in the shopping mall. For the shopping mall scene, the services that need to be performed include passenger flow statistics, passenger flow tracking, and shopping guide tracking, etc., and different services need to be performed based on images captured by the image acquisition devices.
[0054] FIG1 is an illustration using a merchant's two-dimensional planar engineering drawing as an example.
[0055] In practice, it is not limited to the above scenarios. For example, taking the target scenario as a factory scenario, especially an unmanned factory, multiple image acquisition devices are deployed in the factory, where multiple image acquisition devices can be the factory's original equipment. For the factory, the business that needs to be carried out includes monitoring the movement trajectory of robots in the factory, monitoring the trajectory of shipping vehicles, and statistics on the distribution number of shipping vehicles in multiple production lines, so that the factory's shipping equipment can be reasonably scheduled to adapt to the production capacity of the production line.
[0056] Of course, in addition to the above application scenarios, other application scenarios may also be included in practice. The area calibration method proposed in the present disclosure can be applicable to multiple application scenarios where an image acquisition device needs to be arranged for image acquisition and the acquired images need to be processed and analyzed.
[0057] In one embodiment, the region calibration method disclosed herein is introduced and illustrated in conjunction with FIG1 .
[0058] 2 and 3 , FIG2 shows a flowchart of the steps of the region calibration method in this embodiment, and FIG3 shows a schematic flow chart of the region calibration method in this embodiment applied in the implementation environment shown in FIG1 ; as shown in FIG2 , this region calibration method can be applied to a first electronic device, which can be located in a target scene or not, and can specifically include the following steps:
[0059] Step S201: displaying scene images captured by at least one first image capture device located in a target scene;
[0060] The first image acquisition device is used to acquire images of a part or all of the area in the target scene.
[0061] In this embodiment, the target scene can be a scene constructed by an actual physical area, which can be understood as a physical scene constructed by a building area and the items located within the building area. For example, a furniture store is a furniture sales scene formed by arranging various types of furniture in a house. The target scene includes the building area and the items located within the building area.
[0062] Among them, the first image acquisition device can be an inherent device originally installed in the building area, which can be understood as a device used for other tasks when the target scene is initially built. For example, the first image acquisition device can be installed in a furniture store when it is being decorated. In this way, the existing image acquisition device in the target scene can be reused.
[0063] Among them, the first image acquisition device can be a camera, a dome camera or other imaging equipment, but is not limited to the above equipment. It should be noted that when it is a dome camera, since the dome camera can realize the rotation of the camera, thereby adjusting the image acquisition range, in the embodiment of the present application, the dome camera can be fixed at a certain rotation position, so as to calibrate the calibration area of the dome camera at the fixed position.
[0064] In the target scene, one or more first image acquisition devices may be provided. When one first image acquisition device is provided, the first image acquisition device may capture images of part or all of the target scene, preferably all of the target scene. When multiple first image acquisition devices are provided, different first image acquisition devices may capture images of different areas of the target scene, allowing for overlap in the image acquisition ranges of different first image acquisition devices. Specifically, the image acquisition range of the first image acquisition device may be determined based on the size of the target scene.
[0065] Among them, the first image acquisition device performs image acquisition on the entire area of the target scene, which means that the scene image acquired by the first image acquisition device includes the entire area of the target scene, that is, all areas are located within the acquisition range of the first image acquisition device; similarly, the first image acquisition device performs image acquisition on a partial area of the target scene, which means that a partial area of the target scene is located within the acquisition range of the first image acquisition device.
[0066] Step S202: In response to the region drawing operation performed on the scene image, the candidate region selected by the region drawing operation is displayed in the simulation image of the simulated target scene.
[0067] In this embodiment, the scene image captured by the first image acquisition device can be sent to the first electronic device and displayed in the system installed in the first electronic device. The system can be used to perform area calibration. Specifically, the scene image can be displayed in the first image drawing interface of the system. The first image drawing interface can be an interface for drawing the area of the scene image.
[0068] Among them, tools for area drawing, such as graphic boxes, lines, etc., are provided in the first image drawing interface. Users can use the drawing tools by clicking on the drawing tools or using shortcut keys to draw the area to be selected on the scene image (hereinafter referred to as the drawing area). The area to be selected is an area that does not contain invalid information or contains less invalid information.
[0069] Among them, the area drawing operation can be performed by the user on the first image drawing interface for the scene image. In practice, a layer can be added to the scene image, and the area drawing operation can be performed on this layer. After the drawing is completed, the drawing area drawn on the layer can be associated with the scene image and saved.
[0070] It should be noted that the drawing area drawn by the area drawing operation can be determined according to the object to be identified in the target scene (the object to be identified). If the object to be identified is a human body, the selected area can be the area where the human body is usually located. Taking a furniture store as an example, if the object to be identified is a human body, the selected area usually does not include walls, cabinets and other areas in the target scene.
[0071] Therefore, by providing the user with an image drawing interface, the image captured by the image acquisition device can be manually calibrated to identify areas that do not contain invalid information or contain less invalid information, thereby improving the accuracy of area calibration and providing the user with a visual operation. Compared with the later use of calculation algorithms, which requires setting different algorithms for different situations, requires a large amount of development, and consumes too many computing resources, the use of user-based area drawing operations can provide an area calibration solution that consumes less software and hardware resources and is more time-saving and labor-saving.
[0072] In this embodiment, since the drawing area is based on the user's area drawing operation, its shape can be any shape that adapts to the coverage area, thereby improving the matching degree between the calibration area and the area where the valid information in the target scene is located.
[0073] In response to the region drawing operation, the drawn region can be mapped to a simulated image of the target scene for display, thereby obtaining a candidate region within the simulated image. The simulated image can be a scaled engineering plan view of the target scene. In some examples, it can also be a scaled three-dimensional image of the target scene, without limitation. It should be noted that the simulated image can include a simulated portion of a building area within the target scene and simulated portions of objects arranged within the building area.
[0074] When the simulated image is a three-dimensional stereoscopic image, the drawing area drawn by the area drawing operation needs to be mapped into a three-dimensional space, so that the selected area can be presented more three-dimensionally.
[0075] As shown in Figure 3, taking the scene shown in Figure 1 as an example, the scene image captured by the first image acquisition device 101 in the upper right corner of the target scene is sent to the first electronic device, and the first electronic device displays it on the first drawing interface. On the first drawing interface, the user uses the drawing work to perform an area drawing operation to obtain a drawing area 301, and then maps the drawing area 301 to the simulation map of the target scene to obtain a candidate area 302 on the simulation image.
[0076] It should be noted that a scene image captured by a first image capture device may allow one drawing area to be drawn, or may allow multiple drawing areas to be drawn. For example, in a scene image, two or even three drawing areas may be drawn, and different drawing areas may include different image areas in the scene image.
[0077] Step S203: determining calibration areas corresponding to the plurality of first image acquisition devices based on the candidate areas corresponding to the respective first image acquisition devices;
[0078] There is no overlap between at least two adjacent calibration areas.
[0079] In this embodiment, it is possible to determine whether each candidate area covers a valid area in the target scene that requires information analysis and processing based on the candidate areas corresponding to each of the multiple first image acquisition devices displayed in the simulated image, wherein the valid area may refer to: an area where the recognition result of the object to be recognized is valid. For example, taking the target scene as a furniture store scene, in this scene, the valid area can be determined based on the object to be recognized. If the object to be recognized is the location of a customer entering the store, the determined valid area may refer to: the furniture display area excluding walls, furniture, and stocking areas. In this case, the recognition of the customer's position in the display area is valid, while the recognition results in areas such as walls and furniture are invalid. This can avoid identifying walls and furniture (such as desktops and cabinets) as the customer's location when recognizing the customer's position.
[0080] In the case where multiple candidate areas all cover the valid area in the target scene, each candidate area can be used as a calibration area corresponding to the corresponding first image acquisition device.
[0081] In order to avoid valid information appearing in candidate areas corresponding to different first image acquisition devices, which leads to repeated analysis of the candidate areas, the overlapping areas between the candidate areas can be displayed in the simulated image. Then, the overlapping areas are processed so that there are no overlapping areas between the candidate areas, or at least there are no overlapping areas between the candidate areas that do not need to be repeatedly analyzed, so that there are no overlapping areas between at least two adjacent calibration areas.
[0082] Among them, processing the overlapping area may include: removing the overlapping area in one of the candidate areas while retaining the integrity of the other candidate area; or dividing the overlapping area into two overlapping candidate areas in proportion according to their overlapping shapes, so that the two candidate areas each take part of the overlapping area. For example, taking the overlapping area of candidate area 1 and candidate area 2 as a triangle, the perpendicular line of the triangle can be determined first, and then the triangular area can be divided into two triangular areas according to the perpendicular line. One of the two triangular areas is divided into candidate area 1, and the other triangular area is divided into candidate area 2. In this way, the two candidate areas no longer overlap.
[0083] The overlapping area processing method is adopted so that the processed calibration area is still located in the original candidate area, but there is no overlapping area between the calibration areas. In this way, there will be no repeated valid information, thus avoiding repeated analysis of valid information.
[0084] Step S204: Associating the plurality of calibration areas with the plurality of first image acquisition devices respectively, so as to instruct to recognize the object to be recognized in the image area framed by the calibration area in the image acquired by the first image acquisition device.
[0085] After determining the calibration area corresponding to each first image acquisition device, the calibration area can be associated with the first image acquisition device, wherein the coordinate position of the calibration area in the scene image acquired by the first image acquisition device can be associated with the first image acquisition device, that is, what is associated with the first image acquisition device may be the coordinates of the calibration area in the image coordinate system, and the image coordinate system here may refer to the coordinate system of the first image acquisition device.
[0086] After the calibration area is associated with the first image acquisition device, only the object to be identified in the image area corresponding to the calibration area in the image acquired by the first image acquisition device may be identified.
[0087] As shown in Figure 3, when the first image capture device in the upper right corner captures a scene image during use, it only recognizes the objects to be recognized within the image area selected by calibration area 302, and does not recognize other areas of the image. Assuming that calibration area 302 corresponds to the area corresponding to drawing area 301 (i.e., the candidate area corresponding to drawing area 301 is not adjusted in the simulated image, but is directly determined as the calibration area corresponding to the first image capture device in the upper right corner), then when recognizing the objects to be recognized in the image captured by the first image capture device in the upper right corner, the objects to be recognized in the image are recognized within drawing area 301.
[0088] As shown in Figure 3, the shape of the calibration area is not fixed; it can be drawn into any shape. This allows for accurate screening of areas containing only valid information. The first electronic device can then return the image coordinates of the calibration area in the scene image to the first image acquisition device in the upper right corner.
[0089] Using the region calibration method proposed in this embodiment, a region drawing operation can be performed on a scene image to manually select a drawing region on the scene image. The drawn regions of each first image acquisition device are then mapped onto a simulated image of the target scene, thereby obtaining candidate regions corresponding to each first image acquisition device. Based on the representation of each candidate region on the simulated image, a calibration region corresponding to each first image acquisition device can be determined. Once the calibration region is determined, the calibration region can be bound to the first image acquisition device. Thus, when analyzing and processing images captured by the first image acquisition device, objects to be identified within the image region selected by the calibration region can be analyzed and processed, while objects to be identified within uncalibrated image regions can be analyzed and processed. Even if invalid information irrelevant to the service is present in the image captured by the image acquisition device, the invalid information can be excluded during analysis based on the calibration region, and only valid information in the image can be processed. This eliminates interference caused by the invalid information in the analysis and improves analysis accuracy. By reducing the processing of invalid information, the computational complexity is reduced, while simultaneously improving analysis and processing efficiency.
[0090] In some embodiments, when displaying candidate areas on a simulated image, the candidate areas and the overlapping areas between each candidate area can be displayed. Specifically, the overlapping areas between each candidate area can be determined on the simulated image, and each candidate area and the overlapping areas between each candidate area can be displayed.
[0091] In this embodiment, the simulated image, candidate areas and overlapping areas can be displayed in the second image drawing interface, wherein a display window is provided in the second image drawing interface, and the simulated image, candidate areas and overlapping areas are displayed in the display window, and the simulated image, candidate areas and overlapping areas are all based on the same coordinate system, that is, the coordinate system of the simulated image.
[0092] During display, two layers may be provided, one for displaying the simulated image and the other for displaying the candidate regions and the overlapping regions between them. In yet other embodiments, three layers may be provided, one for displaying the simulated image, another for displaying the candidate regions, and a third for displaying the overlapping regions between the candidate regions. Of course, to facilitate adjustment of the candidate regions in the second image drawing interface, the overlapping regions and the candidate regions may be displayed on the same layer.
[0093] Among them, since the overlapping area is the overlapping part of the candidate area, the overlapping area in the candidate area and the part that does not overlap with other candidate areas can be displayed with different transparencies, for example, the transparency of the overlapping area is lower, so that the user can distinguish the overlapping part and the non-overlapping part in the candidate area.
[0094] In some further embodiments, in addition to displaying the overlapping area, the uncovered area not covered by the candidate area may also be displayed. The uncovered area may include: an area that is not within the acquisition range of the first image acquisition device, and an area that is within the acquisition range of the first image acquisition device but is not covered by the candidate area.
[0095] In this embodiment, in one case, the first image acquisition device in the target scene can capture images of a portion of the target scene. Therefore, the scene images captured by the multiple first image acquisition devices may not fully cover the target scene, and thus the candidate regions may not fully cover the simulated image. In another case, even if the first image acquisition device fully covers the target scene, since the candidate regions are only partial regions of the scene images, each candidate region may not fully cover the target scene, resulting in uncovered regions that are not covered by the candidate regions. Thus, the uncovered regions may include only regions that are not within the capture range of the first image acquisition device, or only regions that are within the capture range of the first image acquisition device but not covered by the candidate regions. Alternatively, when there are multiple uncovered regions, they may include both regions that are not within the capture range of the first image acquisition device and regions that are within the capture range of the first image acquisition device but not covered by the candidate regions.
[0096] Since overlapping areas and uncovered areas can be displayed in the simulated image, multiple candidate areas with overlapping areas can be adjusted. For uncovered areas, a second image acquisition device 102 can be added to the target scene to cover the uncovered areas, thereby achieving full coverage of the target scene.
[0097] In some embodiments, the calibration area can be determined based on the user's adjustment of the candidate area on the second image drawing interface. In other words, the user can be allowed to adjust the candidate area, and the calibration area corresponding to each image acquisition device is determined based on the user's adjustment of the candidate area. This can facilitate on-site construction personnel to accurately divide the calibration area, providing a convenient way for on-site construction personnel to manually operate.
[0098] In yet other embodiments, the calibration region can be determined by an electronic device. Specifically, based on the overlap between candidate regions, the overlapping candidate regions can be adaptively adjusted to obtain calibration regions corresponding to the first image acquisition devices. This approach allows the electronic device to adaptively determine the calibration region based on a pre-stored software program, eliminating the need for manual intervention and reducing worker workload.
[0099] Wherein, in the case where the user is allowed to adjust the candidate regions, the calibration regions corresponding to the plurality of first image acquisition devices may be determined in response to the region adjustment operation on some or all of the plurality of candidate regions;
[0100] When the candidate areas are adaptively adjusted by the electronic device, each candidate area can be adaptively adjusted based on the overlapping area between the candidate areas in the simulated image to obtain the calibration areas corresponding to the multiple first image acquisition devices.
[0101] First, the user's operation of adjusting the candidate area is explained:
[0102] In a specific example of this embodiment, the simulated image and the candidate area on the simulated image can be displayed in the second image drawing interface, and a graphic drawing tool is provided in the second image drawing interface, which provides a tool for adjusting the candidate area in the simulated image.
[0103] As described above, the simulated image and the candidate area can be located in different layers so that the simulated image and the candidate area can be modified separately, wherein the simulated image can be set to a state where modification is not allowed, that is, the layer where the simulated image is located is locked to prohibit modification of the simulated image. Among them, the adjustment tool provided can be used to adjust the shape and size of the candidate area on the layer of the candidate area. The user can adjust the shape and size of the candidate area by selecting the corresponding tool. Specifically, the above-mentioned area adjustment operation can include: based on the selected tool, the adjustment operation on the shape and size of the candidate area is performed on the layer where the candidate area is located.
[0104] After the region adjustment operation is completed, the user may trigger a save operation, and the first electronic device may display the adjusted candidate region on the simulated image in response to the save operation, and may save the adjusted candidate region.
[0105] Of course, in some examples, the candidate region can be adjusted multiple times, wherein the regions obtained after multiple adjustments can be saved so that the user can select an adjusted region as the calibration region. Alternatively, after obtaining the current adjusted region, the current adjusted region can overwrite the previous adjusted region, so that the saved adjusted regions are all the latest adjusted regions.
[0106] For example, reference may be made to FIG4 , which shows a schematic diagram of the interface changes before and after performing an area adjustment operation on a candidate area on a simulated image. As shown in FIG4 , there are multiple tools in the image drawing interface, including at least a tool for adjusting the shape and a tool for adjusting the size. The user can use these tools to adjust the shape and size of the candidate area by clicking on the tools on the interface or by triggering shortcut keys on the keyboard.
[0107] Figure 4 shows the changes in candidate regions 302 and 303 before and after the adjustment. It can be seen that before the adjustment, there was an overlap between candidate regions 301 and 302. After the adjustment, there is no overlap between the two candidate regions. This ensures that the calibration regions contain minimal duplicate information.
[0108] After performing a region adjustment operation on a candidate region, the adjusted candidate region and other candidate regions can be displayed on the second image drawing interface so that the user can observe whether there is overlap between the adjusted candidate regions. If so, the candidate region can be adjusted again.
[0109] In an optional method, after the shape of the candidate area is adjusted, the size of the corresponding candidate area is also changed at the same time. Since the candidate area is generally an area with valid information selected in the scene image, if the shape remains unchanged and the size is simply adjusted, some valid information may be missed after the adjustment. Therefore, in this optional method, the zoom function that changes the size of the candidate area as a whole can be locked, that is, the tool for zooming the area can be locked so that the candidate area will not be reduced as a whole, and by changing the shape, there will be no overlapping areas between the candidate areas.
[0110] Next, the adaptive adjustment of each candidate area by the first electronic device is described:
[0111] The adaptive adjustment of each candidate region may be performed based on the overlapping area between the candidate regions. In one adaptive adjustment, when two candidate regions have overlapping areas, the overlapping area may be subtracted from any of the two overlapping candidate regions, and the remaining area of the candidate region is used as the calibration area.
[0112] The method of subtracting the overlapping area from any of the two candidate areas may be: determining the first position coordinates of the overlapping area in the candidate area to be adjusted and the second position coordinates of the candidate area to be adjusted, and calculating the first position coordinates and the second position coordinates to remove the overlapping area from the candidate area to be adjusted. Alternatively, after determining the position of the overlapping area in the candidate area to be adjusted, the overlapping area may be removed from the candidate area to be adjusted using a cutout technique.
[0113] For example, referring to Figure 5a, a schematic diagram of an interface for adaptively adjusting the candidate area is shown. As shown in Figure 5a, there is an overlapping area between the candidate area 501 and the candidate area 502. The shaded part in Figure 5a is the overlapping area. The overlapping area will be subtracted from the candidate area 501, and the area retained in the candidate area 501 will be used as the calibration area of the first image acquisition device A, while the complete area of the candidate area 502 will be retained.
[0114] Of course, in some other examples, the overlapping area may be subtracted from the candidate area 502 , and the remaining area in the candidate area 502 may be used as the calibration area of the first image acquisition device B, while the complete area of the candidate area 501 is retained.
[0115] In another adaptive adjustment, if there are overlapping areas in multiple candidate areas, the multiple candidate areas can be divided into first-category candidate areas, second-category candidate areas and third-category candidate areas according to the number of overlapping areas between the candidate areas and other candidate areas. The first-category candidate area means that the candidate area under this category overlaps with only one other candidate area; the second-category candidate area means that the candidate area under this category overlaps with two other candidate areas; the third-category candidate area means that the candidate area under this category overlaps with three or more other candidate areas.
[0116] Among them, when performing adaptive adjustment, each type of candidate area can be iteratively adjusted according to the adjustment mode corresponding to each type of candidate area. The iteration here can mean: first adjust the first type of candidate area, and then adjust the second type of candidate area based on the adjustment, and then adjust the third type of candidate area based on the adjusted second type of candidate area, so as to finally adjust all the candidate areas with overlapping areas to obtain the calibration area.
[0117] For example, as shown in Figure 5b, one adjustment method is given, wherein there is overlap between candidate areas 501-505, wherein candidate area 505 and candidate area 504 have overlapping area b1, candidate area 504 and candidate area 501 have overlapping area b2, candidate area 501 and candidate area 503 have overlapping area b3, candidate area 501 and candidate area 502 have overlapping area b4, wherein the first category of candidate areas are candidate areas 505, 502 and 503; the second category of candidate areas is candidate area 504, and the third category of candidate areas is candidate area 501.
[0118] First, the first category of candidate areas, namely candidate areas 505, 502 and 503, are adjusted. Specifically, the overlapping area b1 can be allocated to candidate area 505, the overlapping area b3 can be allocated from candidate area 503, and the overlapping area b4 can be allocated to candidate area 502. In this way, the overlapping area b2 remains, as shown in Figure 5b. The overlapping area b2 can be allocated to candidate area 504. Since the overlapping area b3 is allocated from candidate area 503, when adjusting candidate area 501, the overlapping area b3 can be allocated to candidate area 501, thereby obtaining the adjusted candidate area.
[0119] Of course, in practice, other adaptive adjustment methods can also be used, such as adjusting according to the position order of the candidate areas on the simulated image, such as adjusting from left to right, or adjusting from right to left, which is no longer limited here.
[0120] Among them, the adjustment of the candidate area may not only be limited to the adjustment of the candidate area when there is an overlapping area between the candidate areas, but may also include the adjustment of the shape of the candidate area. In some cases, the shape of the candidate area may be changed according to actual needs. For example, for multiple candidate areas from the same scene image (a scene image is drawn with multiple drawing areas), the user may change the shape of the multiple candidate areas, for example, splicing multiple candidate areas into one area, thereby causing the shape and size of the candidate area to change.
[0121] In some embodiments, the compatibility of the adjusted candidate region with the scene image captured by the first image capture device can be ensured. This compatibility can include ensuring that the adjusted candidate region is within the capture range of the first image capture device and does not deviate from the original candidate region. Specifically, the adjusted candidate region can be returned to the scene image for display, specifically, the adjusted candidate region can be mapped back to the scene image for display, so that a user can verify whether the adjusted candidate region meets the aforementioned compatibility.
[0122] During specific implementation, the adjusted candidate region may be mapped to the scene image from which the adjusted candidate region originates for display.
[0123] Among them, the adjusted candidate area refers to the candidate area on which the area adjustment operation is performed on the simulated image, or the candidate area on which the adaptive adjustment is performed by the first electronic device. As shown in Figure 3, assuming that the candidate area 301 is adjusted in the simulated image, the adjusted area needs to be mapped to the scene image shown in Figure 3 for display, so that the adjusted area can be compared with the original drawn area 301 in the scene image.
[0124] In some specific examples of this embodiment, a drawing area corresponding to the area drawing operation is displayed in the scene image from which the adjusted candidate area originates; specifically, since the scene image can be displayed on the first image drawing interface and the simulated image is displayed on the second image drawing interface, wherein the adjustment of the candidate area occurs on the second image drawing interface, in practice, the adjusted candidate area can be returned to the first image drawing interface, and the first image drawing interface maps the adjusted candidate area on the scene image.
[0125] In order to distinguish the drawing area from the adjusted candidate area mapped on the scene image (hereinafter referred to as the adjusted drawing area), the drawing area and the adjusted drawing area can be displayed on different layers respectively, and both are displayed with a certain transparency, so that the user can determine whether the adjusted drawing area is within the drawing area.
[0126] The drawn area, when mapped onto the simulated image, becomes the candidate area. This allows the drawn area and the adjusted drawn area (corresponding to the adjusted candidate area in the simulated image) to be compared on the scene image. If the adjusted drawn area is within the drawn area, it indicates that the adjustment has not deviated from the original candidate area and is within the original acquisition range. If there is an intersection between the drawn area and the adjusted drawn area, the adjusted drawn area is not completely within the drawn area. If it is not completely within the drawn area, the adjusted drawn area will contain some invalid information, affecting subsequent image analysis and processing. Therefore, the candidate area needs to be adjusted again on the simulated image.
[0127] The adjusted candidate area mapped to the source scene image for display may include: an area adjusted by the user, and a candidate area adaptively adjusted by the first electronic device. In other words, regardless of whether the user adjusts the candidate area or the first electronic device adaptively adjusts the candidate area, the adjusted candidate area can be returned to the scene image for display as the adjusted drawing area.
[0128] Among them, if the adjusted drawing area is not completely within the drawing area, the adjusted drawing area will contain some invalid information. In this case, the candidate area needs to be adjusted again on the simulated image. Accordingly, when a modification operation on the adjusted candidate area in the scene image is received, the area corresponding to the modification operation can be displayed in the simulated image, and the steps of determining the calibration areas corresponding to the multiple first image acquisition devices based on the candidate areas corresponding to the respective first image acquisition devices are repeated.
[0129] That is to say, the adjusted candidate area can continue to be adjusted in the second image drawing interface. After each adjustment, the adjusted candidate area can be returned to the first image drawing interface, so that the adjusted drawing area corresponding to the adjusted candidate area can be displayed in the scene image of the first image drawing interface to view the adjustment effect.
[0130] Accordingly, when the adjusted candidate area is mapped to the scene image from which the adjusted candidate area originates for display, the calibration area can be determined in the following manner: when a determination operation for the adjusted candidate area is received, the adjusted candidate area is used as the calibration area of the corresponding first image acquisition device.
[0131] Among them, the determination operation can occur on the first image drawing interface or on the second image drawing interface. When it occurs on the first image drawing interface, the current adjusted drawing area can be used as the calibration area of the first image acquisition device. Since the scene image is displayed on the first image drawing interface, the coordinates of the adjusted drawing area are the image coordinates. The coordinates of the adjusted drawing area in the scene image can be directly bound to the first image acquisition device.
[0132] When a confirmation operation occurs on the second image drawing interface, the adjusted candidate area can be used as a calibration area, and the coordinates of the calibration area on the simulated image are bound to the corresponding first image acquisition device.
[0133] It should be noted that when adjusting multiple overlapping candidate regions, the adjusted candidate regions become calibration regions. In some embodiments, the calibration regions must not overlap. In other embodiments, the calibration regions must not only not overlap but also be able to be joined together to completely cover the simulated image.
[0134] Accordingly, as mentioned above, in some scenarios, it is not required that multiple calibration areas are spliced together to completely cover the simulated image; however, in some scenarios, it is required that multiple calibration areas are spliced together to completely cover the simulated image. Such scenarios include shopping malls, where it is necessary to count the customer flow in the mall, and therefore it is necessary to identify the guests in the mall as completely as possible without blind spots.
[0135] Accordingly, in order to completely cover the simulated image after multiple calibrated areas are spliced together, the uncovered area can be displayed in the simulated image; in practice, a second image acquisition device can be added to capture images of the uncovered area through the second image acquisition device, thereby achieving full-range image capture of the target scene.
[0136] The uncovered area may include an area not covered by the selected area, or an area not covered by the marked area.
[0137] Specifically, based on the uncovered area, the installation position of the second image acquisition device in the target scene can be determined, and part or all of the uncovered area can be used as the calibration area of the image acquisition device; wherein the second image acquisition device is used to be added to the target scene to capture images of the uncovered area.
[0138] In some scenarios, even if the calibrated areas of multiple first image acquisition devices are spliced together to completely cover the simulated image, as time goes by, if the target scene expands, for example, the store expands from the original 100 square meters to 200 square meters, the original first image acquisition device can only cover the range of 100 square meters, and the newly added area of 100 square meters requires a second image acquisition device to cover the newly added area of 100 square meters.
[0139] In some embodiments, after obtaining candidate regions on a simulated image based on a region drawing operation, the regions not covered by the candidate regions can be displayed on the simulated image. In this case, a second image acquisition device targeting the uncovered regions can be deployed in the target scene, and then the calibration region corresponding to the second image acquisition device can be determined according to the process of steps S201-S203 described above. In this embodiment, on-site workers can ensure that the target scene is fully covered by the image acquisition ranges of the first and second image acquisition devices before dividing the calibration region containing valid information, thereby determining the calibration regions of each image acquisition device at one time. This embodiment can adapt to the above-mentioned situation where the area of the target scene is expanded.
[0140] In some other embodiments, after determining the calibration areas corresponding to multiple first image acquisition devices, the areas not covered by the calibration areas can be displayed on the simulated image. In this case, a second image acquisition device can be directly deployed in the target scene. After determining that the acquisition range of the second image acquisition device includes the range where the calibration areas are located, there is no need to determine the calibration areas of the second image acquisition device, and the uncovered areas can be directly used as the calibration areas of the second image acquisition device. In this embodiment, after the calibration areas are determined, if additional image acquisition devices are needed in the target scene based on image analysis requirements, the operator does not need to perform area drawing operations on the scene images captured by the second image acquisition device. This simplifies the calibration area determination work when adding a second image acquisition device, and can cope with the need to increase the calibration area due to the expansion of the target scene. This embodiment is applicable to situations where the original first image acquisition device of the target scene does not cover the entire area of the target scene, but it is necessary to identify the objects to be identified in the entire area of the target scene.
[0141] Among them, it is necessary to add a second image acquisition device in the target scene to capture scene images of uncovered areas. In practice, it is necessary to determine the installation position of the second image acquisition device in the target scene so that the image acquisition range of the second image acquisition device at this installation position can include uncovered areas.
[0142] The installation posture generally requires on-site workers to continuously adjust it based on the images captured by the second image acquisition device. The installation posture includes the installation position and posture of the second image acquisition device. For example, for a dome camera, it includes not only the position but also the rotation angle of the dome camera.
[0143] In a specific implementation of this embodiment, a target image can be obtained by capturing the target scene by a second image acquisition device at a candidate position; and the scene area included in the target image is mapped to a simulated image to obtain a capture area; then, based on the capture area and the uncovered area, position adjustment information is output; and based on the position adjustment information, the installation posture is determined.
[0144] In some embodiments, the candidate location can be the installation position of the second image acquisition device predicted by the electronic device based on the location of the uncovered area in the simulated image. A manual on-site determination can then be made as to whether the image acquisition range of the second image acquisition device at this installation position can cover the uncovered area. This can reduce the workload of workers on-site in debugging the position of the newly added image acquisition device.
[0145] The output position adjustment information may be a comparison result between the acquisition area and the uncovered area, or a display of the acquisition area and the uncovered area on a simulated image; the output position adjustment information may be directly located in the electronic device, or may be sent to on-site workers so that the on-site workers can view the position adjustment information and adjust the installation posture of the second image acquisition device until it is determined that the acquisition area covers the uncovered area based on the position adjustment information.
[0146] In an example of this specific implementation, the output position adjustment information may be information indicating how to adjust the candidate position according to the comparison result between the acquisition area and the uncovered area.
[0147] In a specific implementation, the coverage type of the acquisition area covering the uncovered area may be determined; based on the coverage type, position adjustment information may be generated; and then, the position adjustment information may be sent to the user terminal.
[0148] In this case, the position adjustment information may include a moving direction or moving distance for the candidate position of the second image acquisition device. The position adjustment information corresponds to the coverage type. In practice, preset adjustment information corresponding to each coverage type can be sent to the user terminal as the position adjustment information. The preset adjustment information can be pre-stored in the electronic device for easy access.
[0149] The coverage type may include three different types: the acquisition area fully covers the uncovered area, the acquisition area partially overlaps with the uncovered area, and the uncovered area fully covers the acquisition area; each type corresponds to a preset adjustment information.
[0150] For example, in the type where the acquisition area fully covers the uncovered area, no position adjustment information is required, and the position adjustment information of 0 can be output, indicating that the current candidate posture is the correct posture; in the type where the acquisition area partially overlaps with the uncovered area, position adjustment is required. In this case, the corresponding preset adjustment information can be determined according to the position of the overlapping area where the acquisition area partially overlaps with the uncovered area on the simulated image. Different positions correspond to different preset adjustment information. In this case, the preset adjustment information may include the moving direction and the moving distance, thereby indicating the direction in which the second image acquisition device performs installation posture debugging; in the type where the uncovered area fully covers the acquisition area, the height of the second image acquisition device needs to be adjusted to expand its image acquisition range. In practice, the preset adjustment information may indicate raising the installation height of the second image acquisition device.
[0151] For example, referring to Figure 6, taking the application scenario of Figure 1 as an example, a flow chart of this embodiment is shown. As shown in Figure 6, the acquisition area 601 partially overlaps with the uncovered area 602, and the overlapping area is toward the lower right of the simulated image compared to the uncovered area 601. In practice, the second image acquisition device 102 needs to move the acquisition range to the upper left. Therefore, the actual installation posture of the second image acquisition device in the target scene can be adjusted accordingly.
[0152] Using the installation posture determination method of this embodiment, the first electronic device can directly predict a candidate location based on the location of the uncovered area on the simulated image. The field worker then places the second image acquisition device at the candidate location. The area covered by the image captured by the second image acquisition device is directly mapped back to the simulated image. The first electronic device can automatically compare the coverage of the mapped area with the uncovered area (including full coverage, partial coverage, etc.), and then output the comparison result and a position adjustment strategy. The position adjustment strategy can be directly fed back to the field worker via communication. After the field worker adjusts the installation posture according to the position adjustment measurement, the above steps are continued until the second image acquisition device's acquisition range fully covers the uncovered area. The first electronic device then sends an adjustment success message, such as "0", to the field worker. In this case, the field worker can directly fix the second image acquisition device at the last determined installation posture. This avoids the field worker having to repeatedly determine the acquisition range between the first electronic device and the installation posture when installing the second image acquisition device. A single field worker can easily complete the installation of a new image acquisition device, saving manpower and providing a more convenient and reliable installation method for field workers.
[0153] In some other examples of this specific implementation method, the output position adjustment information can be a display screen of the acquisition area and the uncovered area on the simulated image. During specific implementation, the simulated image displaying the acquisition area and the uncovered area can be captured; the captured screen is sent to the user end as the position adjustment information.
[0154] In this example, the on-site workers can hold a portable display device, so that the captured images can be sent to the on-site workers, so that the on-site workers can adjust the installation posture of the second image acquisition device according to the captured images. The image capture can be real-time. Specifically, the second image acquisition device returns the captured scene image to the second image drawing interface in real time. The second image drawing interface maps the entire scene image onto the simulation image, and only the uncovered area can be displayed on the simulation image. The first electronic device records the display screen of the second image drawing interface and feeds back the recorded screen to the portable display device in real time, so that the on-site workers can observe the changes in the scene image captured during the installation posture adjustment process in real time, so as to quickly determine the correct installation posture.
[0155] In this example, workers can install and debug the second image acquisition device on site. Then, based on the received picture, they can determine the coverage area of the image captured by the second image acquisition device on the simulated image, and determine whether the installation posture of the second image acquisition device is correct. Specifically, if the covered area on the simulated image covers the uncovered area, it means that the image acquisition range of the second image acquisition device covers the uncovered area, and the installation posture is correct. If the covered area on the simulated image does not cover the uncovered area, it means that the image acquisition range of the second image acquisition device can fully cover the uncovered range, and the installation posture needs to be adjusted.
[0156] By adopting the method of determining the installation posture of the present embodiment, the on-site worker can directly select a candidate position to install the second image acquisition device, and then instruct the second image acquisition device to capture the image of the target scene at the candidate position. After that, the entire image area of the target image captured by the second image acquisition device is mapped onto the simulation image for display. In this case, only the uncovered area and the area mapped back by the target image can be displayed on the simulation image, so as to facilitate the comparison of whether the uncovered area is within the range covered by the target image; since the screen displaying the simulation image, the selected area and the uncovered area can also be shared with the display device held by the on-site worker, the on-site worker can hold a simple display device at the location where the second image acquisition device is installed to quickly complete the position determination of the second image acquisition device. Therefore, the operation of the on-site worker can be simplified, and visual and operable hardware and software facilities are provided for the workers who perform on-site area calibration, thereby improving the efficiency of area calibration work.
[0157] Among them, the process of this application is to describe how to map the drawing area drawn by the area drawing operation in the scene image captured by the image acquisition device (including the first image acquisition device and the second image acquisition device) to the simulated image to become a candidate area.
[0158] In some embodiments, in response to the area drawing operation, a drawing area selected by the area drawing operation can be determined on the scene image; then, according to the conversion relationship between the image coordinate system of the image acquisition device and the image coordinate system of the simulated image, the drawing area is mapped to the simulated image to obtain candidate areas; and each candidate area is displayed on the simulated image.
[0159] In a specific implementation, in response to an area drawing operation, first area coordinates of a drawing area selected by the area drawing operation can be determined on a scene image; the first area coordinates can be converted into second area coordinates on a simulated image; and then, based on the second area coordinates, the image area can be mapped onto the simulated image to obtain a candidate area.
[0160] In this implementation, the coordinates of the first area are based on the image coordinate system of the scene image, that is, based on the coordinate system of the first image acquisition device, and the coordinates of the second area are based on the coordinate system of the simulated image. Generally speaking, the simulated image is a proportionally reduced image of the target scene, which is obtained based on the world coordinate system and the proportional relationship.
[0161] The first region coordinates include the coordinates of multiple endpoints of the drawing region. In some cases, they may also include the coordinates of the center of the drawing region. Specifically, they can be determined based on the shape of the drawing region. The second region coordinates correspond to the first region coordinates. If the first region coordinates include the coordinates of multiple endpoints, the second region coordinates also include the coordinates of the multiple endpoints on the simulated image.
[0162] The mapping relationship between the first region coordinates and the second region coordinates may be a conversion relationship between the image coordinate system and the coordinate system of the simulated image.
[0163] As mentioned above, the scene image is generally based on the image coordinate system of the image acquisition device, while the simulation image is based on the coordinate system provided by the engineering drawing software when drawing the target scene. The conversion between the two can be carried out with the world coordinate system as the conversion intermediary. Specifically, the world coordinate system can be understood as the coordinate system of the real world. In this embodiment, a coordinate conversion relationship between the image coordinate system and the world coordinate system can be constructed, as well as a conversion relationship between the world coordinate system and the coordinate system of the simulation image can be constructed. With the world coordinate system as the coordinate system intermediary, coordinate conversion is realized between the image coordinate system of the image acquisition device and the coordinate system of the simulation image, so that the image area in the scene image is mapped to the simulation image based on this conversion relationship.
[0164] In practice, when constructing the coordinate transformation relationship between the image coordinate system and the world coordinate system, the image acquisition device located in the target scene can be calibrated first. Specifically, calibration can be performed by obtaining the position parameters of the image acquisition device in the target scene. These position parameters are mainly used to determine the coordinate transformation relationship between its image coordinate system and the world coordinate system.
[0165] In a specific implementation, the above-mentioned position parameters can be obtained by setting multiple marking icons in the target scene, and the multiple marking icons can be used to determine the eight position parameters.
[0166] 7 , a schematic flow chart of steps for determining coordinate conversion parameters in this embodiment is shown. As shown in FIG7 , the following steps may be specifically included:
[0167] Step S701: Acquire a marked image captured by an image capture device.
[0168] The marked image includes images of a plurality of marked icons located in the target scene; and the image acquisition device includes the first image acquisition device and the second image acquisition device.
[0169] In this embodiment, the image acquisition device is located in the target scene, and it can be the first image acquisition device or the second image acquisition device, which is not limited here.
[0170] Among them, the marking icon can be a number, letter, graphic or other icon printed on a white cardboard. In practice, when the marking icon is identified, the number, letter, graphic or other icon can be identified.
[0171] Among them, each time the coordinate conversion relationship between an image acquisition device and the world coordinate system is determined, multiple marking icons can be placed within the image acquisition range of the image acquisition device; or, the placement positions of the multiple marking icons can enable multiple image acquisition devices to capture them at the same time, that is, the multiple marking icons are located within the acquisition range of multiple image acquisition devices.
[0172] Step S702: determining the image coordinates of the plurality of mark icons in the mark image, and determining the world coordinates of the plurality of mark icons.
[0173] Among them, the image coordinates of multiple mark icons in the mark image can be identified. Specifically, the image coordinates of icons such as numbers, letters, graphics, etc. in the mark icon can be identified. The specific identification method can refer to relevant technologies and will not be repeated here.
[0174] Among them, the world coordinates of the multiple marking icons can be obtained by on-site workers through measurement on site, reflecting the real coordinate positions of the multiple marking icons in the target scene. Specifically, the real coordinate position can be a three-dimensional coordinate position, but the height coordinate in the three-dimensional coordinate position can be set to 0 or 1.
[0175] Step S703: Based on the image coordinates and the world coordinates, determine the first coordinate conversion parameters corresponding to each image acquisition device.
[0176] In this embodiment, based on the image coordinates of each of the multiple marking icons in the marking image and their world coordinates in the world coordinate system, the first coordinate conversion parameters can be calculated using the method of converting the camera image coordinate system to the world global coordinate system in related technologies, where the first coordinate conversion parameters can include eight position parameters.
[0177] Step S704: Determine a second coordinate conversion parameter based on the world coordinate system and the coordinate system of the simulated image.
[0178] Since the simulation image is a plane engineering drawing or a three-dimensional stereogram of the target scene in equal proportion, the second coordinate transformation parameter can be obtained according to the proportional relationship.
[0179] For example, in a two-dimensional engineering drawing, the upper left corner of the drawing is designated as the coordinate origin, the horizontal direction is designated as the x-axis, and the vertical direction is designated as the y-axis. The coordinate origin, x-axis, and y-axis directions correspond to locations on the ground in the real world, forming the origin and coordinate system of the world coordinate system.
[0180] Assume the coordinates in the world coordinate system are (x w1 ,y w1 ), the entire area is H meters high (y direction), and W meters wide (x direction), and the corresponding coordinates in the engineering drawing coordinate system are (x w2 ,y w2 ), height h meters, width w meters.
[0181] but:
[0182] Therefore, according to formula (1), the coordinates of an object in the target scene in the simulated image when it is mapped into the simulated image can be obtained.
[0183] The drawing area selected by the area drawing operation on the scene image can be mapped to a candidate area in the simulated image based on the first coordinate conversion parameters and the second coordinate conversion parameters. Specifically, the world coordinates of the drawing area in the world coordinate system can be first obtained based on the first coordinate conversion parameters. That is, the first area coordinates can be converted to world area coordinates based on the first coordinate conversion parameters. Then, the world coordinates of the drawing area can be converted to coordinates on the simulated image based on the second coordinate conversion parameters. That is, the world area coordinates can be converted to second area coordinates on the simulated image. This results in the candidate area on the simulated image.
[0184] After obtaining the first coordinate conversion parameter and the second conversion parameter, the first coordinate parameter is associated with the corresponding image acquisition device and the second conversion parameter is associated with the corresponding simulated image. Thus, a coordinate conversion relationship among the image acquisition device (the first image acquisition device and the second image acquisition device) - the target scene - the simulated image is constructed, so that it can be directly called during subsequent area mapping.
[0185] Among them, after multiple image acquisition devices are installed in the target scene (including the first image acquisition device and the second image acquisition device), the installation posture of one or more image acquisition devices may change due to some circumstances, causing their image acquisition range to deviate from the original calibrated range. In this case, since the image acquisition range changes, the original image coordinate system is changed. If the scene area is framed based on the original calibrated area, different areas will be selected out of the frame, resulting in invalid information in the subsequent image processing and analysis.
[0186] Therefore, the first coordinate transformation parameters need to be re-determined.
[0187] When re-determining the first coordinate conversion parameters of the image acquisition device whose position has changed, the contents of steps S701-S703 above can be used. In other words, the positions of the multiple marker icons can be re-positioned so that they are within the acquisition range of the image acquisition device after the position change. Of course, in some embodiments, if the positions of the multiple marker icons are still within the acquisition range of the image acquisition device after the position change, the world coordinates of the multiple marker icons can be used, and the first coordinate conversion parameters can be determined by simply re-determining the image coordinates of the multiple marker icons in the scene image.
[0188] In a specific implementation, the world coordinates corresponding to each of the multiple calibration points can be stored; and when a change in the posture of the image acquisition device is detected, the image acquisition device with the changed posture is controlled to re-capture images of the multiple calibration points; then, based on the image coordinates of the multiple calibration points in the re-captured marked image and the world coordinates corresponding to the stored multiple calibration points, the first coordinate conversion parameters are updated; thereafter, the first coordinate conversion parameters corresponding to the image acquisition device whose position has changed are updated to the updated first coordinate conversion parameters.
[0189] Among them, the image acquisition device whose posture changes can be called a third image acquisition device in this embodiment. For example, if the posture of a first image acquisition device changes, the first image acquisition device is called a third image acquisition device. If the posture of the second image acquisition device changes, the second image acquisition device is called a third image acquisition device.
[0190] In some embodiments, when the posture of the first image acquisition device changes, if it is determined that the image acquisition range corresponding to the first image acquisition device after the posture change includes the area selected by the original calibration area, the coordinates of the original calibration area can be transformed based on the existing image coordinates and the original image coordinates of the first image acquisition device to obtain a new calibration area of the first image acquisition device, and then the new calibration area is bound to the first image acquisition device.
[0191] In some other embodiments, since the calibration area is used to indicate the area on the scene image where the object to be identified needs to be identified, the coordinate system it uses can be the image coordinate system of the image acquisition device. Since there is a first coordinate conversion parameter between the image coordinate system and the world coordinate system, the calibration coordinates of the calibration area in the world coordinate system can be obtained, that is, the position of the calibration area in the real world can be obtained. In this way, when the posture of the third image acquisition device changes, as long as its image acquisition range still includes the area framed by the calibration area, the world coordinate system can be used to build a bridge of the image coordinate systems of the third image acquisition device before and after the posture change, that is, based on the world coordinate system, the change of the calibration area of the third image acquisition device before and after the posture change is obtained, so as to ensure that the calibration area can correspond to the same area in the real scene before and after the posture change of the third image acquisition device, without the need to recalibrate the changed third image acquisition device.
[0192] In a specific implementation, referring to FIG8 , a flow chart of the steps for adjusting the calibration area after the position of the third image acquisition device changes is shown. As shown in FIG8 , the steps may include:
[0193] Step S801: Based on the first coordinate conversion parameter, the calibration coordinates of the calibration area associated with the image acquisition device in the world coordinate system may be determined.
[0194] In this embodiment, since the calibration area has image coordinates in the image acquisition device, the calibration coordinates of the calibration area in the world coordinate system can be obtained using the first coordinate transformation parameter. The calibration coordinates may include the coordinates of multiple endpoints of the calibration area and, in some cases, the coordinates of the center point of the calibration area. Thus, the actual area of the calibration area in the target scene can be determined.
[0195] Step S802: when a third image acquisition device with a changed posture is detected, if it is determined that the scene image acquired by the third image acquisition device includes an associated calibration area, the first coordinate conversion parameter corresponding to the third image acquisition device is updated.
[0196] Among them, the process of updating the first coordinate conversion parameters corresponding to the third image acquisition device can refer to the above process, that is, updating the first coordinate conversion parameters based on the image coordinates of multiple calibration points in the re-captured marked image and the world coordinates corresponding to the stored multiple calibration points.
[0197] Step S803: determining the calibration area corresponding to the position change of the third image acquisition device based on the updated first coordinate conversion parameter and the calibration coordinates corresponding to the initial calibration area;
[0198] Step S804: Associating the calibration area corresponding to the position change with the third image acquisition device.
[0199] After the first coordinate transformation parameter of the third image acquisition device is updated, a new transformation relationship between the image coordinate system of the third image acquisition device after the posture change and the world coordinate system is established. Therefore, based on the new transformation relationship, the actual area where the calibration area is located in the target scene can be re-mapped back to the scene image captured by the third image acquisition device after the posture change, thereby obtaining the calibration area in the newly captured scene image.
[0200] Among them, the calibration area corresponding to the third image acquisition device after the posture change and the calibration area corresponding to it before the position change are actually still aimed at the same area in the target scene, but its shape or position in the scene image captured by the third image acquisition device after the posture change is different, but it contains valid information and contains no or less invalid information.
[0201] Among them, posture change can refer to position change, angle change, etc. Anything that can affect the image acquisition range can be called posture change.
[0202] When the position of the image acquisition device changes, the conversion relationship between the image coordinate system of the changed image acquisition device and the world coordinate system can be obtained based on the first coordinate conversion parameter obtained after the change. The calibration area generally represents a fixed area in the real scene after being calibrated. Therefore, when the position of the image acquisition device changes, the position, size and shape of the fixed area on the captured image will also change.
[0203] In this embodiment, the fixed area in the real scene can be directly mapped back to the scene image captured after the position change according to the updated first coordinate transformation parameters, thereby obtaining the new size and new position of the calibration area in the scene image captured after the position change, thereby migrating the calibration area before the position change to the scene image captured after the position change. This avoids the need for manual recalibration of the calibration area for the image acquisition device with the changed position, which requires on-site workers to perform. Therefore, when no recalibration is required, labor costs can be saved.
[0204] In some embodiments, the area drawing operation may include touch and click operations on multiple location points on the scene image, and may also include line drawing operations on the scene image, wherein the touched or clicked location points can be understood as the endpoints of the area to be selected, and the drawn lines can be understood as the edges of the area to be selected.
[0205] In a specific implementation, a closed image area can be generated on the scene image in response to multiple position points where preset operations are performed on the scene image, and / or multiple lines drawn on the scene image; then, the candidate area to which the closed image area is mapped is displayed in the simulated image.
[0206] In a specific implementation of this embodiment, a single click or touch of a location point and a line drawing can be performed simultaneously on the scene image. In this case, an image area can be fitted based on the multiple triggered location points and drawn lines.
[0207] In a specific implementation of this embodiment, only a single click or touch of a location point can be performed on the scene image. In this case, multiple location points can be connected into a closed image area using preset lines based on the multiple triggered location points.
[0208] In a specific implementation of this embodiment, only line drawing operations may be performed on the scene image. In this case, multiple lines may be smoothed based on the multiple lines drawn, thereby fitting a closed image area.
[0209] When this implementation is adopted, it is convenient for on-site workers to draw a drawing area on the scene image, thereby optimizing the area drawing experience of on-site workers.
[0210] The following is a complete description of the region calibration method disclosed in the present invention through an example:
[0211] Taking Figure 1 as an example, the specific process includes the following:
[0212] When a furniture store needs to conduct customer flow statistics, on-site workers arrive at the store to perform calibration. They can carry a spare camera and four marking icons. The spare camera is used to add a new camera to the store. The first electronic device is used to assist in completing the calibration work. A system can be run in the first electronic device, and the system can include an area calibration module for calibrating the calibration area of the camera in the store.
[0213] S1: Start calibrating the coordinates of each camera in the store; specifically including:
[0214] S11: Place four marker icons within the range of image acquisition by camera i, then obtain the image captured by camera i, and start measuring the world coordinates of the marker icons at the placement positions; where camera i is any camera from the multiple cameras, i is greater than or equal to 1 and less than or equal to the total number of cameras;
[0215] S12: Input the acquired image into the neural network model to obtain the image coordinates of the four marker icons in the image, and obtain a total of four image coordinates;
[0216] S13: Using the four image coordinates and the four world coordinates, according to the existing method of converting the camera image coordinate system to the world global coordinate system, calculate the conversion matrix parameters based on the image coordinates to the world coordinates, i.e., the first coordinate conversion parameters, which include eight position parameters and are stored in the coordinate conversion file corresponding to camera i;
[0217] S14: Simulating a simulated image of the furniture shop, which may be a planar engineering drawing of the furniture shop, which is scaled in the same proportion as the actual furniture shop. From this, a coordinate transformation relationship between the coordinate system of the planar engineering drawing and the world coordinate system, i.e., a second coordinate transformation parameter, can be obtained;
[0218] With this, the calibration of all cameras is complete, and on-site workers can upload the first and second coordinate conversion parameters to the system for subsequent regional calibration. The system can be located in the furniture store's computer equipment and can be downloaded in advance or by on-site workers.
[0219] S2: Import the cameras in the store into the system so that the system can obtain and display the scene images captured by the cameras.
[0220] S3: Displaying the scene image captured by camera i on a system interface, specifically, a first image drawing interface specifically for drawing the scene image, and then the on-site worker uses the drawing tools provided in the first image drawing interface to draw an area excluding invalid information from the scene image;
[0221] S4: The system automatically maps the drawn area to the plane engineering drawing according to the first coordinate transformation parameter and the second coordinate transformation parameter corresponding to the camera i, and obtains the candidate area i located in the plane engineering drawing;
[0222] After the above steps S3 and S4, candidate areas of all cameras in the plane engineering drawing can be obtained. Assuming that there are N cameras in total, there are N candidate areas.
[0223] S5: In the second image drawing interface, adjusting the candidate regions on the plane engineering drawing. Specifically, if multiple candidate regions among the N candidate regions overlap, adjusting the shapes or sizes of the overlapping candidate regions so that none of the multiple candidate regions overlap.
[0224] S6: Mapping the adjusted candidate region i back to the first image drawing interface and displaying it in the scene image of camera i. If it is determined that the adjusted candidate region i is still located in the scene image of camera i and does not include invalid information, then the adjusted candidate region i is used as the calibration region of camera i; otherwise, continue to execute steps S4 and S5;
[0225] S7: Determine whether the calibration areas of the N existing cameras of the store fully cover the engineering plan;
[0226] If yes, go to step S8, if no, go to steps S9-S10:
[0227] S8: After the calibration work is completed, the calibration area corresponding to each camera can be saved, such as returned to the camera so that the scene image sent by the camera carries the image coordinates of the calibration area to indicate that only the calibration area is recognized; or it can be stored in the system and called when the image collected by the camera arrives.
[0228] S9: In the second image drawing interface, the uncovered area not covered by the calibrated area is displayed. The on-site worker pre-installs a camera k (from a backup camera carried or a camera provided by the merchant) at a candidate location S in the store, displays the image captured by camera k in the first image drawing interface, and maps the entire area covered by the image onto the engineering plan;
[0229] The system takes a screenshot of the second image drawing interface. The screenshot includes the image area captured by camera k and the uncovered area. The system then displays the screenshot on a portable display device carried by the on-site worker.
[0230] If it is determined that the image area captured by camera k covers the uncovered area, then camera k is installed at position S; if not, the position of camera K is adjusted and the above steps are repeated until the specific installation posture is determined;
[0231] S10: After the position of camera K is determined, the uncovered area can be directly returned to camera K as the calibration area of camera K, or the area can be calibrated for camera K according to the process of steps S3-S4 above.
[0232] Of course, generally speaking, according to the needs of the store, it is not necessarily required that the camera cover the entire store area. Therefore, in practice, the S9-S10 process can be omitted, and the merchant’s old camera can be used directly to complete this new function.
[0233] Example 2
[0234] Based on the same inventive concept, the present disclosure also provides an information processing method, which utilizes the calibration area of each image acquisition device determined by the above-mentioned area calibration method, so that only the calibration area in the scene image captured by the image acquisition device needs to be identified, avoiding the identification of invalid information.
[0235] 9 , a flow chart of the steps of the information processing method is shown. The method can be applied to a second electronic device. It should be noted that the second electronic device may be located in the target scene or may not be located in the target scene. Generally speaking, the second electronic device may be a device that needs to identify the object to be identified. The second electronic device may be the same device as the first electronic device or may be a different device. In the case of different devices, the first electronic device may communicate with the second electronic device and send the calibration areas corresponding to the multiple image acquisition devices to the second electronic device. The second electronic device and the first electronic device may be respectively connected to communicate with the multiple image acquisition devices to execute the information processing method.
[0236] Specifically, as shown in FIG9 , the process includes the following steps:
[0237] Step S901: acquiring scene images captured by multiple image acquisition devices of a target scene, and calibration regions corresponding to the multiple image acquisition devices, respectively; wherein the calibration regions are determined according to the region calibration method of the first embodiment above;
[0238] Step S902: Identify the object to be identified in the target area of the scene image to obtain target information; wherein the target area is the area selected by the calibration area;
[0239] Step S903: performing information processing corresponding to the scenario task on each target information, wherein the scenario task is a task corresponding to the target scenario.
[0240] In this embodiment, multiple image acquisition devices can be located in the target scene, which may include a first image acquisition device and a newly added second image acquisition device; wherein, the multiple image acquisition devices can perform image acquisition tasks after the calibration area is calibrated, and in the image acquisition task, the scene image captured for the target scene can be sent to the second electronic device.
[0241] Among them, since multiple image acquisition devices correspond to their own calibration areas respectively, and the calibration areas are drawn on-site according to needs, they contain no or little invalid information. Therefore, when identifying the objects to be identified in the scene image, only the objects to be identified in the area selected by the calibration area can be identified, thereby avoiding the interference of invalid information in the scene image and increasing the amount of calculation, and improving the calculation efficiency and accuracy.
[0242] The objects to be identified can be determined based on the information processing requirements. For example, in passenger flow analysis, the objects to be identified can be human beings, while in robot trajectory and distribution density analysis, the objects to be identified can be robots. Of course, in other scenarios, the objects to be identified can also be other types, and this is not a limitation.
[0243] Among them, the scene task is the task required in the target scene. For example, if the target scene is a furniture store, the scene task can be a customer flow analysis task, and the corresponding information processing can be an analysis of the number and density of human bodies in the store.
[0244] Of course, depending on the scene task, the information processing may also be different, and the identification of the identification object may also be different. In practice, no matter what the scene task is, since each image acquisition device can calibrate the calibration area according to the scene task, the analysis and processing of invalid information can be avoided when executing the scene task, thereby improving processing efficiency.
[0245] In this embodiment, the method of obtaining the calibration area may include the following two methods:
[0246] Method 1: Determine the calibration area corresponding to the current image acquisition device from a preset calibration area library; wherein the preset calibration area library stores the area coordinates of the calibration areas corresponding to multiple image acquisition devices.
[0247] When this method is used, the first electronic device can feed back the coordinates of the calibration area corresponding to each image acquisition device in the scene image to the second electronic device, and the second electronic device stores them in a preset calibration area library. Alternatively, the first electronic device directly stores the coordinates of the calibration area corresponding to each image acquisition device in the scene image in the preset calibration area library. The preset calibration area library can be shared by the first electronic device and the second electronic device. Thus, the second electronic device can directly retrieve the calibration area corresponding to the current image acquisition device from the preset calibration area library.
[0248] It should be noted that the current image acquisition device may be the first image acquisition device or the second image acquisition device.
[0249] Method 2: Determine the calibration area based on the area coordinates carried in the image captured by the image capture device.
[0250] When adopting the second method, the first electronic device can feed back the coordinates of the calibration area corresponding to each image acquisition device in the scene image to the image acquisition device, so that each time the image acquisition device acquires a scene image, the coordinates of the calibration area in the scene image are carried in the image file. The second electronic device parses the coordinates from the image file to obtain the target area selected by the frame, and then identifies the object to be identified in the target area.
[0251] Methods 1 and 2 can be used in combination. If the calibration area is not obtained from the preset calibration area library, it is obtained from the area coordinates carried in the image. In other words, the coordinates of the calibration area in the scene image can be carried in the scene image or stored in the preset calibration area library for the second electronic device to call.
[0252] In some embodiments, the target information may include the location information of the object to be identified, and the scene task may include: a location display task and / or an object counting task. Then, when performing information processing corresponding to the scene task on each location information, the following processing may be included:
[0253] Processing 1: When the scene task is a position display task, each position information is mapped to a simulated image of the simulated target scene for display;
[0254] In this processing, the position of the object to be identified in the target area can be identified to obtain position information, which can be the position of the object to be identified in the scene image. In practice, the position information of the object to be identified can be displayed in the simulated image, and the position of the object to be identified in the scene image can be mapped to the simulated image for display.
[0255] In one example, a third drawing interface may be provided, in which a simulated image of a target scene may be displayed. The third drawing interface may provide a variety of image display functions. Different image display functions may perform different processing on the location information, such as performing the above-mentioned processing one and processing two, and displaying the processed results in the simulated image in the third drawing interface. Specifically, multiple layers may also be provided in the third drawing interface, wherein the simulated image is located in one layer, and the processed results may be displayed in another layer.
[0256] Using the first processing method, the positions of the objects to be identified in each target area can be directly displayed in the simulated image, providing a more intuitive understanding of the positional distribution of the objects to be identified in each area of the target scene. In some examples of this method, in response to the first image display function triggered on the third drawing interface, the positions of the objects to be identified in each target area can be mapped to corresponding positions in the simulated image for display. During display, the layer where the position information is located can be different from the layer where the simulated image is located.
[0257] Of course, in this example, in response to the first image display function triggered on the third drawing interface, the position information of each object to be identified at the current time can be mapped to the simulated image for display at every preset time interval, thereby displaying the actual situation of the object to be identified in the target scene on the simulated image.
[0258] Processing 2: When the scene task is an object counting task, statistics are performed on multiple position information to obtain statistical information, and the statistical information is rendered in the simulated image.
[0259] In some examples of this approach, in response to the second image display function being triggered on the third drawing interface, the location information of the objects to be identified in each target area can be counted, and the statistical results can be rendered and displayed at the corresponding locations in the simulated image. During display, the statistical results can be displayed on a layer different from the layer on which the simulated image is located.
[0260] In this second processing, the positions of the objects to be identified in the target area can also be counted according to the corresponding dimensions to obtain statistical information in the corresponding dimensions, wherein the statistical information can be rendered in the simulated image so that the statistical results can be intuitively observed from the simulated image.
[0261] Specifically, in the second process, since it is necessary to count the positions of the objects to be identified in multiple scene images, statistical information is obtained, wherein the statistics include the dimension of statistics according to the object and the dimension of statistics according to the calibration area.
[0262] The dimension of object statistics may refer to: performing statistics on position information of a same to-be-identified object in multiple scene images to obtain trajectory information of the same to-be-identified object.
[0263] In this dimension, the multiple scene images may be different scene images, or the multiple scene images may include different scene images or the same scene images. Since different scene images are acquired by different image acquisition devices, different image acquisition devices correspond to different calibration areas, that is, to different areas in the target scene. Therefore, by counting the position information of the same object to be identified in different scene images, the activity trajectory of the object to be identified in different areas (calibration areas) in the target scene can be obtained. If the multiple scene images include different scene images and the same scene images, the activity trajectory of the same object to be identified at different position points in the target scene can be obtained.
[0264] Among them, when counting, the positions of the same object to be identified in the corresponding areas can be sorted according to the acquisition time of the scene image, so as to obtain multiple position points sorted in time order, and the multiple position points can be mapped to the simulation image to obtain trajectory information.
[0265] Specifically, in a further example, in response to the first sub-statistical function under the second image display function on the third drawing interface, the position information of the same object to be identified in multiple scene images can be counted to obtain the trajectory information of the same object to be identified, thereby mapping each trajectory point in the trajectory information to the corresponding position in the simulated image for display.
[0266] The dimension of statistics according to the calibration area may refer to: performing statistics on each position information according to the calibration area where the position represented by the position information is located, and obtaining the quantity information of the objects to be identified in different calibration areas.
[0267] In this dimension, since different scene images are captured by different image acquisition devices, and different image acquisition devices correspond to different calibration areas, it is possible to obtain multiple scene images captured by the same image acquisition device within a preset time period, and identify the number of objects to be identified in the multiple scene images, thereby obtaining the number of objects to be identified in the calibration area of the image acquisition device. Similarly, the number of objects to be identified in each of the multiple calibration areas within the preset time period can be obtained, thereby obtaining information on the number of objects to be identified in different calibration areas.
[0268] Specifically, in a further example, in response to the second sub-statistical function under the second image display function on the third drawing interface, statistics can be performed on each position information in each calibration area according to the calibration area where the position represented by the position information is located, so as to obtain the quantity information of the objects to be identified in different calibration areas, and then display the quantity information of the objects to be identified in different calibration areas in the form of a heat map in the simulated image.
[0269] In this type of statistics, the number of items corresponding to different calibrated areas can be displayed in the simulated image as a heat map, allowing users to directly observe the popularity of different calibrated areas. For example, taking customer flow statistics as an example, a heat map can intuitively understand which areas are popular and most popular with customers.
[0270] In some embodiments, the object to be identified may be a human body, which can be applied to customer flow statistics in shops. In this case, the position of the human body needs to be identified. The purpose of customer flow statistics can be achieved by identifying the position of the human body's feet. Since the human body's feet can reflect the real position of the human body in the real scene, the accuracy of identification can be improved.
[0271] In a specific implementation, the feet of a human body in the target area may be detected first; if the feet are detected, the position of the feet is identified; if the feet are not detected, the position of the feet is predicted based on the position of at least one target part of the human body;
[0272] The target parts are other parts of the human body except the feet.
[0273] Among them, other parts may include the head, torso, limbs, etc. To ensure the accuracy of the prediction of the foot position, the more other parts there are, the more accurate it is.
[0274] Among them, the method of detecting the human feet can adopt relevant technologies, such as using a neural network to detect the human feet in the target area. In this method, it is only necessary to use multiple image samples containing human feet as training samples to train a neural network model that can recognize human feet. Then, the target area in the scene image is cropped out and sent to the neural network model to obtain the foot detection result.
[0275] When the foot is detected, the position of the foot in the scene image can be further detected to obtain position information, and the position of the foot can also be mapped back to the simulation image display.
[0276] If the feet are not detected, the position of the feet can be predicted based on the position and size of other parts of the human body in the scene image. Furthermore, other parts can include the head and torso. Therefore, the position of the human body can be determined based on the positional relationship between the head and torso positions and the detected head and torso sizes, thereby predicting the position of the human feet.
[0277] In a specific implementation, when predicting the position of the feet based on the position of at least one target part of the human body, the head position and the torso position of the human body in the target area can be detected; then, based on the head position and / or torso position, the body posture category of the human body is determined; and then, based on the body posture category, the position of the feet is predicted.
[0278] Among them, the posture category can reflect the posture of the human body.
[0279] In practice, the above-mentioned neural network model can also detect the human head and the human body at the same time; and can detect the position of the human head and the human body. In practice, the neural network model can use an existing model in the relevant technology, which will not be repeated here.
[0280] Furthermore, when a head position frame where the head is located and a torso position frame where the torso is located are detected, the body posture category can be determined based on the size ratio between the head position frame and the torso frame; when only the head position frame or the torso position frame is detected, the body posture category can be determined as a preset category corresponding to the detected position frame.
[0281] Among them, body posture categories can include three categories: standing, squatting, and sitting. The size ratio of the human head and torso varies in different categories. In practice, the size ratio of the head and torso of different people in different body posture categories can be obtained through multiple experiments. Then, the comprehensive ratio value of the human head and torso in different body posture categories can be estimated, and the body posture category of the human body can be measured using this comprehensive ratio value. Among them, each body posture category can correspond to a preset position relationship between the foot position and the head and torso position respectively. Using this preset relationship, the position of the foot can be obtained by the head or torso position.
[0282] Therefore, when the head position frame where the head is located and the body position frame where the torso is located are detected, the body position category can be obtained based on the size ratio between the head position frame and the torso frame and the comprehensive ratio value. Then, the foot position can be obtained based on the position of the head position frame or the torso position frame and the preset position relationship corresponding to the body position category.
[0283] If the head position frame is not detected, but the torso position frame is detected, the posture category can be directly determined as the standing category, and the foot position is determined based on the preset position relationship corresponding to the standing category and the position of the torso frame.
[0284] Among them, if the head position frame where the head is located is detected, but the body position frame where the torso is located is not detected, the posture category can be directly determined as the standing category, and the position of the sole of the foot is determined based on the preset position relationship corresponding to the standing category and the position of the torso frame.
[0285] The following is a complete description of the region calibration method and information processing method of the present application with an example:
[0286] Taking Figure 1 as an example, assuming that the target scene is a furniture store, it is necessary to count the customer flow in the furniture store. Referring to Figure 10, a flow chart of this example is shown. In Figure 10, 101 is a first image acquisition device, and 102 is a second image acquisition device. Specifically, the process includes the following steps:
[0287] The worker arrives at the store and uses the first electronic device 1001 provided by the store to calibrate the calibration areas corresponding to the cameras in the store in the area calibration module provided by the system.
[0288] S1: Place four marker icons within the range of camera i, then obtain the image captured by camera i, and start measuring the world coordinates of the marker icons at the placement position; where camera i is any camera from a plurality of cameras, i is greater than or equal to 1 and less than or equal to the total number of cameras;
[0289] S2: Input the acquired image into the neural network model to obtain the image coordinates of the four marker icons in the image, and obtain a total of 4 image coordinates;
[0290] S3: Using the four image coordinates and the four world coordinates, according to the existing method of converting the camera image coordinate system to the world global coordinate system, calculate the conversion matrix parameters based on the image coordinates to the world coordinates, i.e., the first coordinate conversion parameters, which include eight position parameters and are stored in the coordinate conversion file corresponding to camera i;
[0291] S4: Simulating a simulated image of a furniture store, which may be a planar engineering drawing of the furniture store, which is scaled proportionally with the actual furniture store. From this, a coordinate transformation relationship between the coordinate system of the planar engineering drawing and the world coordinate system, i.e., a second coordinate transformation parameter, can be obtained;
[0292] With this, the calibration of all cameras is complete, and on-site workers can upload the first and second coordinate conversion parameters to the system for subsequent regional calibration. The system can be located in the furniture store's computer equipment and can be downloaded in advance or by on-site workers.
[0293] S5: Import the cameras in the store into the system so that the system can obtain and display the scene images captured by the cameras.
[0294] S6: Displaying the scene image captured by camera i on a system interface, specifically, a first image drawing interface specifically for drawing the scene image, and then the on-site worker draws an area excluding invalid information from the scene image using a drawing tool provided by the first image drawing interface;
[0295] S7: The system automatically maps the drawn area to the plane engineering drawing according to the first coordinate transformation parameter and the second coordinate transformation parameter corresponding to the camera i, to obtain a candidate area i located in the plane engineering drawing;
[0296] After the above steps S3 and S4 , candidate areas of all cameras in the plane engineering drawing can be obtained. Assuming there are N cameras in total, there are N candidate areas 1003 .
[0297] S8: In the second image drawing interface, adjusting the candidate regions on the plane engineering drawing. Specifically, if multiple candidate regions among the N candidate regions overlap, adjusting the shapes or sizes of the overlapping candidate regions so that none of the multiple candidate regions overlap.
[0298] S9: Mapping the adjusted candidate region i back to the first image drawing interface and displaying it in the scene image of camera i. If it is determined that the adjusted candidate region i is still located in the scene image of camera i and does not include invalid information, then the adjusted candidate region i is used as the calibration region of camera i; otherwise, continue to execute steps S4 and S5;
[0299] S10: Determine whether the calibration areas of the store's existing N cameras fully cover the area included in the valid information;
[0300] S11: If yes, the calibration work is completed, and the coordinates of the calibration area corresponding to each camera in the image coordinate system can be returned to the camera, so that the scene image sent by the camera carries the image coordinates of the calibration area.
[0301] S12: If not, displaying an uncovered area that is not covered by the calibrated area in the second image drawing interface. The uncovered area here may refer to an area that is not covered by valid information.
[0302] The on-site worker pre-installs a camera k (either a backup camera carried by the worker or a camera provided by the merchant) at a candidate location S in the store, displays the image captured by the camera k in the first image drawing interface, and maps the entire area covered by the image to the engineering plan;
[0303] The system takes a screenshot of the second image drawing interface. The screenshot includes the image area captured by camera k and the uncovered area. The system then displays the screenshot on a portable display device carried by the on-site worker.
[0304] If it is determined that the image area captured by camera k covers the uncovered area, then camera k is installed at position S; if not, the position of camera K is adjusted and the above steps are repeated until the specific installation posture is determined;
[0305] S13: After the position of camera k (102 in Figure 10) is determined, the uncovered area can be directly returned to camera K as the calibration area of camera K, or the area can be calibrated for camera K according to the process of steps S5-S8 above.
[0306] Next, the store staff activates the customer flow statistics module in the system and starts to count the customer flow. The second electronic device 1002 may be used. If the first electronic device 1001 and the second electronic device 1002 are the same device, the user can log in to the customer flow statistics module in the system to count the customer flow, including:
[0307] S14: acquiring scene images of the target scene captured by multiple image acquisition devices in real time;
[0308] S15: identifying the foot position of the human body in the area selected by the calibration area in each scene image;
[0309] S16: Map the number of human feet identified in the multiple scene images captured by different image acquisition devices to the simulated image, wherein the number of feet can be reflected in the simulated image by heat, thereby forming a statistical heat map of passenger flow, as shown in Figure 10, where the statistical heat map is represented by the origin distribution density.
[0310] Based on the same inventive concept, the present disclosure further provides a region calibration device, as shown in FIG11 , which shows a schematic structural diagram of the region calibration device, which may specifically include the following modules:
[0311] A display module 1101 is configured to display scene images captured by at least one first image capture device located in a target scene, wherein the first image capture device is configured to capture images of a portion or all of the target scene;
[0312] A drawing module 1102, in response to a region drawing operation performed on the scene image, displays a candidate region selected by the region drawing operation in a simulation image simulating the target scene;
[0313] The determination module 1103 determines, based on the candidate regions corresponding to the respective first image acquisition devices, calibration regions corresponding to the plurality of first image acquisition devices; wherein at least two adjacent calibration regions do not overlap;
[0314] The associating module 1104 associates the plurality of calibration areas with the plurality of first image acquisition devices respectively, so as to instruct to identify the object to be identified in the image area framed by the calibration area in the image acquired by the first image acquisition device.
[0315] Optionally, the determining module 1103 is specifically configured to:
[0316] In response to a region adjustment operation on some or all of the plurality of candidate regions, determining calibration regions corresponding to the plurality of first image acquisition devices;
[0317] Alternatively, based on the overlapping areas of the candidate areas in the simulated image, the candidate areas are adaptively adjusted to obtain the calibration areas corresponding to the plurality of first image acquisition devices.
[0318] Optionally, the device further comprises:
[0319] a first mapping module, configured to map the adjusted candidate region to a scene image from which the adjusted candidate region originates for display;
[0320] a repeating module configured to, upon receiving a modification operation on the adjusted candidate area in the scene image, display the area corresponding to the modification operation in the simulated image, and repeat the step of determining, based on the candidate areas corresponding to the respective first image acquisition devices, a calibration area corresponding to each of the plurality of first image acquisition devices;
[0321] The determination of the calibration area is to use the adjusted candidate area as the calibration area of the corresponding first image acquisition device when a determination operation for the adjusted candidate area is received.
[0322] Optionally, the drawing module 1102 includes:
[0323] an overlapping region determining unit, configured to determine an overlapping region of each of the candidate regions on the simulated image;
[0324] The display unit is configured to display each of the candidate regions and an overlapping area of each of the candidate regions.
[0325] Optionally, the device further comprises:
[0326] An uncovered area display module, configured to display an uncovered area in the simulated image; wherein the uncovered area includes an area not covered by the candidate area, or an area not covered by the calibrated area;
[0327] A posture determination module is configured to determine an installation posture of a second image acquisition device in the target scene based on the uncovered area, and use part or all of the uncovered area as a calibration area of the second image acquisition device;
[0328] The second image acquisition device is used to be added to the target scene to acquire images of the uncovered area.
[0329] Optionally, the posture determination module includes:
[0330] A target image acquisition unit, configured to acquire a target image obtained by the second image acquisition device acquiring an image of the target scene at a candidate position;
[0331] an acquisition region determining unit, configured to map the scene region included in the target image to the simulation image to obtain an acquisition region;
[0332] an information output unit, configured to output position adjustment information based on the acquisition area and the uncovered area;
[0333] A posture determination unit is used to determine the installation posture based on the position adjustment information.
[0334] Optionally, the information output unit includes:
[0335] A type determination subunit, configured to determine a coverage type of the uncovered area covered by the acquisition area;
[0336] An information generating subunit, configured to generate the position adjustment information based on the coverage type; wherein the position adjustment information includes a moving direction and / or a moving distance of the candidate position of the second image acquisition device;
[0337] The information sending subunit is used to send the position adjustment information to the user terminal.
[0338] Optionally, the information output unit includes:
[0339] A screenshot subunit, configured to capture a screen shot of the simulated image showing the acquisition area and the uncovered area;
[0340] The sending subunit is used to send the captured image as the position adjustment information to the user terminal.
[0341] Optionally, the drawing module 1102 includes:
[0342] a first coordinate acquiring unit, configured to determine, in response to the area drawing operation, first area coordinates of a drawing area selected by the area drawing operation on the scene image;
[0343] A second coordinate acquisition unit, configured to convert the first region coordinates into second region coordinates on a simulated image;
[0344] A mapping unit is configured to map the drawing area onto the simulation image based on the second area coordinates to obtain the candidate area.
[0345] Optionally, the drawing module 1102 includes:
[0346] an image region generating unit, configured to generate a closed image region on the scene image in response to a plurality of position points on which preset operations are performed on the scene image and / or a plurality of lines drawn on the scene image;
[0347] An image region mapping unit is configured to display, in the simulated image, a candidate region to which the closed image region is mapped.
[0348] Optionally, the device further comprises:
[0349] a marker image acquisition module, configured to acquire a marker image acquired by an image acquisition device, wherein the marker image includes images of a plurality of marker icons located in the target scene; the image acquisition device includes the first image acquisition device and the second image acquisition device;
[0350] An image coordinate acquisition module, configured to respectively determine the image coordinates of the plurality of marking icons in the marking image, and determine the world coordinates where the plurality of marking icons are located;
[0351] A first conversion module, configured to determine first coordinate conversion parameters corresponding to each of the image acquisition devices based on the image coordinates and the world coordinates;
[0352] A second conversion module is used to determine second coordinate conversion parameters based on a world coordinate system and a coordinate system of the simulated image; wherein the first coordinate conversion parameters and the second coordinate conversion parameters are used to map the image area selected by the area drawing operation on the scene image to a candidate area in the simulated image.
[0353] Optionally, the device further comprises:
[0354] a calibration coordinate storage module, configured to determine, based on the first coordinate conversion parameters, calibration coordinates of a calibration area associated with the image acquisition device in the world coordinate system;
[0355] a parameter updating module configured to, upon detecting a third image acquisition device with a changed posture, update the first coordinate transformation parameters corresponding to the third image acquisition device if it is determined that the scene image acquired by the third image acquisition device includes an associated calibration area;
[0356] an area updating module, configured to determine a calibration area corresponding to the third image acquisition device after the posture change based on the updated first coordinate conversion parameter and the calibration coordinates corresponding to the third image acquisition device;
[0357] The area association module is used to associate the calibration area corresponding to the position change with the third image acquisition device.
[0358] Based on the same inventive concept, the present disclosure also provides an information processing device, as shown in FIG12 , which shows a schematic structural diagram of the information processing device, which may specifically include the following modules:
[0359] An image acquisition module 1201 is configured to acquire scene images captured by a plurality of image acquisition devices of a target scene, and calibration regions corresponding to the plurality of image acquisition devices, respectively; wherein the calibration regions are determined by the region calibration method;
[0360] The recognition module 1202 is configured to recognize an object to be recognized in a target area of the scene image to obtain target information; wherein the target area is the area selected by the calibration area;
[0361] The information processing module 1203 is configured to perform information processing corresponding to a scenario task on each of the target information, wherein the scenario task is a task corresponding to the target scenario.
[0362] Optionally, the information processing module 1203 is specifically configured to, in response to a target image display function triggered on the third drawing interface, perform information processing corresponding to the target image display function on each target information; and display a processing result of the information processing on a simulation image simulating the target scene;
[0363] The third drawing interface is preset with multiple image display functions, and different image display functions correspond to different scene tasks.
[0364] Optionally, the target information includes the location information of the object to be identified, the scene task includes: a location display task and / or an object counting task, and the information processing module 1203 includes:
[0365] A first processing unit is configured to, when the scene task is the position display task, map each piece of position information into a simulation image simulating the target scene for display;
[0366] The second processing unit is configured to, when the scene task is the object counting task, perform statistics on the plurality of position information to obtain statistical information, and render the statistical information in the simulation image.
[0367] Optionally, the second processing unit is configured to perform at least one of the following steps:
[0368] The position information of the same object to be identified in multiple scene images is counted to obtain the trajectory information of the same object to be identified;
[0369] According to the calibration area where the position represented by the position information is located, statistics are performed on each piece of the position information to obtain quantity information of the objects to be identified in different calibration areas.
[0370] Optionally, the object to be identified is a human body, and the identification module 1202 includes:
[0371] a detection unit, configured to detect a foot of a human body in the target area;
[0372] a first recognition unit, configured to recognize a position of the foot if the foot is detected;
[0373] The second recognition unit is used to predict the position of the foot based on the position of at least one target part of the human body if the foot is not detected; the target part is other parts of the human body except the foot.
[0374] Optionally, the second identification unit is specifically configured to perform the following steps:
[0375] Detecting the head position and the torso position of the human body in the target area;
[0376] determining a body type of the human body based on the head position and / or the torso position;
[0377] Based on the posture category, the position of the foot is predicted.
[0378] Optionally, the step of detecting the head position and the torso position of the human body in the target area includes:
[0379] Detecting a head position frame where the head is located and a body position frame where the body is located;
[0380] Determining the body type of a human body based on the head position and / or the torso position includes:
[0381] When the head position frame and the body position frame are detected, determining the body posture category based on a size ratio between the head position frame and the body frame;
[0382] When the head position frame or the torso position frame is detected, the posture category is determined to be a preset category corresponding to the detected position frame.
[0383] Optionally, the step of determining the calibration areas corresponding to the plurality of image acquisition devices respectively includes:
[0384] Determine a calibration area corresponding to the current image acquisition device from a preset calibration area library; wherein the preset calibration area library stores the area coordinates of the calibration areas corresponding to the plurality of image acquisition devices;
[0385] Alternatively, the calibration area is determined based on the area coordinates carried in the scene image captured by the image capture device.
[0386] Based on the same inventive concept, the present disclosure further provides an information processing system. Referring to FIG. 13 , a schematic diagram of the information processing system framework is shown, comprising: a first module 1301, a second module 1302, and a plurality of image acquisition devices 101 located in a target scene; wherein the first module and the second module are both connected to the plurality of image acquisition devices;
[0387] Wherein, the first module is used for the area calibration method;
[0388] The second module is used to execute the information processing method based on the calibration area corresponding to each of the image acquisition devices.
[0389] Wherein, the first module is used to send the calibration areas corresponding to the plurality of image acquisition devices to the second module;
[0390] And / or, used to return the calibration areas corresponding to the multiple image acquisition devices to the image acquisition device.
[0391] The first module may be a functional module in a system, and the second module may also be a functional module in a system; or, the first module may be an electronic device, such as a first electronic device, and the second module may be an electronic device, such as a second electronic device.
[0392] Based on the same inventive concept, the present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When executed, the processor implements the area calibration method described in Example 1 or the information processing method described in Example 2.
[0393] The embodiments of the present disclosure further provide a computer-readable storage medium, which stores a computer program that enables a processor to execute the area calibration method described in the first embodiment of the present disclosure, or the information processing method described in the second embodiment.
[0394] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity, or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, commodity, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, commodity, or device that includes the element.
[0395] The above is a detailed introduction to the area calibration method, information processing method and system, device, equipment and medium provided by the present disclosure. Specific examples are used in this article to illustrate the principles and implementation methods of the present disclosure. The description of the above embodiments is only used to help understand the method of the present disclosure and its core ideas. At the same time, for general technical personnel in this field, based on the ideas of the present disclosure, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present disclosure.
[0396] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.
[0397] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
[0398] References herein to "one embodiment," "an embodiment," or "one or more embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. Furthermore, please note that instances of the phrase "in one embodiment" do not necessarily all refer to the same embodiment.
[0399] In the description provided herein, numerous specific details are described. However, it is understood that embodiments of the present disclosure may be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of this description.
[0400] In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in the claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present disclosure may be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.
[0401] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present disclosure.
Claims
1. A method for region calibration, in, The method comprises: Displaying scene images captured by at least one first image acquisition device located in the target scene, wherein the first image acquisition device is used to capture images of a part or all of the area in the target scene; In response to a region drawing operation performed on the scene image, displaying a candidate region selected by the region drawing operation in a simulation image simulating the target scene; Determine, based on the candidate areas corresponding to the respective first image devices, the calibration areas corresponding to the plurality of first image acquisition devices; The plurality of calibrated areas are respectively associated with the plurality of first image acquisition devices to indicate that an object to be identified in an image area framed by the calibrated area in an image acquired by the first image acquisition device is to be identified.
2. The area calibration method according to claim 1, in, There is no overlap between at least two adjacent calibration areas; The determining, based on the candidate areas corresponding to the respective first image acquisition devices, the calibration areas corresponding to the plurality of first image acquisition devices respectively comprises: In response to a region adjustment operation on some or all of the candidate regions among the plurality of candidate regions, determining calibration regions corresponding to the plurality of first image acquisition devices respectively; Or, based on the overlapping areas between the candidate areas in the simulated image, each candidate area is adaptively adjusted to obtain the calibration areas corresponding to each of the plurality of first image acquisition devices.
3. The method according to claim 2, in, When the candidate region is adjusted, the method further includes: Mapping the adjusted candidate area to the source scene image for display; When a modification operation on the adjusted candidate area in the scene image is received, the area corresponding to the modification operation is displayed in the simulation image, and the step of determining the calibration areas corresponding to the plurality of first image acquisition devices respectively based on the candidate areas corresponding to the respective first image acquisition devices is repeated; The determination of the calibration area includes: When a determination operation for the adjusted candidate area is received, the adjusted candidate area is used as a calibration area of the corresponding first image acquisition device.
4. The method according to claim 1, in, The step of displaying the candidate region selected by the region drawing operation in a simulated image simulating the target scene comprises: Determining, on the simulated image, an overlapping area of each of the candidate areas; The candidate regions and the overlapping regions of the candidate regions are displayed.
5. The method according to claim 1, in, After the step of displaying the candidate area selected by the area drawing operation in the simulated image simulating the target scene, the method further comprises: In the simulated image, an uncovered area is displayed; wherein the uncovered area includes an area not covered by the candidate area, or an area not covered by the calibrated area; Based on the uncovered area, determining the installation posture of the second image acquisition device in the target scene, and using part or all of the uncovered area as a calibration area of the second image acquisition device; The second image acquisition device is used to be added to the target scene to acquire images of the uncovered area.
6. The method according to claim 5, in, The step of determining the installation position of the second image acquisition device in the target scene based on the uncovered area includes: Acquire a target image obtained by capturing the target scene by the second image acquisition device at a candidate position; Mapping the scene area included in the target image into the simulation image to obtain a collection area; Outputting position adjustment information based on the acquisition area and the uncovered area; Based on the position adjustment information, the installation posture is determined.
7. The method according to claim 6, in, The outputting position adjustment information based on the acquisition area and the uncovered area includes: Determine the coverage type of the acquisition area covering the uncovered area; Based on the coverage type, generating the position adjustment information; wherein the position adjustment information includes a moving direction or a moving distance of the candidate position; The position adjustment information is sent to the user terminal.
8. The method according to claim 6, in, The outputting position adjustment information based on the acquisition area and the uncovered area includes: Capturing a simulated image showing the acquisition area and the uncovered area; The captured picture is sent to the user terminal as the position adjustment information, so that the captured picture is displayed on the user terminal.
9. The method according to any one of claims 1 to 8, in, In response to the region drawing operation performed on the scene image acquired by the first image acquisition device, displaying the candidate region selected by the region drawing operation in a simulation image simulating the target scene includes: In response to the area drawing operation, determining first area coordinates of a drawing area selected by the area drawing operation on the scene image; Converting the first region coordinates into second region coordinates on a simulated image; Based on the second region coordinates, the drawing region is mapped onto the simulation image to obtain the candidate region.
10. The method according to claim 1, in, In response to the region drawing operation performed on the scene image acquired by the first image acquisition device, displaying the candidate region selected by the region drawing operation in a simulation image simulating the target scene includes: In response to a plurality of position points on which a preset operation is performed on the scene image, and / or a plurality of lines drawn on the scene image, generating a closed image area on the scene image; The closed image area is mapped to the simulated image for display to obtain the candidate area.
11. The method according to any one of claims 1 to 7 or any one of claims 9 to 10, in, Before displaying the candidate area selected by the area drawing operation in the simulated image simulating the target scene, the method further includes: Acquire a marked image acquired by an image acquisition device, wherein the marked image includes images of a plurality of marked icons located in the target scene; the image acquisition device includes the first image acquisition device and the second image acquisition device; respectively determining the image coordinates of the plurality of marking icons in the marking image, and determining the world coordinates where the plurality of marking icons are located; Based on the image coordinates and the world coordinates, determining first coordinate conversion parameters corresponding to each of the image acquisition devices; Based on the world coordinate system and the coordinate system of the simulated image, a second coordinate transformation parameter is determined; wherein the first coordinate transformation parameter and the second coordinate transformation parameter are used to map the image area selected by the area drawing operation on the scene image to a candidate area in the simulated image.
12. The method according to claim 11, in, The method further comprises: Determining calibration coordinates of a calibration area associated with the image acquisition device in the world coordinate system based on the first coordinate transformation parameters; When a third image acquisition device with a changed posture is detected, updating a first coordinate conversion parameter corresponding to the third image acquisition device; Based on the updated first coordinate conversion parameters and the calibration coordinates corresponding to the third image acquisition device, determining a calibration area corresponding to the third image acquisition device after the posture change; The calibration area corresponding to the position change is associated with the third image acquisition device.
13. An information processing method, in, The method comprises: Respectively acquiring scene images captured by a plurality of image acquisition devices of a target scene, and determining calibration areas corresponding to the plurality of image acquisition devices respectively; wherein the calibration areas are determined according to the area calibration method according to any one of claims 1 to 12; Identify the object to be identified in the target area of the scene image to obtain target information; wherein the target area is the area framed by the calibration area; For each of the target information, information processing corresponding to a scene task is performed, wherein the scene task is a task corresponding to the target scene.
14. The method according to claim 13, in, The performing of information processing corresponding to the scene task on each of the target information includes: In response to the target image display function triggered on the third drawing interface, performing information processing corresponding to the target image display function on each of the target information; wherein the third drawing interface is preset with multiple image display functions, and different image display functions correspond to different scene tasks; The processing result of the information processing is displayed on a simulation image simulating the target scene.
15. The method according to claim 13 or 14, in, The target information includes the position information of the object to be identified, the scene task includes: a position display task and / or an object counting task, and the information processing corresponding to the scene task is performed on each of the position information, including: In the case where the scene task is the position display task, each of the position information is mapped to a simulation image simulating the target scene for display; In the case where the scene task is the object counting task, statistics are performed on the plurality of position information to obtain statistical information, and the statistical information is rendered in the simulation image.
16. The method according to claim 15, in, The positions of the objects to be identified in the plurality of scene images are counted to obtain statistical information, including at least one of the following: The position information of the same object to be identified in multiple scene images is counted to obtain the trajectory information of the same object to be identified; Statistics are collected on the calibrated area to which the position information belongs, to obtain quantity information of the objects to be identified in different calibrated areas.
17. The method according to claim 13, in, The object to be identified is a human body, and identifying the object to be identified in the target area of the scene image includes: Detecting the feet of a human body in the target area; If the foot is detected, identifying the position of the foot; If the foot is not detected, the position of the foot is predicted based on the position of at least one target part of the human body; the target part is other parts of the human body except the foot.
18. The method according to claim 17, in, The predicting the position of the foot based on the position of at least one target part of the human body comprises: Detecting the head position and the torso position of the human body in the target area; Determining a body type of the human body based on the head position and / or the torso position; Based on the posture category, the position of the foot is predicted.
19. The method according to claim 18, in, Detecting the head position and the torso position of a human body in the target area includes: Detecting a head position frame where the head is located, and a body position frame where the body is located; The step of determining the body type of a human body based on the head position and / or the body position includes: When the head position frame and the body position frame are detected, determining the body posture category based on a size ratio between the head position frame and the body frame; When the head position frame or the body position frame is detected, the posture category is determined as a preset category corresponding to the detected position frame.
20. The method according to any one of claims 13 to 19, in, Determining the calibration areas corresponding to the plurality of image acquisition devices respectively comprises: Determine a calibration area corresponding to the current image acquisition device from a preset calibration area library; wherein the preset calibration area library stores the area coordinates of the calibration areas corresponding to the multiple image acquisition devices respectively; Alternatively, the calibration area is determined based on the area coordinates carried in the scene image captured by the image capture device.
21. A region calibration device, in, include: A display module, used to display scene images captured by at least one first image acquisition device located in the target scene, wherein the first image acquisition device is used to capture images of a part or all of the area in the target scene; a drawing module, in response to a region drawing operation performed on the scene image, displaying a candidate region selected by the region drawing operation in a simulation image simulating the target scene; A determination module, based on the candidate areas corresponding to the first image acquisition devices, determines the calibration areas corresponding to the plurality of first image acquisition devices, wherein at least two adjacent calibration areas do not overlap; The associating module associates the plurality of calibrated areas with the plurality of the first image acquisition devices respectively, so as to instruct to identify the object to be identified in the image area framed by the calibrated area in the image acquired by the first image acquisition device.
22. An information processing device, in, include: An image acquisition module, used to respectively acquire scene images acquired by multiple image acquisition devices of a target scene, and calibration areas corresponding to the multiple image acquisition devices; wherein the calibration areas are determined according to the area calibration method according to any one of claims 1 to 12; A recognition module, used to recognize an object to be recognized in a target area of the scene image to obtain target information; wherein the target area is the area selected by the calibration area; The information processing module is used to perform information processing corresponding to the scene task on each of the target information, wherein the scene task is a task corresponding to the target scene.
23. An information processing system, in, include: A first module, a second module, and a plurality of image acquisition devices located in a target scene; wherein the first module and the second module are both connected to the plurality of image acquisition devices; Wherein, the first module is used to execute the area calibration method according to any one of claims 1 to 12; The second module is used to execute the information processing method described in any one of claims 13-20 based on the calibration area corresponding to each of the image acquisition devices.
24. The information processing system according to claim 23, in, The first module is used to send the calibration areas corresponding to the plurality of image acquisition devices to the second module; And / or, used to return the calibration areas corresponding to each of the plurality of image acquisition devices to the image acquisition device.
25. An electronic device, in, include: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor is executed, the method for demarcating an area as described in any one of claims 1 to 12 or the method for processing information as described in any one of claims 13 to 20 is implemented.
26. A computer-readable storage medium, in, The computer program stored therein enables the processor to execute the area calibration method described in any one of claims 1 to 12, or the information processing method described in any one of claims 13 to 20.
Citation Information
Patent Citations
Video monitoring system of high-speed railway station for realizing passenger traffic management based on scene judgment
CN107483894A
Area monitoring method and device
CN110505440A
Self-adaptive security camera monitoring method and system
CN116193251A
Video monitoring system and video monitoring method
US20160353064A1
Image processing method and apparatus, electronic device, computer-readable storage medium, and computer program product
WO2023024787A1
