Region calibration method, information processing method, system, device, equipment and medium

By determining and associating the calibration area without overlap in the image acquisition device, the problem of invalid information interference in the image is solved, and the accuracy and efficiency of image analysis processing are improved.

CN119067910APending Publication Date: 2024-12-03BOE TECHNOLOGY GROUP CO LTD +1
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202310637609.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-31
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

In the prior art, there is invalid information in the images collected by the image acquisition device that is independent of service, resulting in a decrease in the analysis processing efficiency and accuracy.

Method used

By displaying the images acquired by the image acquisition device in the target scene, the user is allowed to draw candidate areas in the simulated image and determine the corresponding calibration areas, ensuring that there is no overlap between the calibration areas, and correlating these areas with the image acquisition device to indicate to be identified The object recognition area.

Benefits of technology

Effectively eliminate invalid information in the image, improve the accuracy and efficiency of analysis and processing, reduce the amount of calculation, and enhance the targeted processing of effective information.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119067910A_ABST
    Figure CN119067910A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a region calibration method, an information processing method and system, a device, equipment and a medium, and belongs to the technical field of information processing.The region calibration method comprises the steps that scene images collected by at least one first image collection device located in a target scene are displayed, the first image acquisition device is used for performing image acquisition on a part of or all areas in a target scene; in response to a region drawing operation performed on the scene image, displaying a candidate region selected by the region drawing operation in a simulation image simulating the target scene; based on the candidate areas corresponding to the first image acquisition devices, calibration areas corresponding to the first image acquisition devices are determined; and associating the plurality of calibration areas with a plurality of first image acquisition devices respectively so as to indicate to identify a to-be-identified object in an image acquired by the first image acquisition devices based on the calibration areas.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of information processing technology, and in particular to a region calibration method, an information processing method and system, a device, equipment and a medium. Background Art

[0002] With the development of computer technology, image acquisition devices are used to acquire images of target scenes, so as to perform services such as the flow of people in the target area and the motion tracking of the objects to be identified based on the results of the analysis and processing of the acquired images. For example, taking the target scene as a shopping mall scene, multiple cameras are deployed in the mall, and the images acquired by the cameras include shopping guides and customers entering the mall for consumption. By identifying the customers, the customer flow of the mall can be analyzed.

[0003] In the related art, invalid information irrelevant to the business may exist in the images collected by the image acquisition device, which may interfere with the analysis and processing and reduce the efficiency and accuracy of the analysis and processing. Summary of the invention

[0004] In a first aspect, the present disclosure provides a method for region calibration, the method comprising:

[0005] Displaying scene images captured by at least one first image acquisition device located in the target scene, wherein the first image acquisition device is used to capture images of a part or all of the area in the target scene;

[0006] In response to a region drawing operation performed on the scene image, displaying a candidate region selected by the region drawing operation in a simulation image simulating the target scene;

[0007] Based on the candidate areas corresponding to the first image acquisition devices, determine the calibration areas corresponding to the plurality of first image acquisition devices, wherein at least two adjacent calibration areas do not overlap;

[0008] The plurality of calibrated areas are respectively associated with the plurality of first image acquisition devices to indicate that the objects to be identified in the images acquired by the first image acquisition devices are to be identified based on the calibrated areas.

[0009] A second aspect of the present disclosure provides an information processing method, the method comprising:

[0010] Respectively acquiring scene images captured by a plurality of image acquisition devices of a target scene, and calibration areas corresponding to the plurality of image acquisition devices; wherein the calibration areas are determined according to the area calibration method described in the first aspect;

[0011] Identify the object to be identified in the target area of ​​the scene image to obtain target information; wherein the target area is the area framed by the calibration area;

[0012] For each of the target information, information processing corresponding to a scene task is performed, wherein the scene task is a task corresponding to the target scene.

[0013] A third aspect of the present disclosure provides an information processing system, comprising: a first module, a second module, and a plurality of image acquisition devices located in a target scene; wherein the first module and the second module are both connected to the plurality of image acquisition devices;

[0014] Wherein, the first module is used to execute the area calibration method described in the first aspect;

[0015] The second module is used to execute the information processing method described in the second aspect based on the calibration area corresponding to each of the image acquisition devices.

[0016] A fourth aspect of the present disclosure provides a region calibration device, comprising:

[0017] A display module, used to display scene images captured by at least one first image acquisition device located in the target scene, wherein the first image acquisition device is used to capture images of a part or all of the area in the target scene;

[0018] a drawing module, in response to a region drawing operation performed on the scene image, displaying a candidate region selected by the region drawing operation in a simulation image simulating the target scene;

[0019] A determination module, based on the candidate areas corresponding to the first image acquisition devices, determines the calibration areas corresponding to the plurality of first image acquisition devices, wherein at least two adjacent calibration areas do not overlap;

[0020] The associating module associates the plurality of calibrated areas with the plurality of the first image acquisition devices respectively, so as to instruct to identify the image area framed by the calibrated areas in the images acquired by the first image acquisition devices.

[0021] According to a fifth aspect of the present disclosure, there is provided an information processing device, comprising:

[0022] An image acquisition module, used to respectively acquire scene images acquired by a plurality of image acquisition devices of a target scene, and calibration areas respectively corresponding to the plurality of image acquisition devices; wherein the calibration areas are determined by the area calibration method;

[0023] A recognition module, used to recognize an object to be recognized in a target area of ​​the scene image to obtain target information; wherein the target area is the area selected by the calibration area;

[0024] The information processing module is used to perform information processing corresponding to the scene task on each of the target information, wherein the scene task is a task corresponding to the target scene.

[0025] The present disclosure also discloses an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes, the method for demarcating an area as described in the first aspect or the method for processing information as described in the second aspect is implemented.

[0026] The fourth aspect of the embodiments of the present disclosure further discloses a computer-readable storage medium, which stores a computer program that enables a processor to execute the area calibration method as described in the first aspect, or the information processing method as described in the embodiments of the second aspect of the present disclosure.

[0027] The region calibration method according to the first aspect of the present disclosure can be applied to a target scene. Specifically, the scene images captured by each of the multiple first image acquisition devices located in the target scene can be displayed, and in response to a region drawing operation performed on the scene image, the candidate region selected by the region drawing operation can be displayed in a simulated image simulating the target scene; then, based on the candidate regions corresponding to the respective first image acquisition devices, the calibration regions corresponding to the multiple first image acquisition devices are determined; wherein there is at least no overlap between two adjacent calibration regions; and then, the multiple calibration regions are respectively associated with the multiple first image acquisition devices to indicate that the objects to be identified in the images of the first image acquisition devices are to be identified based on the calibration regions.

[0028] Since the candidate area corresponding to the drawing operation can be determined based on the area drawing operation performed on the scene image collected by the first image acquisition device, that is, the candidate area on the scene image can be selected, and then the candidate area is displayed in the simulated image of the target scene, so that the calibration area corresponding to each first image acquisition device can be determined based on the candidate area displayed in the simulated image, and after the calibration area is determined, the first image acquisition device can be bound to the calibration area. In this way, when analyzing and processing the image collected by the first image acquisition device, the object to be identified in the image can be identified based on the calibration area, thereby improving the identification of the object to be identified in the image and avoiding the analysis and processing of invalid information (such as invalid area) in the image. In this way, even if there is invalid information irrelevant to the business in the image collected by the image acquisition device, in the analysis and processing, the invalid information can be excluded based on the calibration area, and only the valid information in the image is processed, thereby eliminating the interference of the invalid information on the analysis and processing, improving the analysis accuracy, and reducing the processing of invalid information, while improving the analysis and processing efficiency.

[0029] By adopting the image processing method of the second aspect of the present disclosure, the calibration area corresponding to the image acquisition device in the target scene can be used to identify the object to be identified in the target area framed by the calibration area in the scene image acquired by the image acquisition device, thereby obtaining target information, and then performing information processing on the obtained target information. Since each image acquisition device has a corresponding calibration area, when identifying the object to be identified in the image acquired by the image acquisition device, it is possible to avoid identifying the object to be identified in the invalid area outside the target area, thereby improving the pertinence of image processing, avoiding the processing of invalid information, and further reducing the amount of calculation and improving the accuracy of information processing.

[0030] The above description is only an overview of the technical solution of the present disclosure. In order to more clearly understand the technical means of the present disclosure, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present disclosure more obvious and easy to understand, the specific implementation methods of the present disclosure are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the description of the embodiments of the present disclosure will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0032] Figure 1A schematic diagram showing an application scenario in which the area calibration method and information processing method disclosed herein are applied;

[0033] Figure 2 A flowchart of the steps of the area calibration method in an embodiment of the present disclosure is shown;

[0034] Figure 3 The area calibration method in the embodiment of the present disclosure is applied to Figure 1 A flowchart of the implementation environment shown;

[0035] Figure 4 A schematic diagram showing interface changes before and after a region adjustment operation is performed on a candidate region on a simulated image in an embodiment of the present disclosure is shown;

[0036] Figure 5a A schematic diagram of an interface for adaptively adjusting a candidate area in an embodiment of the present disclosure is shown;

[0037] Figure 5b A schematic diagram showing another interface for adaptively adjusting the candidate area in the embodiment of the present disclosure

[0038] Figure 6 The embodiment of the present disclosure is shown in Figure 1 Taking the application scenario of as an example, a schematic diagram of the process of determining the installation posture;

[0039] Figure 7 A schematic diagram of the process flow of determining the coordinate conversion parameters in an embodiment of the present disclosure is shown;

[0040] Figure 8 A schematic diagram of the steps of adjusting the calibration area after the position of the image acquisition device changes in an embodiment of the present disclosure is shown;

[0041] Fig. 9 A schematic diagram showing the steps of the information processing method in the second embodiment of the present disclosure is shown;

[0042] Fig.10 The present disclosure shows Figure 1 A complete flow chart showing a combination of a regional calibration method and an information processing method, taking a furniture store as an example;

[0043] Fig.11 A schematic diagram of the structure of a region calibration device in an embodiment of the present disclosure is shown;

[0044] Fig.12 A schematic diagram showing the structure of an information processing device in an embodiment of the present disclosure is shown;

[0045] Fig.13 A schematic diagram of the structure of an information processing system in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0046] In order to make the above-mentioned purposes, features and advantages of the present disclosure more obvious and understandable, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0047] In response to the problems in the related technology, some conventional solutions are: first identify the invalid information in the image, for example, identify the invalid information through the content correlation between images captured by multiple image acquisition devices, and then remove the invalid information from the image and process the remaining valid information; this method will consume a large amount of computing power due to the identification of invalid information, and the recognition accuracy is not necessarily reliable, and may not match the analysis business to be targeted.

[0048] In view of this, the present disclosure proposes an improved idea for solving the above-mentioned technical problems: in the target scene, the image captured by the image acquisition device can determine its image acquisition range, and a calibration area is determined within the image acquisition range. Thereafter, the calibration area is associated with the image acquisition device. Thus, the calibration area can be used to indicate the area of ​​the image captured by the image acquisition device that needs to be identified. In this way, invalid information in the image is eliminated from the source, thereby avoiding the identification and elimination of invalid information, reducing the amount of calculation and improving efficiency, and at the same time, improving the targeted analysis and processing of valid information, thereby improving the accuracy of the analysis and processing.

[0049] Reference Figure 1 As shown, a schematic diagram of an application scenario in which the area calibration method and information processing method of the present disclosure are applied is shown. Figure 1 As shown, taking the target scene as a shopping mall scene as an example, a plurality of image acquisition devices 101 are deployed in the shopping mall, wherein the plurality of image acquisition devices 101 may be the original equipment of the shopping mall. For the shopping mall scene, the services to be performed include customer flow statistics service, customer flow tracking service, shopping guide tracking service, etc. Different services need to be performed based on the images captured by the image acquisition devices.

[0050] in, Figure 1 The explanation is made using a merchant's two-dimensional plane engineering drawing as an example.

[0051] In practice, it is not limited to the above scenarios. For example, taking the target scenario as a factory scenario, especially an unmanned factory, a plurality of image acquisition devices are deployed in the factory, wherein the plurality of image acquisition devices may be the original equipment of the factory. For the factory, the services to be performed include monitoring the movement trajectory of robots in the factory, monitoring the trajectory of loading vehicles, and counting the number of loading vehicles distributed in multiple production lines, etc., so that the factory's loading equipment can be reasonably scheduled to adapt to the production capacity of the production line.

[0052] Of course, in addition to the above application scenarios, other application scenarios may also be included in practice. The area calibration method proposed in the present disclosure may be applicable to multiple application scenarios where an image acquisition device needs to be arranged for image acquisition and the acquired images need to be processed and analyzed.

[0053] In one embodiment, in combination Figure 1 As shown, the area calibration method disclosed in the present invention is introduced and explained.

[0054] Reference Figure 2 and Figure 3 As shown, Figure 2 FIG. 4 shows a flow chart of the steps of the area calibration method in this embodiment. Figure 3 It shows that the area calibration method in this embodiment is applied to Figure 1 The flowchart in the implementation environment shown in FIG. Figure 2 As shown, the area calibration method can be applied to a first electronic device, and the first electronic device can be located in a target scene or not in a target scene, and can specifically include the following steps:

[0055] Step S201: displaying scene images captured by at least one first image capture device located in a target scene;

[0056] The first image acquisition device is used to acquire images of a part or all of the area in the target scene.

[0057] In this embodiment, the target scene can be a scene constructed by an actual physical area, which can be understood as a physical scene constructed by a building area and objects located in the building area, such as a furniture store, which is a furniture sales scene formed by arranging various types of furniture in a house. Among them, the target scene includes the building area and the objects located in the building area.

[0058] Among them, the first image acquisition device can be an inherent device originally installed in the building area, which can be understood as a device used for other tasks deployed when the target scene is initially built. For example, the first image acquisition device can be installed in a furniture store when it is decorated. In this way, the existing image acquisition device in the target scene can be reused.

[0059] Among them, the first image acquisition device can be a camera, a dome camera or other imaging equipment, but is not limited to the above-mentioned equipment. It should be noted that when it is a dome camera, since the dome camera can realize the rotation of the camera, thereby adjusting the image acquisition range, in the embodiment of the present application, the dome camera can be fixed at a certain rotation position, so as to calibrate the calibration area of ​​the dome camera at the fixed position.

[0060] Among them, one or more first image acquisition devices can be set in the target scene. In the case of setting one first image acquisition device, the first image acquisition device can capture images of part or all of the target scene, preferably, all of the target scene; in the case of setting multiple first image acquisition devices, different first image acquisition devices can capture images of different areas in the target scene, which can allow the image acquisition ranges of different first image acquisition devices to overlap. Specifically, the image acquisition range of the first image acquisition device can be determined according to the size of the target scene.

[0061] Among them, the first image acquisition device performs image acquisition on the entire area of ​​the target scene means that: the scene image acquired by the first image acquisition device includes the entire area of ​​the target scene, that is, all areas are located within the acquisition range of the first image acquisition device; similarly, the first image acquisition device performs image acquisition on a partial area of ​​the target scene means that: the partial area of ​​the target scene is located within the acquisition range of the first image acquisition device.

[0062] Step S202: In response to the region drawing operation performed on the scene image, the candidate region selected by the region drawing operation is displayed in the simulation image of the simulation target scene.

[0063] In this embodiment, the scene image captured by the first image acquisition device can be sent to the first electronic device and displayed in the system installed in the first electronic device. The system can be used to perform area calibration. Specifically, the scene image can be displayed in the first image drawing interface of the system. The first image drawing interface can be an interface for drawing the area of ​​the scene image.

[0064] Among them, tools for area drawing, such as graphic boxes, lines, etc., are provided in the first image drawing interface. Users can use the drawing tools by clicking on the drawing tools or using shortcut keys to draw the area to be selected on the scene image (hereinafter referred to as the drawing area). The area to be selected is an area that contains no invalid information or less invalid information.

[0065] Among them, the area drawing operation can be performed by the user on the first image drawing interface for the scene image. In practice, a layer can be added to the scene image, and the area drawing operation can be performed on the layer. After the drawing is completed, the drawing area drawn on the layer can be associated with the scene image and saved.

[0066] It should be noted that the drawing area drawn by the area drawing operation can be determined according to the object that needs to be identified in the target scene (the object to be identified). If the object to be identified is a human body, the selected area can be an area where the human body is usually located. Taking a furniture store as an example, if the object to be identified is a human body, the selected area usually does not include walls, cabinets and other areas in the target scene.

[0067] Therefore, by providing an image drawing interface to the user, the image captured by the image acquisition device can be manually calibrated to identify areas that do not contain invalid information or contain little invalid information, thereby improving the accuracy of area calibration and providing users with visual operations. Compared with the later use of calculation algorithms, which requires setting different algorithms for different situations, requires a large amount of development, and consumes too many computing resources, the use of user-based area drawing operations can provide an area calibration solution that consumes less software and hardware resources and is more time-saving and labor-saving.

[0068] In this embodiment, since the drawing area is based on the user's area drawing operation, its shape can be any shape that matches the coverage area, thereby improving the matching degree between the calibration area and the area where the valid information in the target scene is located.

[0069] In response to the drawn area drawn by the area drawing operation, the drawn area can be mapped to the simulated image of the simulated target scene for display, thereby obtaining the candidate area located in the simulated image. The simulated image can be a scaled engineering plan view of the target scene. Of course, in some examples, it can also be a scaled three-dimensional stereogram of the target scene, which is not specifically limited here. It should be noted that the simulated image can include a simulated part of the building area in the target scene and a simulated part of the objects arranged in the building area.

[0070] When the simulated image is a three-dimensional stereoscopic image, it is necessary to map the drawing area drawn by the area drawing operation into the three-dimensional space, so that the selected area will be presented more three-dimensionally.

[0071] like Figure 3 As shown, Figure 1Taking the scene shown as an example, the scene image captured by the first image acquisition device 101 in the upper right corner of the target scene is sent to the first electronic device, and the first electronic device displays it on the first drawing interface. On the first drawing interface, the user performs an area drawing operation by drawing work to obtain a drawing area 301, and then maps the drawing area 301 to the simulation map of the target scene to obtain a candidate area 302 on the simulation image.

[0072] It should be noted that a scene image captured by a first image capture device may allow one drawing area to be drawn, or may allow multiple drawing areas to be drawn. For example, in a scene image, two or even three drawing areas may be drawn, and different drawing areas may include different image areas in the scene image.

[0073] Step S203: determining the calibration areas corresponding to the plurality of first image acquisition devices based on the candidate areas corresponding to the respective first image acquisition devices;

[0074] There is no overlap between at least two adjacent calibration areas.

[0075] In this embodiment, it is possible to determine whether each candidate area covers a valid area in the target scene that needs to be subjected to information analysis and processing, based on the candidate areas corresponding to each of the multiple first image acquisition devices displayed in the simulated image, wherein the valid area may refer to: an area where the recognition result of the object to be recognized is valid. For example, taking the target scene as a furniture store scene, in this scene, the valid area can be determined based on the object to be recognized. If the object to be recognized is the location of a customer entering the store, the determined valid area may refer to: a furniture display area excluding walls, furniture, and stocking areas. In this case, the recognition of the customer's position in the display area is valid, while the recognition results in areas such as walls and furniture are invalid. This can avoid identifying walls and furniture (such as desktops and cabinets) as the customer's location when recognizing the customer's location.

[0076] Among them, when multiple candidate areas all cover the effective area in the target scene, each candidate area can be used as the calibration area corresponding to the corresponding first image acquisition device. The calibration area can be understood as the area to be identified in the image acquired by the first image acquisition device, where the object to be identified is identified. That is to say, the calibration area is used to specify the area to be identified in the image. As shown above, if the coordinates used in the calibration area are the coordinates in the simulated image, they need to be mapped back to the scene image, so that the area mapped back to the scene image is the area to be identified; for example, if the coordinates used in the calibration area are the coordinates in the scene image, then the calibration area in the scene image is the area to be identified. Of course, in order to reduce the difficulty of calculation and reduce the process, the coordinates of the calibration area can directly use its coordinates in the scene image.

[0077] In order to avoid valid information appearing in candidate areas corresponding to different first image acquisition devices, which leads to repeated analysis of the candidate areas, the overlapping areas between the candidate areas can be displayed in the simulated image, and then the overlapping areas are processed so that there are no overlapping areas between the candidate areas, or at least there are no overlapping areas between the candidate areas that do not need to be repeatedly analyzed, so that there are at least two adjacent calibration areas without overlapping areas.

[0078] Among them, processing the overlapping area may include: removing the overlapping area in one of the candidate areas while retaining the integrity of the other candidate area; or, dividing the overlapping area into two overlapping candidate areas in proportion according to their overlapping shapes, so that the two candidate areas each take a partial overlapping area. For example, taking the overlapping area of ​​candidate area 1 and candidate area 2 as a triangle, the vertical line of the triangle can be determined first, and then the triangular area can be divided into two triangular areas according to the vertical line. One of the two triangular areas is divided into candidate area 1, and the other triangular area is divided into candidate area 2. In this way, the two candidate areas no longer overlap.

[0079] The overlapping area processing method is adopted so that the processed calibration area is still located in the original candidate area, but there is no overlapping area between the calibration areas. In this way, there will be no repeated valid information, thereby avoiding repeated analysis of valid information.

[0080] Step S204: Associating the plurality of calibrated areas with the plurality of first image acquisition devices respectively, to indicate that the objects to be identified in the images acquired by the first image acquisition devices are to be identified based on the calibrated areas.

[0081] After determining the calibration area corresponding to each first image acquisition device, the calibration area can be associated with the first image acquisition device, wherein the coordinate position of the calibration area in the scene image acquired by the first image acquisition device can be associated with the first image acquisition device, that is, associated with the first image acquisition device may be the coordinates of the calibration area in the image coordinate system, and the image coordinate system here may refer to the coordinate system of the first image acquisition device.

[0082] Among them, after the calibration area is associated with the first image acquisition device, when it is necessary to subsequently identify the object to be identified in the image acquired by the first image acquisition device, the object to be identified in the image can be identified based on the calibration area. Specifically, only the object to be identified in the image area corresponding to the calibration area in the acquired image can be identified, and the object to be identified outside the calibration area is not identified. The image area corresponding to the calibration area refers to: the area framed by the calibration area in the image acquired by the first image acquisition device.

[0083] like Figure 3 As shown, when the first image acquisition device in the upper right corner acquires a scene image during use, it will only recognize the object to be recognized in the image area framed by the calibration area 302 in the image, and will not recognize other areas in the image. Assuming that the calibration area 302 is the area corresponding to the drawing area 301 (that is, the candidate area corresponding to the drawing area 301 is not adjusted in the simulated image, but is directly determined as the calibration area corresponding to the first image acquisition device in the upper right corner), when recognizing the object to be recognized in the image captured by the first image acquisition device in the upper right corner, the object to be recognized located in the drawing area 301 in the image is directly recognized.

[0084] from Figure 3 It can be seen that the shape of the calibration area is not fixed, and it can be formed into an area of ​​any shape through drawing operations, thereby accurately screening out areas with only valid information. Then, the first electronic device can return the image coordinates of the calibration area in the scene image to the first image acquisition device in the upper right corner.

[0085] By adopting the region calibration method proposed in this embodiment, the region drawing operation can be performed on the scene image to realize the manual selection of the drawing region on the scene image, and then the drawing regions drawn by each first image acquisition device are mapped to the simulated image of the target scene, so that the candidate regions corresponding to each first image acquisition device can be obtained. Based on the display of each candidate region on the simulated image, the calibration region corresponding to each first image acquisition device can be determined. After the calibration region is determined, the calibration region can be bound to the first image acquisition device. In this way, when analyzing and processing the image acquired by the first image acquisition device, the object to be identified in the image region framed by the calibration region in the image can be analyzed and processed, while avoiding analyzing and processing the object to be identified in the uncalibrated image region in the image. In this way, even if there is invalid information irrelevant to the business in the image acquired by the image acquisition device, in the analysis and processing, the invalid information can be excluded based on the calibration region, and only the valid information in the image is processed, thereby eliminating the interference of the invalid information on the analysis and processing, improving the analysis accuracy, reducing the amount of calculation due to reducing the processing of invalid information, and improving the analysis and processing efficiency.

[0086] In some embodiments, when displaying candidate areas on a simulated image, the candidate areas and the overlapping areas between the candidate areas can be displayed. Specifically, the overlapping areas between the candidate areas can be determined on the simulated image, and the candidate areas and the overlapping areas between the candidate areas can be displayed.

[0087] In this embodiment, a simulated image, candidate areas, and overlapping areas can be displayed in a second image drawing interface, wherein a display window is provided in the second image drawing interface, and the simulated image, candidate areas, and overlapping areas are displayed in the display window, and the simulated image, candidate areas, and overlapping areas are all based on the same coordinate system, that is, the coordinate system of the simulated image.

[0088] When displaying, two layers may be set, one of which is used to display the simulated image, and the other is used to display the candidate area and the overlapping area between the candidate areas; in some other embodiments, three layers may be set, one of which is used to display the simulated image, another is used to display the candidate area, and another is used to display the overlapping area between the candidate areas. Of course, in order to facilitate the adjustment of the candidate area in the second image drawing interface, the overlapping area and the candidate area may be displayed on the same layer.

[0089] Among them, since the overlapping area is the overlapping part of the candidate area, the overlapping area in the candidate area and the part that does not overlap with other candidate areas can be displayed with different transparencies, for example, the overlapping area has a lower transparency, so that the user can distinguish the overlapping part and the non-overlapping part in the candidate area.

[0090] In some other embodiments, in addition to displaying the overlapping area, an uncovered area not covered by the candidate area may also be displayed. The uncovered area may include: an area that is not within the acquisition range of the first image acquisition device, and an area that is within the acquisition range of the first image acquisition device but is not covered by the candidate area.

[0091] In this embodiment, in one case, the first image acquisition device in the target scene can perform image acquisition on a partial area in the target scene, so the scene images acquired by multiple first image acquisition devices may not cover all the target scene, so that the candidate area does not cover all the simulated images; in another case, even if the first image acquisition device covers the target scene in its entirety, since the candidate area is a partial area in the scene image, each candidate area may not cover the entire target scene, so there is an uncovered area that is not covered by the candidate area. Therefore, the uncovered area may only include areas that are not within the acquisition range of the first image acquisition device, or only include areas that are within the acquisition range of the first image acquisition device but are not covered by the candidate area; or when there are multiple uncovered areas, they include both areas that are not within the acquisition range of the first image acquisition device and areas that are within the acquisition range of the first image acquisition device but are not covered by the candidate area.

[0092] Since overlapping areas and uncovered areas can be displayed in the simulated image, multiple candidate areas with overlapping areas can be adjusted. For uncovered areas, a second image acquisition device 102 can be added to the target scene to cover the uncovered areas, thereby achieving full coverage of the target scene.

[0093] In some embodiments, the calibration area can be determined based on the user's adjustment of the candidate area on the second image drawing interface, that is, the user can be allowed to adjust the candidate area, so as to determine the calibration area corresponding to each image acquisition device based on the user's area adjustment operation on the candidate area. In this case, it is convenient for on-site construction personnel to accurately divide the calibration area, providing a convenient way for on-site construction personnel to operate manually.

[0094] In some other embodiments, the determination of the calibration area can be performed by the electronic device. Specifically, the overlapping candidate areas can be adaptively adjusted according to the overlapping areas between the candidate areas, so as to obtain the calibration area corresponding to each first image acquisition device. In this way, the electronic device can adaptively obtain the calibration area according to the pre-stored software program, without manual intervention, and can reduce the workload of workers.

[0095] Wherein, in the case where the user is allowed to adjust the candidate regions, the calibration regions corresponding to the plurality of first image acquisition devices may be determined in response to the region adjustment operation on some or all of the plurality of candidate regions;

[0096] When the candidate areas are adaptively adjusted by the electronic device, each candidate area can be adaptively adjusted based on the overlapping area between the candidate areas in the simulated image to obtain the calibration areas corresponding to each of the multiple first image acquisition devices.

[0097] First, the user's operation of adjusting the candidate area is explained:

[0098] In a specific example of this embodiment, the simulated image and the candidate area on the simulated image may be displayed in a second image drawing interface, and a graphic drawing tool is provided in the second image drawing interface, the graphic drawing tool providing a tool for adjusting the candidate area in the simulated image.

[0099] As described above, the simulated image and the candidate area can be located in different layers so that the simulated image and the candidate area can be modified separately, wherein the simulated image can be set to a state where modification is not allowed, that is, the layer where the simulated image is located is locked to prohibit modification of the simulated image. Among them, the adjustment tool provided can be used to adjust the shape and size of the candidate area on the layer of the candidate area. The user can adjust the shape and size of the candidate area by selecting the corresponding tool. Specifically, the above-mentioned area adjustment operation can include: based on the selected tool, the adjustment operation of the shape and size of the candidate area is performed on the layer where the candidate area is located.

[0100] After the region adjustment operation is completed, the user may trigger a save operation, and the first electronic device may display the adjusted candidate region on the simulated image in response to the save operation, and may save the adjusted candidate region at the same time.

[0101] Of course, in some examples, the candidate area can be adjusted multiple times, wherein the areas obtained after multiple adjustments can be saved so that the user can select an adjusted area as the calibration area. Alternatively, after obtaining the area adjusted this time, the area adjusted this time can overwrite the area adjusted last time before this time, so that the saved adjusted areas are all the latest adjusted areas.

[0102] For example, see Figure 4 , Figure 4 FIG. 4 shows a schematic diagram of the interface changes before and after the region adjustment operation is performed on the candidate region on the simulated image. Figure 4As shown, there are multiple tools in the image drawing interface, including at least a tool for adjusting shape and a tool for adjusting size. The user can use these tools to adjust the shape and size of the candidate area by clicking the tools on the interface or by triggering shortcut keys on the keyboard.

[0103] Figure 4 FIG. 3 is a schematic diagram of the changes of candidate regions 302 and 303 before and after the adjustment. It can be seen that before the adjustment, there is an overlap between candidate regions 301 and 302, and after the adjustment, there is no overlap between the two candidate regions. Thus, it can be ensured that the calibration regions contain no repeated information as much as possible.

[0104] After performing a region adjustment operation on a candidate region, the adjusted candidate region and other candidate regions can be displayed on the second image drawing interface so that the user can observe whether there is overlap between the adjusted candidate regions. If so, the candidate region can be adjusted again.

[0105] In an optional method, after the shape of the candidate area is adjusted, the size of the corresponding candidate area is also changed at the same time. Since the candidate area is generally an area with valid information selected in the scene image, if the shape remains unchanged and the size is simply adjusted, some valid information may be missed after the adjustment. Therefore, in this optional method, the zoom function that changes the size of the candidate area as a whole can be locked, that is, the tool for zooming the area can be locked so that the candidate area will not be reduced as a whole, and by changing the shape, there is no overlapping area between the candidate areas.

[0106] Next, the first electronic device adaptively adjusts each candidate area as follows:

[0107] The adaptive adjustment of each candidate region may be performed based on the overlapping region between the candidate regions. In one adaptive adjustment, when two candidate regions have overlapping regions, the overlapping region may be subtracted from any of the two overlapping candidate regions, and the remaining region of the candidate region is used as the calibration region.

[0108] The method of subtracting the overlapping area from any one of the two candidate areas may be:

[0109] Determine the first position coordinates of the overlapping area in the candidate area to be adjusted, and the second position coordinates of the candidate area to be adjusted, calculate the first position coordinates and the second position coordinates, so as to remove the overlapping area from the candidate area to be adjusted. Alternatively, after determining the position of the overlapping area in the candidate area to be adjusted, the overlapping area can be removed from the candidate area to be adjusted using a cutout technique.

[0110] For example, refer to Figure 5a As shown, a schematic diagram of an interface for adaptively adjusting the candidate area is shown, such as Figure 5a As shown, there is an overlapping area between candidate area 501 and candidate area 502. Figure 5a The shaded portion is an overlapping area, and the overlapping area will be subtracted from the candidate area 501, and the remaining area in the candidate area 501 will be used as the calibration area of ​​the first image acquisition device A, while the complete area of ​​the candidate area 502 will be retained.

[0111] Of course, in some other examples, the overlapping area may be subtracted from the candidate area 502, and the remaining area in the candidate area 502 is used as the calibration area of ​​the first image acquisition device B. The complete area of ​​the candidate area 501 is retained.

[0112] In another adaptive adjustment, if there are overlapping areas among multiple candidate areas, the multiple candidate areas can be divided into first category candidate areas, second category candidate areas and third category candidate areas according to the number of overlapping areas between the candidate areas and other candidate areas, wherein the first category candidate areas refer to: the candidate areas under this category overlap with only one other candidate area; the second category candidate areas refer to: the candidate areas under this category overlap with two other candidate areas; the third category candidate areas refer to: the candidate areas under this category overlap with three or more other candidate areas.

[0113] Among them, when performing adaptive adjustment, each type of candidate area can be iteratively adjusted according to the adjustment mode corresponding to each type of candidate area. The iteration here can mean: first adjust the first type of candidate area, and then adjust the second type of candidate area based on the adjustment, and then adjust the third type of candidate area based on the adjusted second type of candidate area, so as to finally adjust all the candidate areas with overlapping areas to obtain the calibrated area.

[0114] For example, Figure 5b As shown, one of the adjustment methods is given, wherein candidate areas 501-505 overlap, wherein candidate area 505 and candidate area 504 have overlapping area b1, candidate area 504 and candidate area 501 have overlapping area b2, candidate area 501 and candidate area 503 have overlapping area b3, candidate area 501 and candidate area 502 have overlapping area b4, wherein the first category of candidate areas are candidate areas 505, 502 and 503; the second category of candidate areas is candidate area 504, and the third category of candidate areas is candidate area 501.

[0115] First, the first type of candidate regions are adjusted to candidate regions 505, 502, and 503. Specifically, the overlapping region b1 can be allocated to candidate region 505, the overlapping region b3 can be allocated from candidate region 503, and the overlapping region b4 can be allocated to candidate region 502. In this way, the remaining overlapping region b2 is as follows: Figure 5b As shown, the overlapping area b2 can be allocated to the candidate area 504, and since the candidate area 503 allocates the overlapping area b3, when the candidate area 501 is adjusted, the overlapping area b3 can be allocated to the candidate area 501, thereby obtaining the adjusted candidate area.

[0116] Of course, in practice, other adaptive adjustment methods may be used, such as adjusting according to the position order of the candidate regions on the simulated image, such as adjusting from left to right, or from right to left, which is not limited here.

[0117] Among them, the adjustment of the candidate area may not only be limited to the adjustment of the candidate area when there are overlapping areas between the candidate areas, but may also include the adjustment of the shape of the candidate area. For example, in some cases, the shape of the candidate area may be changed according to actual needs. For example, for multiple candidate areas from the same scene image (a scene image is drawn with multiple drawing areas), the user may change the shape of the multiple candidate areas, for example, splicing multiple candidate areas into one area, thereby causing the shape and size of the candidate areas to change.

[0118] In some embodiments, the compatibility of the adjusted candidate region with the scene image captured by the first image capture device can be ensured, and the compatibility can mean: ensuring that the adjusted candidate region is within the capture range of the first image capture device and does not deviate from the original candidate region. Specifically, the adjusted candidate region can be returned to the scene image for display, and specifically, the adjusted candidate region is mapped back to the scene image for display, so that the user can check whether the adjusted candidate region meets the above-mentioned compatibility.

[0119] During specific implementation, the adjusted candidate region may be mapped to the scene image from which the adjusted candidate region originates for display.

[0120] The adjusted candidate region refers to a candidate region on which a region adjustment operation is performed on a simulated image, or an adaptive adjustment is performed by the first electronic device, such as Figure 3 As shown, assuming that the candidate region 301 is adjusted in the simulated image, it is necessary to map the adjusted region to Figure 3 The scene image shown is displayed so that the adjusted area can be compared with the original drawing area 301 in the scene image.

[0121] In some specific examples of this embodiment, a drawing area corresponding to the area drawing operation is displayed in the scene image from which the adjusted candidate area originates; specifically, since the scene image can be displayed on the first image drawing interface and the simulated image is displayed on the second image drawing interface, wherein the adjustment of the candidate area occurs on the second image drawing interface, in practice, the adjusted candidate area can be returned to the first image drawing interface, and the first image drawing interface maps the adjusted candidate area on the scene image.

[0122] In order to distinguish the drawing area from the adjusted candidate area (hereinafter referred to as the adjusted drawing area) mapped on the scene image, the drawing area and the adjusted drawing area can be displayed on different layers respectively, and both are displayed with a certain transparency, so that the user can determine whether the adjusted drawing area is within the drawing area.

[0123] Among them, the drawing area is the candidate area when it is mapped to the simulation image. In this way, the drawing area and the adjusted drawing area (corresponding to the adjusted candidate area in the simulation image) can be compared on the scene image. If the adjusted drawing area is within the drawing area, it indicates that its adjustment has not deviated from the original candidate area and is within the original acquisition range; if there is an intersection between the drawing area and the adjusted drawing area, the adjusted drawing area is not completely within the drawing area. If it is not completely within the drawing area, the adjusted drawing area will contain some invalid information, affecting the subsequent image analysis and processing. Therefore, it is necessary to adjust the candidate area again on the simulation image.

[0124] The adjusted candidate area mapped to the source scene image for display may include: the area after the user performs an area adjustment operation, and the candidate area after the first electronic device performs adaptive adjustment. In other words, whether the user adjusts the candidate area or the first electronic device adjusts the candidate area adaptively, the adjusted candidate area can be returned to the scene image as the adjusted drawing area for display.

[0125] Among them, if the adjusted drawing area is not completely within the drawing area, the adjusted drawing area will contain some invalid information. In this case, the candidate area needs to be adjusted again on the simulated image. Accordingly, when a modification operation on the adjusted candidate area in the scene image is received, the area corresponding to the modification operation can be displayed in the simulated image, and the step of determining the calibration areas corresponding to each of the multiple first image acquisition devices based on the candidate areas corresponding to each first image acquisition device is repeated.

[0126] That is to say, the adjusted candidate area can continue to be adjusted in the second image drawing interface, and after each adjustment, the adjusted candidate area can be returned to the first image drawing interface, so that the adjusted drawing area corresponding to the adjusted candidate area can be displayed in the scene image of the first image drawing interface to view the adjustment effect.

[0127] Accordingly, when the adjusted candidate area is mapped to the scene image from which the adjusted candidate area originates for display, the calibration area can be determined in the following manner: when a determination operation for the adjusted candidate area is received, the adjusted candidate area is used as the calibration area of ​​the corresponding first image acquisition device.

[0128] Among them, the determination operation can occur on the first image drawing interface or on the second image drawing interface. When it occurs on the first image drawing interface, the current adjusted drawing area can be used as the calibration area of ​​the first image acquisition device. Since the scene image is displayed on the first image drawing interface, the coordinates of the adjusted drawing area are the image coordinates, and the coordinates of the adjusted drawing area in the scene image can be directly bound to the first image acquisition device.

[0129] When a confirmation operation occurs on the second image drawing interface, the adjusted candidate area can be used as a calibration area, and the coordinates of the calibration area on the simulated image are bound to the corresponding first image acquisition device.

[0130] It should be noted that when adjusting multiple candidate areas with overlapping areas, the adjusted candidate areas will become calibration areas. In some embodiments, the calibration areas need to meet the condition of not overlapping each other. In some other embodiments, the calibration areas need to be able to be spliced ​​with each other on the basis of not overlapping each other to completely cover the simulated image.

[0131] Accordingly, as mentioned above, in some scenarios, it is not required that multiple calibrated areas be spliced ​​together to completely cover the simulated image; however, in some scenarios, it is required that multiple calibrated areas be spliced ​​together to completely cover the simulated image. In such scenarios, such as shopping malls, it is necessary to count the customer flow of the mall, so it is necessary to identify the guests in the mall as completely as possible.

[0132] Accordingly, in order to completely cover the simulated image after multiple calibrated areas are spliced ​​together, the uncovered area can be displayed in the simulated image; in practice, a second image acquisition device can be added to capture the image of the uncovered area through the second image acquisition device, thereby achieving full-range image acquisition of the target scene.

[0133] The uncovered area may include an area not covered by the selected area, or an area not covered by the marked area.

[0134] Specifically, based on the uncovered area, the installation position of the second image acquisition device in the target scene can be determined, and part or all of the uncovered area can be used as the calibration area of ​​the image acquisition device; wherein the second image acquisition device is used to be added to the target scene to capture images of the uncovered area.

[0135] In some scenarios, even if the calibrated areas of multiple first image acquisition devices are spliced ​​together to completely cover the simulated image, as time goes by, if the target scene expands, for example, the store expands from the original 100 square meters to 200 square meters, the original first image acquisition device can only cover an area of ​​100 square meters, and the newly added area of ​​100 square meters requires a second image acquisition device to cover the newly added area of ​​100 square meters.

[0136] In some embodiments, after obtaining the candidate area on the simulated image based on the area drawing operation, the area not covered by the candidate area can be displayed on the simulated image. In this case, the second image acquisition device for the uncovered area can be deployed in the target scene first, and then the calibration area corresponding to the second image acquisition device can be determined according to the process of the above steps S201-S203. In this embodiment, the on-site workers can divide the calibration area containing valid information while ensuring that the target scene is fully covered by the image acquisition range of the first image acquisition device and the second image acquisition device, so that the calibration area of ​​each image acquisition device can be determined at one time. This embodiment can be adapted to the situation where the area of ​​the above target scene is expanded.

[0137] In some other embodiments, after determining the calibration areas corresponding to the plurality of first image acquisition devices, the area not covered by the calibration area can be displayed on the simulated image. In this case, the second image acquisition device can be directly arranged in the target scene. After determining that the acquisition range of the second image acquisition device includes the range where the calibration area is located, it is not necessary to determine the calibration area of ​​the second image acquisition device, and the uncovered area can be directly used as the calibration area of ​​the second image acquisition device. In this embodiment, the target scene can be made according to the image analysis requirements. After the calibration area is determined, if an image acquisition device needs to be added later, it is not necessary for the worker to perform the area drawing operation on the scene image collected by the second image acquisition device, thereby simplifying the work of determining the calibration area when adding the second image acquisition device, so as to cope with the need to increase the calibration area due to the expansion of the area of ​​the target scene. This embodiment can be applicable to the situation where the original first image acquisition device of the target scene does not cover the entire area of ​​the target scene, and it is necessary to identify the objects to be identified in the entire area of ​​the target scene.

[0138] Among them, it is necessary to add a second image acquisition device in the target scene to acquire scene images of the uncovered area. In practice, it is necessary to determine the installation position of the second image acquisition device in the target scene so that the image acquisition range of the second image acquisition device at this installation position can include the uncovered area.

[0139] The installation posture generally requires on-site workers to continuously debug according to the images collected by the second image acquisition device. The installation posture includes the installation position and posture of the second image acquisition device. For example, for a ball camera, it includes not only the position but also the rotation angle of the ball camera.

[0140] In a specific implementation of this embodiment, a target image can be obtained by acquiring the target scene by the second image acquisition device at the candidate position; and the scene area included in the target image is mapped to the simulation image to obtain the acquisition area; then, based on the acquisition area and the uncovered area, position adjustment information is output; based on the position adjustment information, the installation posture is determined.

[0141] In some embodiments, the candidate position can be the installation position of the second image acquisition device predicted by the electronic device based on the position of the uncovered area in the simulated image, and then the image acquisition range of the second image acquisition device at the installation position can be manually determined on site to see whether it can cover the uncovered area. In this way, the workload of workers in debugging the position of the newly added image acquisition device on site can be reduced.

[0142] The output position adjustment information may be a comparison result between the acquisition area and the uncovered area, or a display of the acquisition area and the uncovered area on a simulated image; the output position adjustment information may be directly located in the electronic device, or may be sent to on-site workers so that the on-site workers can view the position adjustment information and adjust the installation position of the second image acquisition device until it is determined that the acquisition area covers the uncovered area based on the position adjustment information.

[0143] In an example of this specific implementation, the output position adjustment information may be information indicating how to adjust the candidate position according to the comparison result between the acquisition area and the uncovered area.

[0144] In a specific implementation, it is possible to determine the coverage type of the acquisition area covering the uncovered area; based on the coverage type, generate position adjustment information; and then send the position adjustment information to the user terminal.

[0145] In this case, the position adjustment information may include the moving direction and / or moving distance of the candidate position of the second image acquisition device. The position adjustment information corresponds to the coverage type. In practice, the preset adjustment information corresponding to each coverage type may be sent to the user terminal as the position adjustment information. The preset adjustment information may be pre-stored in the electronic device for easy retrieval.

[0146] The coverage type may include three different types: the acquisition area fully covers the uncovered area, the acquisition area partially overlaps the uncovered area, and the uncovered area fully covers the acquisition area; each type corresponds to a preset adjustment information.

[0147] For example, in the type where the acquisition area fully covers the uncovered area, no position adjustment information is required, and position adjustment information of 0 can be output, indicating that the current candidate posture is the correct posture; in the type where the acquisition area partially overlaps with the uncovered area, position adjustment is required. In this case, the corresponding preset adjustment information can be determined according to the position of the overlapping area where the acquisition area partially overlaps with the uncovered area on the simulated image. Different positions correspond to different preset adjustment information. In this case, the preset adjustment information may include the moving direction and the moving distance, so as to indicate the direction in which the second image acquisition device is to be installed and posture debugged; in the type where the uncovered area fully covers the acquisition area, the height of the second image acquisition device needs to be adjusted to expand its image acquisition range. In practice, the preset adjustment information may indicate to raise the installation height of the second image acquisition device.

[0148] For example, refer to Figure 6 As shown, Figure 1 Taking the application scenario of as an example, a flow chart of this implementation method is shown, as shown in Figure 6As shown, the acquisition area 601 partially overlaps with the uncovered area 602, and the overlapping area is toward the lower right of the simulated image compared to the uncovered area 601. In practice, the second image acquisition device 102 needs to move the acquisition range to the upper left, thereby adjusting the actual installation position of the second image acquisition device in the target scene.

[0149] By adopting the method for determining the installation posture of the present embodiment, the first electronic device can directly predict a candidate position according to the position of the uncovered area on the simulated image, and then the on-site worker first places the second image acquisition device on the candidate position. The area covered by the image captured by the second image acquisition device will be directly mapped back to the simulated image. The first electronic device can automatically compare the coverage of the mapped back area with the uncovered area (including full coverage, partial coverage, etc.), and then output the comparison result and the position adjustment strategy. The position adjustment strategy can be directly fed back to the on-site worker through communication. After the on-site worker adjusts the installation posture according to the position adjustment measurement, the above steps are continued until the acquisition range of the second image acquisition device fully covers the uncovered area. The first electronic device sends an adjustment success message, such as "0", to the on-site worker. In this case, the on-site worker can directly fix the second image acquisition device on the last determined installation posture, thereby avoiding the problem that the on-site worker needs to go back and forth between the first electronic device and the installation posture to continuously determine the acquisition range when installing the second image acquisition device, so that a single on-site worker can easily complete the installation of the newly added image acquisition device, saving manpower expenditure, and providing a more convenient and reliable way to install new equipment for the on-site worker.

[0150] In some other examples of this specific implementation, the output position adjustment information may be a display screen of the acquisition area and the uncovered area on a simulated image. In a specific implementation, a screen capture may be performed on the simulated image displaying the acquisition area and the uncovered area; and the captured screen may be sent to the user end as the position adjustment information.

[0151] In this example, the on-site workers can hold a portable display device, so that the captured images can be sent to the on-site workers, so that the on-site workers can adjust the installation posture of the second image acquisition device according to the captured images, wherein the image capture can be real-time capture, specifically, the second image acquisition device returns the captured scene image to the second image drawing interface in real time, the second image drawing interface maps the entire mapping of the scene image to the simulation image, and only the uncovered area can be displayed on the simulation image, and the first electronic device records the display screen of the second image drawing interface, and feeds back the recorded screen to the portable display device in real time, so that the on-site workers can observe the changes in the scene image captured during the installation posture adjustment process in real time, so as to quickly determine the correct installation posture.

[0152] In this example, the workers can install and debug the second image acquisition device on site, and then determine the coverage area of ​​the image acquired by the second image acquisition device on the simulated image based on the received picture, and determine whether the installation posture of the second image acquisition device is correct. Specifically, if the covered area on the simulated image covers the uncovered area, then it means that the image acquisition range of the second image acquisition device covers the uncovered area, and the installation posture is correct; if the covered area on the simulated image does not cover the uncovered area, then it means that the image acquisition range of the second image acquisition device can fully cover the uncovered range, and the installation posture needs to be adjusted.

[0153] By adopting the method of determining the installation posture of the present embodiment, the on-site workers can directly select a candidate position to install the second image acquisition device, and then instruct the second image acquisition device to acquire images of the target scene at the candidate position. After that, the entire image area of ​​the target image acquired by the second image acquisition device is mapped to the simulation image for display. In this case, only the uncovered area and the area mapped back by the target image can be displayed on the simulation image, so as to facilitate the comparison of whether the uncovered area is within the range covered by the target image; since the screen displaying the simulation image, the selected area and the uncovered area can also be shared with the display device held by the on-site workers, the on-site workers can hold a simple display device at the location where the second image acquisition device is installed to quickly complete the position determination of the second image acquisition device. Therefore, the operation of the on-site workers can be simplified, and visual and operable hardware and software facilities are provided for the workers who perform on-site area calibration, thereby improving the efficiency of area calibration at work.

[0154] Among them, the process of mapping the drawing area drawn by the area drawing operation in the scene image captured by the image acquisition device (including the first image acquisition device and the second image acquisition device) to the simulated image to become a candidate area is explained in the present application.

[0155] In some embodiments, in response to the area drawing operation, a drawing area selected by the area drawing operation can be determined on the scene image; then, according to the conversion relationship between the image coordinate system of the image acquisition device and the image coordinate system of the simulated image, the drawing area is mapped to the simulated image to obtain candidate areas; and each candidate area is displayed on the simulated image.

[0156] In a specific implementation, in response to an area drawing operation, first area coordinates of a drawing area selected by the area drawing operation can be determined on a scene image; and the first area coordinates can be converted into second area coordinates on a simulated image; then, based on the second area coordinates, the image area can be mapped onto the simulated image to obtain a candidate area.

[0157] In this implementation, the first area coordinates are based on the image coordinate system of the scene image, that is, based on the coordinate system of the first image acquisition device, and the second area coordinates are based on the coordinate system of the simulated image. Generally speaking, the simulated image is a scaled-down image of the target scene, which is obtained based on the world coordinate system and the proportional relationship.

[0158] The first region coordinates include the coordinates of multiple endpoints of the drawing region, and in some cases, may also include the coordinates of the region center of the drawing region, which may be determined specifically according to the shape of the drawing region. The second region coordinates correspond to the first region coordinates, and if the first region coordinates include the coordinates of multiple endpoints, the second region coordinates also include the coordinates of the multiple endpoints on the simulated image.

[0159] The mapping relationship between the first region coordinates and the second region coordinates may be a conversion relationship between an image coordinate system and a simulated image coordinate system.

[0160] As described above, the scene image is generally based on the image coordinate system of the image acquisition device, while the simulation image is based on the coordinate system provided by the engineering drawing software when drawing the target scene. The conversion between the two can be carried out with the world coordinate system as the conversion intermediary. Specifically, the world coordinate system can be understood as the coordinate system of the real world. In this embodiment, a coordinate conversion relationship between the image coordinate system and the world coordinate system can be constructed, as well as a conversion relationship between the world coordinate system and the coordinate system of the simulation image can be constructed. With the world coordinate system as the coordinate system intermediary, coordinate conversion is realized between the image coordinate system of the image acquisition device and the coordinate system of the simulation image, so that the image area in the scene image is mapped to the simulation image based on this conversion relationship.

[0161] In practice, when constructing the coordinate transformation relationship between the image coordinate system and the world coordinate system, the image acquisition device located in the target scene can be calibrated first. Specifically, the calibration can be performed by obtaining the position parameters of the image acquisition device in the target scene, and the position parameters are mainly used to determine the coordinate transformation relationship between its image coordinate system and the world coordinate system.

[0162] In a specific implementation, the above-mentioned position parameters can be obtained by setting a plurality of marking icons in the target scene, and the plurality of marking icons can be used to determine the eight position parameters.

[0163] Reference Figure 7 As shown, a schematic diagram of the process flow of determining the coordinate transformation parameters in this embodiment is shown, such as Figure 7 As shown, the following steps may be specifically included:

[0164] Step S701: Acquire a marked image acquired by an image acquisition device.

[0165] Wherein, the marked image includes images of multiple marked icons located in the target scene; and the image acquisition device includes the first image acquisition device and the second image acquisition device.

[0166] In this embodiment, the image acquisition device is located in the target scene, and it can be a first image acquisition device or a second image acquisition device, which is not limited here.

[0167] Among them, the marking icon can be icons such as numbers, letters, graphics, etc. printed on a white-backed cardboard. In practice, when identifying the marking icon, icons such as numbers, letters, graphics, etc. can be identified.

[0168] Among them, when each coordinate conversion relationship between an image acquisition device and the world coordinate system is determined, multiple marking icons can be placed within the image acquisition range of the image acquisition device; or, the placement positions of the multiple marking icons can enable multiple image acquisition devices to capture them at the same time, that is, the multiple marking icons are located within the acquisition range of multiple image acquisition devices.

[0169] Step S702: respectively determining the image coordinates of the plurality of mark icons in the mark image, and determining the world coordinates of the plurality of mark icons.

[0170] Among them, the image coordinates of multiple mark icons in the mark image can be identified. Specifically, the image coordinates of icons such as numbers, letters, and graphics in the mark icons can be identified. The specific identification method can refer to the relevant technology and will not be repeated here.

[0171] Among them, the world coordinates of the multiple mark icons can be obtained by on-site workers measuring on-site, reflecting the real coordinate positions of the multiple mark icons in the target scene. Specifically, the real coordinate position can be a three-dimensional coordinate position, but the height coordinate in the three-dimensional coordinate position can be set to 0 or 1.

[0172] Step S703: Based on the image coordinates and the world coordinates, determine the first coordinate conversion parameters corresponding to each of the image acquisition devices.

[0173] In this embodiment, based on the image coordinates of each of the multiple marking icons in the marking image and their world coordinates in the world coordinate system, the first coordinate conversion parameters can be calculated using the method of converting the camera image coordinate system to the world global coordinate system in related technologies, where the first coordinate conversion parameters can include eight position parameters.

[0174] Step S704: Determine a second coordinate conversion parameter based on the world coordinate system and the coordinate system of the simulated image.

[0175] The simulated image is a plane engineering drawing or a three-dimensional stereogram of the target scene in equal proportion, so the second coordinate transformation parameter can be obtained according to the proportional relationship.

[0176] For example, taking a plane engineering drawing as an example, the upper left corner of the engineering drawing is agreed to be the origin of the coordinate system, the horizontal direction is the x direction, and the vertical direction is the y direction. The origin of the coordinate system, the x-axis, and the y-axis directions correspond to the positions on the ground in the real world, which are the origin and coordinate system in the world coordinate system.

[0177] Assume the coordinates in the world coordinate system are (x w1 ,y w1 ), the entire area is H meters high (y direction) and W meters wide (x direction), and the corresponding coordinates in the engineering drawing coordinate system are (x w2 ,y w2 ), height h meters, width w meters.

[0178] but:

[0179] Therefore, according to formula (1), the coordinates of an object in the target scene in the simulated image when it is mapped into the simulated image can be obtained.

[0180] According to the first coordinate conversion parameter and the second coordinate conversion parameter, the drawing area selected by the area drawing operation on the scene image can be mapped to the candidate area in the simulation image. Specifically, the world coordinates of the drawing area in the world coordinate system can be obtained according to the first coordinate conversion parameter, that is, the first area coordinates can be converted into world area coordinates according to the first coordinate conversion parameter; then, according to the second coordinate conversion parameter, the world coordinates of the drawing area are converted into coordinates on the simulation image, that is, the world area coordinates are converted into second area coordinates on the simulation image, thereby obtaining the candidate area on the simulation image.

[0181] Among them, after obtaining the first coordinate conversion parameter and the second conversion parameter, the first coordinate parameter is stored in association with the corresponding image acquisition device, and the second conversion parameter is stored in association with the corresponding simulated image, thereby constructing: a coordinate conversion relationship between the image acquisition device (the first image acquisition device and the second image acquisition device)-the target scene-the simulated image, so that it can be directly called when performing subsequent area mapping.

[0182] Among them, after multiple image acquisition devices are installed in the target scene (including the first image acquisition device and the second image acquisition device), the installation posture of one or more of the image acquisition devices may change due to some circumstances, causing their image acquisition range to deviate from the original calibrated range. In this case, since the image acquisition range changes, the original image coordinate system is changed. If the scene area is selected based on the original calibrated area, different areas will be selected, resulting in invalid information in the subsequent image processing and analysis.

[0183] Therefore, it is necessary to redefine the first coordinate transformation parameters.

[0184] Among them, when re-determining the first coordinate conversion parameters of the image acquisition device whose position has changed, the contents of the above steps S701-S703 can be used. That is to say, the positions of multiple marker icons can be re-placed so that they are within the acquisition range of the image acquisition device after the position change. Of course, in some embodiments, if the positions of multiple marker icons are still within the acquisition range of the image acquisition device after the position change; the world coordinates of the multiple marker icons can be used, and the first coordinate conversion parameters can be determined by only redetermining the image coordinates of the multiple marker icons in the scene image.

[0185] In a specific implementation, the world coordinates corresponding to each of the multiple calibration points can be stored; and when it is detected that the position and posture of the image acquisition device has changed, the image acquisition device with the changed position and posture is controlled to re-capture images of the multiple calibration points; then, based on the image coordinates of the multiple calibration points in the re-captured marking image and the world coordinates corresponding to each of the stored multiple calibration points, the first coordinate conversion parameters are updated; thereafter, the first coordinate conversion parameters corresponding to the image acquisition device with the changed position are updated to the updated first coordinate conversion parameters.

[0186] Among them, the image acquisition device whose posture changes can be called a third image acquisition device in this embodiment. For example, if the posture of a first image acquisition device changes, the first image acquisition device is called a third image acquisition device, and if the posture of the second image acquisition device changes, the second image acquisition device is called a third image acquisition device.

[0187] Among them, in some embodiments, when the posture of the first image acquisition device changes, if it is determined that the image acquisition range corresponding to the first image acquisition device after the posture change includes the area selected by the original calibration area, the coordinates of the original calibration area can be transformed based on the existing image coordinates and the original image coordinates of the first image acquisition device to obtain a new calibration area of ​​the first image acquisition device, and then the new calibration area is bound to the first image acquisition device.

[0188] In some other embodiments, since the calibration area is used to indicate the area on the scene image where the object to be identified needs to be identified, the coordinate system it uses can be the image coordinate system of the image acquisition device. Since there is a first coordinate conversion parameter between the image coordinate system and the world coordinate system, the calibration coordinates of the calibration area in the world coordinate system can be obtained, that is, the position of the calibration area in the real world can be obtained. In this way, when the posture of the third image acquisition device changes, as long as its image acquisition range still includes the area framed by the calibration area, the world coordinate system can be used to build a bridge between the image coordinate systems of the third image acquisition device before and after the posture change, that is, based on the world coordinate system, the change of the calibration area of ​​the third image acquisition device before and after the posture change is obtained, so as to ensure that the calibration area can correspond to the same area in the real scene before and after the posture change of the third image acquisition device, without the need to recalibrate the changed third image acquisition device.

[0189] When implementing it, refer to Figure 8 FIG. 4 is a flow chart showing the steps of adjusting the calibration area after the position of the third image acquisition device changes, as shown in FIG. Figure 8 As shown, the following steps may be specifically included:

[0190] Step S801: based on the first coordinate conversion parameter, the calibration coordinates of the calibration area associated with the image acquisition device in the world coordinate system may be determined.

[0191] In this embodiment, since the calibration area has image coordinates in the image acquisition device, the calibration coordinates of the calibration area in the world coordinate system can be obtained by using the first coordinate conversion parameter, wherein the calibration coordinates may include the coordinates of multiple endpoints of the calibration area, and in some cases, may also include the coordinates of the center point of the calibration area. Thus, the actual area of ​​the calibration area in the target scene can be determined.

[0192] Step S802: when a third image acquisition device with a changed posture is detected, if it is determined that the scene image acquired by the third image acquisition device includes an associated calibration area, the first coordinate conversion parameter corresponding to the third image acquisition device is updated.

[0193] Among them, the process of updating the first coordinate conversion parameters corresponding to the third image acquisition device can refer to the above process, that is, based on the image coordinates of multiple calibration points in the re-captured marked image and the world coordinates corresponding to the stored multiple calibration points, the first coordinate conversion parameters are updated.

[0194] Step S803: determining the calibration area corresponding to the third image acquisition device after the position change based on the updated first coordinate conversion parameter and the calibration coordinates corresponding to the initial calibration area;

[0195] Step S804: Associating the calibration area corresponding to the position change with the third image acquisition device.

[0196] After the first coordinate transformation parameter of the third image acquisition device is updated, a new transformation relationship between the image coordinate system of the third image acquisition device after the posture change and the world coordinate system is established. Therefore, based on the new transformation relationship, the actual area where the calibration area is located in the target scene can be re-mapped back to the scene image captured by the third image acquisition device after the posture change, thereby obtaining the calibration area in the newly captured scene image.

[0197] Among them, the calibration area corresponding to the third image acquisition device after the posture change and the calibration area corresponding to it before the position change are actually still the same area in the target scene, but its shape or position in the scene image captured by the third image acquisition device after the posture change is different, but it contains valid information and contains no or less invalid information.

[0198] Among them, posture change can refer to position change, angle change, etc. Anything that can affect the image acquisition range can be called posture change.

[0199] When the position of the image acquisition device changes, the conversion relationship between the image coordinate system of the changed image acquisition device and the world coordinate system can be obtained based on the first coordinate conversion parameter obtained after the change. The calibration area generally represents a fixed area in the real scene after being calibrated. Therefore, when the position of the image acquisition device changes, the position, size and shape of the fixed area on the captured image will also change.

[0200] In this embodiment, the fixed area in the real scene can be directly mapped back to the scene image collected after the position changes according to the updated first coordinate conversion parameter, and the new size and new position of the calibration area in the scene image collected after the position changes are obtained, so that the calibration area before the position changes is migrated to the scene image collected after the position changes. This avoids manual recalibration of the calibration area for the image acquisition device with a changed position, because the recalibration needs to be performed by on-site workers, thereby saving labor costs when no recalibration is required.

[0201] In some embodiments, the area drawing operation may include touch and click operations on multiple location points on the scene image, and may also include line drawing operations on the scene image, wherein the touched or clicked location points can be understood as the endpoints of the area to be selected, and the drawn lines can be understood as the edges of the area to be selected.

[0202] In a specific implementation, a closed image area can be generated on the scene image in response to multiple position points where preset operations are performed on the scene image, and / or multiple lines drawn on the scene image; then, the candidate area to which the closed image area is mapped is displayed in the simulated image.

[0203] In a specific implementation of this embodiment, a single click or touch of a position point and a line drawing can be performed simultaneously on the scene image. In this case, an image area can be fitted based on multiple triggered position points and drawn lines.

[0204] In a specific implementation of this embodiment, only a single click or touch of a position point may be performed on the scene image. In this case, the multiple position points may be connected into a closed image area using preset lines based on the multiple triggered position points.

[0205] In a specific implementation of this embodiment, only line drawing operations may be performed on the scene image. In this case, multiple lines may be smoothed and processed based on the multiple lines drawn, so as to fit a closed image area.

[0206] When this implementation is adopted, it is convenient for on-site workers to draw a drawing area on the scene image, so as to optimize the area drawing experience of the on-site workers.

[0207] Below, the area calibration method disclosed in the present invention is fully described through an example A:

[0208] by Figure 1 For example, the specific process includes the following:

[0209] When a furniture store needs to conduct customer flow statistics, on-site workers arrive at the store to perform calibration. They can carry a spare camera and four marking icons. The spare camera is used to add a new camera to the store. The calibration work is assisted by a first electronic device. A system can be running in the first electronic device. The system can include an area calibration module for calibrating the calibration area of ​​the cameras in the store.

[0210] S1: Start calibrating the coordinates of each camera in the store; specifically including:

[0211] S11: placing four marking icons within the range that camera i can capture images, then obtaining the image captured by camera i, and starting to measure the world coordinates of the marking icons at the placement positions; wherein camera i is any camera among multiple cameras, i is greater than or equal to 1 and less than or equal to the total number of cameras;

[0212] S12: input the acquired image into the neural network model to obtain the image coordinates of the four marking icons in the image, and obtain a total of 4 image coordinates;

[0213] S13: using the four image coordinates and the four world coordinates, according to the existing method of converting the camera image coordinate system to the world global coordinate system, calculate the conversion matrix parameters based on the image coordinates to the world coordinates, that is, the first coordinate conversion parameters, which include eight position parameters and are stored in the coordinate conversion file corresponding to the camera i;

[0214] S14: simulating a simulated image of a furniture store, which may be a plane engineering drawing of the furniture store, and which is proportionally scaled with the real furniture store, thereby obtaining a coordinate transformation relationship between the coordinate system of the plane engineering drawing and the world coordinate system, i.e., a second coordinate transformation parameter;

[0215] Thus, the calibration of all cameras is completed, and the on-site workers can upload the first coordinate conversion parameters and the second coordinate conversion parameters to the system to be used by the subsequent system for regional calibration. The system can be located in the computer equipment of the furniture store, can be downloaded in advance, or can be downloaded by the on-site workers.

[0216] S2: Import the cameras in the store into the system so that the system can obtain and display the scene images taken by the cameras.

[0217] S3: Displaying the scene image captured by camera i in a system interface, specifically, a first image drawing interface specially drawn for the scene image, and then the on-site worker draws an area excluding invalid information from the scene image according to the drawing tools provided by the first image drawing interface;

[0218] S4: The system automatically maps the drawn area to the plane engineering drawing according to the first coordinate transformation parameter and the second coordinate transformation parameter corresponding to the camera i, and obtains the candidate area i located in the plane engineering drawing;

[0219] After the above steps S3 and S4, candidate areas of all cameras in the plane engineering drawing can be obtained. Assuming there are N cameras in total, there are N candidate areas.

[0220] S5: In the second image drawing interface, the candidate regions on the plane engineering drawing are adjusted. Specifically, if multiple candidate regions among the N candidate regions overlap, the shapes or sizes of the overlapping candidate regions are adjusted so that none of the multiple candidate regions overlap.

[0221] S6: Mapping the adjusted candidate area i back to the first image drawing interface and displaying it in the scene image of camera i. If it is determined that the adjusted candidate area i is still located in the scene image of camera i and does not include invalid information, the adjusted candidate area i is used as the calibration area of ​​camera i; otherwise, continue to execute steps S4 and S5;

[0222] S7: Determine whether the calibration areas of the existing N cameras of the store fully cover the engineering plan;

[0223] If yes, go to step S8, if no, go to steps S9-S10:

[0224] S8: After the calibration work is completed, the calibration area corresponding to each camera can be saved, such as returned to the camera, so that the scene image sent by the camera carries the image coordinates of the calibration area to indicate that only the calibration area is identified; or, it can be stored in the system and called when the image collected by the camera arrives.

[0225] S9: In the second image drawing interface, the uncovered area not covered by the calibrated area is displayed, and the on-site worker pre-installs a camera k (from a backup camera carried or a camera provided by the merchant) at a candidate position S in the store, displays the image captured by the camera k in the first image drawing interface, and maps all the areas included in the image to the engineering plan;

[0226] The system takes a screenshot of the second image drawing interface, the screenshot includes the image area captured by camera k and the uncovered area, and then displays the screenshot to a portable display device carried by the on-site worker;

[0227] If it is determined that the image area captured by camera k covers the uncovered area, camera k is installed at position S; if not, the position of camera K is adjusted, and the above steps are repeated until a specific installation posture is determined;

[0228] S10: After the position of camera K is determined, the uncovered area can be directly returned to camera K as the calibration area of ​​camera K, or the area can be calibrated for camera K according to the process of steps S3-S4 above.

[0229] Of course, generally speaking, according to the needs of the store, it is not necessarily required that the camera covers the entire store area. Therefore, in practice, the S9-S10 process can be omitted, and the merchant’s old cameras can be used directly to complete this new function.

[0230] In this example A, the operating tools provided by the first image drawing interface and the second image drawing interface may include but are not limited to commonly used drawing tools.

[0231] In addition to the above example A, the present disclosure also provides an example B that can realize area calibration. Based on the technical concept of example A, example B proposes another feasible technical solution. The specific solution is as follows:

[0232] First, the scene images captured by the image capture devices located in the target scene can be displayed, wherein, as described above, the image capture devices are used to capture images of a part or all of the area in the target scene;

[0233] Next, the corresponding area to be identified in the scene image may be determined. Specifically, the following steps may be included:

[0234] Identify the type of objects included in the scene image, which may include a valid identification type and an invalid identification type, such as a wall should be identified as an invalid identification type, while a bed, a chair, a sofa, and a ground without furniture should be identified as a valid identification type;

[0235] Afterwards, the areas where multiple objects of the type "valid type for identification" are located are spliced ​​together. If the areas where multiple objects of the type "valid type for identification" are located can be spliced ​​together continuously, then the continuously spliced ​​areas can be used as areas to be identified; if the areas where multiple objects of the type "valid type for identification" are located cannot be spliced ​​together continuously, that is, they cannot be connected, then the areas that can be spliced ​​together continuously can be used as areas to be identified; and the remaining areas that cannot be spliced ​​together are used as another area to be identified. Thus, at least one area to be identified can be obtained in the scene image.

[0236] The area to be identified refers to a valid area within the acquisition range of the image acquisition device, that is, the calibration area mentioned above.

[0237] Next, the coordinates of the area to be identified in the coordinate system of the image device are determined. Specifically, the coordinates of multiple endpoints of the area to be identified in the coordinate system of the image device can be determined. Then, the coordinates of the multiple endpoints are saved as calibration information, and the calibration information is associated with the image acquisition device.

[0238] Afterwards, the calibration information is written into the information processing thread corresponding to the image acquisition device. When the image acquired by the image acquisition device arrives, the area to be processed in the image can be directly extracted according to the calibration information, and then the object to be identified in the extracted area can be identified, such as the position of the object to be identified, and the identified position is mapped to the simulated image for display, thereby obtaining thermal map information.

[0239] This example B does not require human intervention, but only needs to train a model that can recognize the types of objects included in the scene image. When the function of determining the area to be identified is activated, when the scene image arrives, the scene image can be pre-processed and input into the model. Then, according to the output result of the model, the area where the objects of the effective type are located can be spliced ​​to complete the determination of the area to be identified. Among them, the spliced ​​area to be identified does not need to be mapped to the simulated image. When it does not need to be mapped to the simulated image, it can reduce the workload of the computer equipment and the calibration process, thereby improving the efficiency of determining the calibration area.

[0240] Of course, in another implementation of this example B, the spliced ​​area to be identified can also be mapped to the simulated image. At this time, manual intervention can be made to adjust the area to be identified in the simulated image to avoid overlapping of the areas to be identified between multiple image acquisition devices, so as to finally obtain the area to be identified of the image acquisition device according to the adjustment process of the area to be identified in the above example A. By adopting this implementation method, the process of manual drawing in the scene image can be reduced, and the labor cost can be reduced. Of course, the premise is that it is necessary to train which objects are effective types for identification and which objects are invalid types for identification. Although the cost of model training will be incurred in the early stage, it will be convenient for on-site construction in the later stage. When the customer needs to specify the area to be identified for the camera in the target scene, the customer can operate the computer, and the computer automatically determines the calibration area corresponding to each camera according to the configured model, and minimizes the intervention of workers in the on-site construction.

[0241] Of course, if manual intervention is still needed to adjust the candidate area in the simulated image, the adjustment operation can also be performed by the customer himself because it is relatively simple.

[0242] In addition to the above examples A and B, the present disclosure also provides an example C that can realize area calibration. Based on the technical concept of example A, example C proposes another feasible technical solution. The specific solution is as follows:

[0243] First, the scene images captured by the image capture devices located in the target scene can be displayed, wherein, as described above, the image capture devices are used to capture images of a part or all of the area in the target scene;

[0244] Next, a region (drawing region) can be drawn from the scene image, and the region can be mapped to the simulated image to obtain a candidate region displayed in the simulated image;

[0245] Then, the candidate regions are adjusted to obtain calibration regions corresponding to the image acquisition devices;

[0246] Afterwards, the image coordinates of the calibration area in the scene image are used as calibration information and associated with the image acquisition device. After the association, when it is necessary to subsequently identify the object to be identified in the image acquired by the image acquisition device, the following instructions can be given:

[0247] Indicates to identify the objects to be identified in the image area except the calibrated area in the image. That is to say, the calibrated area in this example C can be used to shield the area that needs to be kept confidential. The objects to be identified in the confidential area are not allowed to be identified. In this case, the calibration information of the calibrated area can be written into the information processing thread, and it is marked that the identification is prohibited. In this way, when performing image processing, only the objects to be identified in the area of ​​the image that is not selected by the calibrated area can be identified.

[0248] In addition to the above examples A and B, the present disclosure also provides an example D that can realize area calibration. Based on the technical concept of example A, example D proposes another feasible technical solution. The specific solution is as follows:

[0249] First, the scene images captured by the image capture devices located in the target scene can be displayed, wherein, as described above, the image capture devices are used to capture images of a part or all of the area in the target scene;

[0250] Next, a region (drawing region) can be drawn from the scene image, and the region can be mapped to the simulated image to obtain a candidate region displayed in the simulated image;

[0251] Then, the candidate regions are adjusted to obtain calibration regions corresponding to the image acquisition devices;

[0252] Afterwards, the image coordinates of the calibration area in the scene image are used as calibration information, and the recognition task of the calibration area can be configured at the same time. The recognition task is used to indicate the content of the recognition of the object to be recognized in the calibration area in the image, such as the recognition of the position of a person or the recognition of a person's posture.

[0253] Next, the calibration information of the calibration area and the recognition task corresponding to the calibration area are associated with the image acquisition device. After the association, when it is necessary to recognize the object to be recognized in the image acquired by the image acquisition device in the future, the following instructions can be given:

[0254] The object to be identified in the image area selected by the calibrated area in the image is identified according to the corresponding identification task. For the same image acquisition device, different calibrated areas may correspond to different identification tasks, so that the user can freely draw the corresponding calibrated area according to the identification task; of course, for different image acquisition devices, since the image ranges collected by different image acquisition devices may be different, the calibrated areas in different image acquisition devices may correspond to different identification tasks.

[0255] For example, the marked area in the image acquisition device c1 is indicated to identify the location of the guest, and the marked area in the image acquisition device c2 is indicated to identify the salesperson's harvesting behavior.

[0256] Based on the same inventive concept, the present disclosure also provides an information processing method, which utilizes the calibration area of ​​each image acquisition device determined by the above-mentioned area calibration method, so that only the calibration area in the scene image acquired by the image acquisition device needs to be identified, thereby avoiding the identification of invalid information.

[0257] Reference Fig. 9 As shown, a schematic flow chart of the steps of the information processing method is shown. The method can be applied to a second electronic device. It should be noted that the second electronic device can be located in the target scene or not. Generally speaking, the second electronic device can be a device that needs to identify the object to be identified. The second electronic device can be the same device as the first electronic device or a different device. In the case of different devices, the first electronic device can communicate with the second electronic device and send the calibration areas corresponding to the multiple image acquisition devices to the second electronic device. The second electronic device and the first electronic device can be respectively connected to communicate with the multiple image acquisition devices to execute the information processing method.

[0258] Specifically, Fig. 9 As shown, the following steps are included:

[0259] Step S901: respectively acquiring scene images captured by multiple image acquisition devices of a target scene, and calibration areas corresponding to the multiple image acquisition devices; wherein the calibration areas are determined according to the area calibration method of the first embodiment above;

[0260] Step S902: Identify the object to be identified in the target area of ​​the scene image to obtain target information; wherein the target area is the area selected by the calibration area;

[0261] Step S903: For each target information, information processing corresponding to the scene task is performed, wherein the scene task is a task corresponding to the target scene.

[0262] In this embodiment, multiple image acquisition devices can be located in the target scene, which may include a first image acquisition device and a newly added second image acquisition device; wherein, the multiple image acquisition devices can perform image acquisition tasks after the calibration area is calibrated, and in the image acquisition task, the scene image captured for the target scene can be sent to the second electronic device.

[0263] Among them, since multiple image acquisition devices correspond to their own calibration areas respectively, and the calibration areas are drawn on-site according to needs, they contain no or little invalid information. Therefore, when identifying the objects to be identified in the scene image, only the objects to be identified in the area framed by the calibration area can be identified, thereby avoiding the interference of invalid information in the scene image and increasing the amount of calculation, and improving the calculation efficiency and accuracy.

[0264] The object to be identified may be determined according to the needs of information processing. For example, in passenger flow analysis, the object to be identified may be a human body, and in robot trajectory and distribution density analysis, the object to be identified may be a robot. Of course, in other scenarios, the object to be identified may also be other types, which are not limited here.

[0265] Among them, the scene task is the task required in the target scene. For example, if the target scene is a furniture store, the scene task can be a customer flow analysis task, and the corresponding information processing can be an analysis of the number and density of human bodies in the store.

[0266] Of course, depending on the scene task, the information processing may be different, and the identification of the identification object may also be different. In practice, no matter what the scene task is, since each image acquisition device can calibrate the calibration area according to the scene task, the analysis and processing of invalid information can be avoided when executing the scene task, thereby improving processing efficiency.

[0267] In this embodiment, the method of obtaining the calibration area may include the following two methods:

[0268] Method 1: Determine the calibration area corresponding to the current image acquisition device from a preset calibration area library; wherein the preset calibration area library stores the area coordinates of the calibration areas corresponding to a plurality of image acquisition devices.

[0269] When this method is adopted, the first electronic device can feed back the coordinates of the calibration area corresponding to each image acquisition device in the scene image to the second electronic device, and the second electronic device stores them in a preset calibration area library. Alternatively, the first electronic device directly stores the coordinates of the calibration area corresponding to each image acquisition device in the scene image in the preset calibration area library, and the preset calibration area library can be shared by the first electronic device and the second electronic device. As a result, the second electronic device can directly retrieve the calibration area corresponding to the current image acquisition device from the preset calibration area library.

[0270] It should be noted that the current image acquisition device may be a first image acquisition device or a second image acquisition device.

[0271] Method 2: Determine the calibration area based on the area coordinates carried in the image captured by the image acquisition device.

[0272] When the second method is adopted, the first electronic device can feed back the coordinates of the calibrated area corresponding to each image acquisition device in the scene image to the image acquisition device, so that when the image acquisition device acquires a scene image, the coordinates of the calibrated area in the scene image are carried in the image file, and the second electronic device parses the coordinates from the image file to obtain the framed target area, and then identifies the object to be identified in the target area.

[0273] Among them, the above-mentioned method 1 and method 2 can be used in combination. If the calibration area is not obtained from the preset calibration area library, it is obtained from the area coordinates carried by the image. In other words, the coordinates of the calibration area in the scene image can be carried in the scene image, or stored in the preset calibration area library for the second electronic device to call.

[0274] In some embodiments, the target information may include the location information of the object to be identified, and the scene task may include: a location display task and / or an object counting task. When performing information processing corresponding to the scene task on each location information, the following processing may be included:

[0275] Processing 1: When the scene task is a position display task, each position information is mapped to a simulated image of a simulated target scene for display;

[0276] In this processing, the position of the object to be identified in the target area can be identified to obtain position information, which can be the position of the object to be identified in the scene image. In practice, the position information of the object to be identified can be displayed in the simulated image, and the position of the object to be identified in the scene image can be mapped to the simulated image for display.

[0277] In one example, a third drawing interface may be provided, in which a simulated image of a target scene may be displayed. The third drawing interface provides a variety of image display functions. Different image display functions may perform different processes on the location information, such as performing the above-mentioned process one and process two, and displaying the processed results in a simulated image in the third drawing interface. Specifically, multiple layers may also be set in the third drawing interface, wherein the simulated image is located in one layer, and the processed results may be displayed in another layer.

[0278] By adopting the processing method 1, the position of the object to be identified in each target area can be directly displayed in the simulated image, and the position distribution of the object to be identified in each area of ​​the target scene can be more intuitively understood. In some examples of this method, in response to the first image display function triggered on the third drawing interface, the position of the object to be identified in each target area can be mapped to the corresponding position in the simulated image for display. Among them, when displaying, the layer where the position information is located can be different from the layer where the simulated image is located.

[0279] Of course, in this example, in response to the first image display function triggered on the third drawing interface, the position information of each object to be identified at the current time can be mapped to the simulated image for display at every preset time interval, thereby, the actual situation of the object to be identified in the target scene can be displayed on the simulated image.

[0280] Processing 2: When the scene task is an object counting task, multiple position information is counted to obtain statistical information, and the statistical information is rendered in a simulated image.

[0281] In some examples of this method, in response to the second image display function triggered on the third drawing interface, the position information of the objects to be identified in each target area can be counted, so that the statistical results are rendered at the corresponding position in the simulated image for display. When displayed, the layer where the statistical results are located can be different from the layer where the simulated image is located.

[0282] In this second processing, the positions of the objects to be identified in the target area can also be counted according to the corresponding dimensions to obtain statistical information in the corresponding dimensions, wherein the statistical information can be rendered in the simulated image so that the statistical results can be intuitively observed from the simulated image.

[0283] Specifically, in the second process, since it is necessary to count the positions of the objects to be identified in the multiple scene images, statistical information is obtained, wherein the statistics include the dimension of statistics according to the object and the dimension of statistics according to the calibrated area.

[0284] The dimension of object statistics may refer to: performing statistics on position information of a same to-be-recognized object recognized in multiple scene images to obtain trajectory information of the same to-be-recognized object.

[0285] In this dimension, multiple scene images may be different scene images, or multiple scene images may include different scene images or the same scene images. Since different scene images are acquired by different image acquisition devices, different image acquisition devices correspond to different calibration areas, that is, they correspond to different areas in the target scene. Therefore, by counting the position information of the same object to be identified in different scene images, the activity trajectory of the object to be identified in different areas (calibration areas) in the target scene can be obtained. If multiple scene images include different scene images and the same scene images, the activity trajectory of the same object to be identified at different positions in the target scene can be obtained.

[0286] Among them, when counting, the positions of the same object to be identified in their respective corresponding areas can be sorted according to the acquisition time of the scene image, so as to obtain multiple position points sorted in time order, and the multiple position points can be mapped to the simulation image to obtain the trajectory information.

[0287] Specifically, in a further example, in response to the first sub-statistical function under the second image display function on the third drawing interface, the position information of the same object to be identified in multiple scene images can be counted to obtain the trajectory information of the same object to be identified, thereby mapping each trajectory point in the trajectory information to the corresponding position in the simulated image for display.

[0288] The dimension of statistics according to the calibration area may refer to: according to the calibration area where the position represented by the position information is located, statistics are performed on each position information to obtain the quantity information of the objects to be identified in different calibration areas.

[0289] In this dimension, since different scene images are captured by different image acquisition devices, and different image acquisition devices correspond to different calibration areas, it is possible to obtain multiple scene images captured by the same image acquisition device within a preset time period, and identify the number of objects to be identified in the multiple scene images, thereby obtaining the number of objects to be identified in the calibration area of ​​the image acquisition device, and so on, it is possible to obtain the number of objects to be identified in each of the multiple calibration areas within the preset time period, thereby obtaining the quantity information of objects to be identified in different calibration areas.

[0290] Specifically, in a further example, in response to the second sub-statistical function under the second image display function on the third drawing interface, statistics can be performed on each position information in each calibration area according to the calibration area where the position represented by the position information is located, so as to obtain the quantity information of the objects to be identified in different calibration areas, and then display the quantity information of the objects to be identified in different calibration areas in the form of a heat map in the simulated image.

[0291] In this type of statistics, the quantity information corresponding to different calibrated areas can be displayed in the simulated image in the form of a heat map, so that users can directly observe the popularity of different calibrated areas. For example, taking customer flow statistics as an example, through the heat map, it is possible to intuitively understand which areas are popular and popular with customers.

[0292] In some embodiments, the object to be identified may be a human body, which can be applied to customer flow statistics in shops. In this case, the position of the human body needs to be identified. The purpose of customer flow statistics can be achieved by identifying the position of the human body's feet. Since the human body's feet can reflect the real position of the human body in the real scene, the accuracy of identification can be improved.

[0293] In a specific implementation, the feet of a human body in the target area may be detected first; if the feet are detected, the position of the feet is identified; if the feet are not detected, the position of the feet is predicted based on the position of at least one target part of the human body;

[0294] The target parts are other parts of the human body except the feet.

[0295] Among them, other parts may include the head, torso, limbs, etc. To ensure the accuracy of the prediction of the foot position, the more other parts there are, the more accurate it is.

[0296] Among them, the method of detecting the human feet can adopt relevant technologies, such as using a neural network to detect the human feet in the target area. In this method, it is only necessary to use multiple image samples containing human feet as training samples to train a neural network model that can recognize human feet. Then, the target area in the scene image is cropped out and sent to the neural network model to obtain the foot detection result.

[0297] When the foot is detected, the position of the foot in the scene image can be further detected to obtain position information, and the position of the foot can also be mapped back to the simulated image display.

[0298] If the feet are not detected, the position of the feet can be predicted based on the position and size of other parts of the human body except the feet in the scene image. Further, other parts may include the head and the torso, so the posture of the human body can be determined based on the positional relationship between the head position and the torso position, the detected head size and the torso size, thereby predicting the position of the human foot.

[0299] In a specific implementation, when predicting the position of the foot based on the position of at least one target part of the human body, the head position and the torso position of the human body in the target area can be detected; then, based on the head position and / or torso position, the body posture category of the human body is determined; thereafter, based on the body posture category, the position of the foot is predicted.

[0300] Among them, the posture category can reflect the posture of the human body.

[0301] In practice, the above neural network model can also detect the human head and human body at the same time; and can detect the position of the human head and the position of the human body. In practice, the neural network model can select the existing model in the relevant technology, which will not be repeated here.

[0302] Furthermore, when a head position frame where the head is located and a body position frame where the torso is located are detected, the body posture category can be determined based on the size ratio between the head position frame and the torso frame; when only the head position frame or the torso position frame is detected, the body posture category can be determined as a preset category corresponding to the detected position frame.

[0303] Among them, the body type categories may include: standing, squatting, sitting and other three categories. In different categories, the size ratio of the head and body of the human body is different. In practice, the size ratio of the head and body of different people in different body types can be obtained based on multiple experiments. Then, the comprehensive ratio of the head and body of the human body in different body types can be estimated, and the body type of the human body can be measured by the comprehensive ratio. Among them, each body type category can correspond to a preset position relationship between the foot position and the head and body position respectively. The preset relationship can be used to obtain the position of the foot through the head or body position.

[0304] Therefore, when the head position frame where the head is located and the body position frame where the torso is located are detected, the body shape category can be obtained according to the size ratio between the head position frame and the torso frame and the comprehensive ratio value. Then, the foot position can be obtained according to the position of the head position frame or the torso position frame and the preset position relationship corresponding to the body shape category.

[0305] If the head position frame where the head is located is not detected, but the body position frame where the body is located is detected, the posture category can be directly determined as the standing category, and the foot position is determined according to the preset position relationship corresponding to the standing category and the position of the body frame;

[0306] Among them, if the head position frame where the head is located is detected, but the body position frame where the torso is located is not detected, the posture category can be directly determined as the standing category, and the sole position is determined according to the preset position relationship corresponding to the standing category and the position of the torso frame.

[0307] The following is a complete description of the region calibration method and information processing method of the present application with an example:

[0308] by Figure 1 For example, assuming that the target scene is a furniture store, it is necessary to count the customer flow in the furniture store. Fig.10 As shown, a flow chart of this example is shown. Fig.10 101 is a first image acquisition device, 102 is a second image acquisition device, and specifically includes the following process:

[0309] The worker arrives at the store and uses the first electronic device 1001 provided by the store to calibrate the calibration areas corresponding to the cameras in the store in the area calibration module provided by the system.

[0310] S1: Place four marker icons within the range that camera i can capture images, then obtain the image captured by camera i, and start measuring the world coordinates of the marker icons at the placement positions; wherein camera i is any camera among multiple cameras, i is greater than or equal to 1 and less than or equal to the total number of cameras;

[0311] S2: Input the acquired image into the neural network model to obtain the image coordinates of the four marking icons in the image, and obtain a total of 4 image coordinates;

[0312] S3: Using the 4 image coordinates and the 4 world coordinates, according to the existing method of converting the camera image coordinate system to the world global coordinate system, calculate the conversion matrix parameters based on the image coordinates to the world coordinates, that is, the first coordinate conversion parameters, which include 8 position parameters and are stored in the coordinate conversion file corresponding to the camera i;

[0313] S4: simulating a simulated image of a furniture store, which may be a plane engineering drawing of the furniture store, which is proportionally scaled with the real furniture store, thereby obtaining a coordinate transformation relationship between the coordinate system of the plane engineering drawing and the world coordinate system, i.e., a second coordinate transformation parameter;

[0314] Thus, the calibration of all cameras is completed, and the on-site workers can upload the first coordinate conversion parameters and the second coordinate conversion parameters to the system to be used by the subsequent system for regional calibration. The system can be located in the computer equipment of the furniture store, can be downloaded in advance, or can be downloaded by the on-site workers.

[0315] S5: Import the cameras in the store into the system so that the system can obtain and display the scene images captured by the cameras.

[0316] S6: Displaying the scene image captured by the camera i in a system interface, specifically, a first image drawing interface specially drawn for the scene image, and then the on-site worker draws an area that does not include invalid information from the scene image according to the drawing tools provided by the first image drawing interface;

[0317] S7: The system automatically maps the drawn area to the plane engineering drawing according to the first coordinate transformation parameter and the second coordinate transformation parameter corresponding to the camera i, and obtains the candidate area i located in the plane engineering drawing;

[0318] After the above steps S3 and S4, candidate areas of all cameras in the plane engineering drawing can be obtained. Assuming there are N cameras in total, there are N candidate areas.

[0319] S8: In the second image drawing interface, the candidate regions on the plane engineering drawing are adjusted. Specifically, if multiple candidate regions overlap among the N candidate regions, the shapes or sizes of the overlapping candidate regions are adjusted so that none of the multiple candidate regions overlap.

[0320] S9: Mapping the adjusted candidate area i back to the first image drawing interface and displaying it in the scene image of camera i. If it is determined that the adjusted candidate area i is still located in the scene image of camera i and does not include invalid information, the adjusted candidate area i is used as the calibration area of ​​camera i; otherwise, continue to execute steps S4 and S5;

[0321] S10: Determine whether the calibration areas of the existing N cameras of the store fully cover the area included in the valid information;

[0322] S11: If yes, the calibration work is completed, and the coordinates of the calibration area corresponding to each camera in the image coordinate system can be returned to the camera, so that the scene image sent by the camera carries the image coordinates of the calibration area.

[0323] S12: If not, in the second image drawing interface, an uncovered area that is not covered by the marked area is displayed, where the uncovered area may refer to an area that is not covered with valid information.

[0324] The on-site worker pre-installs a camera k (from a backup camera carried or a camera provided by the merchant) at a candidate location S in the store, displays the image captured by the camera k in the first image drawing interface, and maps the entire area included in the image to the engineering plan;

[0325] The system takes a screenshot of the second image drawing interface, the screenshot includes the image area captured by camera k and the uncovered area, and then displays the screenshot to a portable display device carried by the on-site worker;

[0326] If it is determined that the image area captured by camera k covers the uncovered area, camera k is installed at position S; if not, the position of camera K is adjusted, and the above steps are repeated until a specific installation posture is determined;

[0327] S13: Camera k( Fig.10 After the position of 102) is determined, the uncovered area can be directly returned to the camera K as the calibration area of ​​the camera K, or the area can be calibrated for the camera K according to the process of the above steps S5-S8.

[0328] Next, the store staff starts the customer flow statistics module in the system and starts to count the customer flow; the second electronic device 1002 may be used. If the first electronic device 1001 and the second electronic device 1002 are the same device, the user can log in to the customer flow statistics module in the system to count the customer flow, including:

[0329] S14: acquiring scene images captured by multiple image acquisition devices of the target scene in real time;

[0330] S15: identifying the foot position of the human body in the area framed by the calibration area in each scene image;

[0331] S16: Mapping the number of human feet identified in the multiple scene images collected by different image acquisition devices to the simulated image, wherein the number of feet can be reflected in the simulated image by heat, thereby forming a statistical heat map of passenger flow, such as Fig.10 As shown, the statistical heat map is represented by the distribution density of the origin.

[0332] Based on the same inventive concept, the present disclosure also provides a region calibration device, referring to Fig.11 FIG. 1 shows a schematic diagram of the structure of the area calibration device, which may specifically include the following modules:

[0333] A display module 1101 is used to display scene images captured by at least one first image acquisition device located in a target scene, wherein the first image acquisition device is used to capture images of a part or all of an area in the target scene;

[0334] A drawing module 1102, in response to a region drawing operation performed on the scene image, displays a candidate region selected by the region drawing operation in a simulation image simulating the target scene;

[0335] The determination module 1103 determines, based on the candidate areas corresponding to the first image acquisition devices, the calibration areas corresponding to the plurality of first image acquisition devices, wherein at least two adjacent calibration areas do not overlap;

[0336] The associating module 1104 associates the plurality of calibrated areas with the plurality of first image acquisition devices respectively, so as to indicate that the to-be-identified objects in the images acquired by the first image acquisition devices are to be identified based on the calibrated areas.

[0337] Optionally, the determining module 1103 is specifically configured to:

[0338] In response to a region adjustment operation on some or all of the plurality of candidate regions, determining calibration regions corresponding to each of the plurality of first image acquisition devices;

[0339] Or, based on the overlapping areas of the candidate areas in the simulated image, the candidate areas are adaptively adjusted to obtain the calibration areas corresponding to the plurality of first image acquisition devices.

[0340] Optionally, the device further comprises:

[0341] A first mapping module, used for mapping the adjusted candidate area to the scene image from which the adjusted candidate area originates for display;

[0342] a repeating module, configured to, upon receiving a modification operation on the adjusted candidate area in the scene image, display the area corresponding to the modification operation in the simulation image, and repeat the step of determining the calibration areas corresponding to each of the plurality of first image acquisition devices based on the candidate areas corresponding to each of the first image acquisition devices;

[0343] The determination of the calibration area is to use the adjusted candidate area as the calibration area of ​​the corresponding first image acquisition device when a determination operation for the adjusted candidate area is received.

[0344] Optionally, the drawing module 1102 includes:

[0345] An overlapping region determining unit, used for determining an overlapping region of each of the candidate regions on the simulated image;

[0346] The display unit is used to display each of the candidate regions and an overlapping area of ​​each of the candidate regions.

[0347] Optionally, the device further comprises:

[0348] An uncovered area display module, used to display an uncovered area in the simulated image; wherein the uncovered area includes an area not covered by the candidate area, or an area not covered by the calibrated area;

[0349] A posture determination module, used to determine the installation posture of the second image acquisition device in the target scene based on the uncovered area, and use part or all of the uncovered area as a calibration area of ​​the second image acquisition device;

[0350] The second image acquisition device is used to be added to the target scene to acquire images of the uncovered area.

[0351] Optionally, the posture determination module includes:

[0352] A target image acquisition unit, used to acquire a target image obtained by the second image acquisition device acquiring an image of the target scene at a candidate position;

[0353] A collection area determination unit, used for mapping the scene area included in the target image to the simulation image to obtain a collection area;

[0354] An information output unit, configured to output position adjustment information based on the acquisition area and the uncovered area;

[0355] A posture determination unit is used to determine the installation posture based on the position adjustment information.

[0356] Optionally, the information output unit includes:

[0357] A type determination subunit, used to determine the coverage type of the acquisition area covering the uncovered area;

[0358] An information generating subunit, configured to generate the position adjustment information based on the coverage type; wherein the position adjustment information includes a moving direction and / or a moving distance of a candidate position of the second image acquisition device;

[0359] The information sending subunit is used to send the position adjustment information to the user terminal.

[0360] Optionally, the information output unit includes:

[0361] A screenshot subunit, used for capturing a screen shot of the simulated image showing the acquisition area and the uncovered area;

[0362] The sending subunit is used to send the captured picture as the position adjustment information to the user terminal.

[0363] Optionally, the drawing module 1102 includes:

[0364] A first coordinate acquisition unit, configured to determine, in response to the area drawing operation, first area coordinates of a drawing area selected by the area drawing operation on the scene image;

[0365] A second coordinate acquisition unit, used for converting the first area coordinates into second area coordinates on a simulated image;

[0366] A mapping unit is used to map the drawing area onto the simulation image based on the second area coordinates to obtain the candidate area.

[0367] Optionally, the drawing module 1102 includes:

[0368] An image region generating unit, configured to generate a closed image region on the scene image in response to a plurality of position points on which a preset operation is performed on the scene image and / or a plurality of lines drawn on the scene image;

[0369] An image region mapping unit is used to display the candidate region to which the closed image region is mapped in the simulated image.

[0370] Optionally, the device further comprises:

[0371] A marked image acquisition module, used to acquire a marked image acquired by an image acquisition device, wherein the marked image includes images of a plurality of marked icons located in the target scene; the image acquisition device includes the first image acquisition device and the second image acquisition device;

[0372] An image coordinate acquisition module, used to respectively determine the image coordinates of the plurality of marking icons in the marking image, and determine the world coordinates where the plurality of marking icons are located;

[0373] A first conversion module, used for determining first coordinate conversion parameters corresponding to each of the image acquisition devices based on the image coordinates and the world coordinates;

[0374] The second conversion module is used to determine a second coordinate conversion parameter based on a world coordinate system and a coordinate system of the simulated image; wherein the first coordinate conversion parameter and the second coordinate conversion parameter are used to map the image area selected by the area drawing operation on the scene image to a candidate area in the simulated image.

[0375] Optionally, the device further comprises:

[0376] a calibration coordinate storage module, configured to determine, based on the first coordinate conversion parameter, calibration coordinates of a calibration area associated with the image acquisition device in the world coordinate system;

[0377] A parameter updating module, configured to update the first coordinate transformation parameter corresponding to the third image acquisition device when a third image acquisition device with a changed posture is detected and if it is determined that the scene image acquired by the third image acquisition device includes an associated calibration area;

[0378] A region updating module, configured to determine a calibration region corresponding to the third image acquisition device after the posture change based on the updated first coordinate conversion parameter and the calibration coordinates corresponding to the third image acquisition device;

[0379] The area association module is used to associate the calibration area corresponding to the position change with the third image acquisition device.

[0380] Based on the same inventive concept, the present disclosure also provides an information processing device, referring to Fig.12 As shown, a schematic diagram of the structure of the information processing is shown, which may specifically include the following modules:

[0381] The image acquisition module 1201 is used to respectively acquire scene images acquired by multiple image acquisition devices of a target scene, and calibration areas corresponding to the multiple image acquisition devices; wherein the calibration areas are determined by the area calibration method;

[0382] The recognition module 1202 is used to recognize the object to be recognized in the target area of ​​the scene image to obtain target information; wherein the target area is the area selected by the calibration area;

[0383] The information processing module 1203 is used to perform information processing corresponding to the scene task on each of the target information, wherein the scene task is a task corresponding to the target scene.

[0384] Optionally, the information processing module 1203 is specifically configured to, in response to a target image display function triggered on the third drawing interface, perform information processing corresponding to the target image display function on each target information; and display a processing result of the information processing on a simulation image simulating the target scene;

[0385] Therein, a plurality of image display functions are preset on the third drawing interface, and different image display functions correspond to different scene tasks.

[0386] Optionally, the target information includes the location information of the object to be identified, the scene task includes: a location display task and / or an object counting task, and the information processing module 1203 includes:

[0387] A first processing unit, configured to map each of the position information to a simulation image simulating the target scene for display when the scene task is the position display task;

[0388] The second processing unit is used for, when the scene task is the object counting task, performing statistics on the plurality of position information to obtain statistical information, and rendering the statistical information in the simulation image.

[0389] Optionally, the second processing unit is configured to perform at least one of the following steps:

[0390] The position information of the same object to be identified in multiple scene images is counted to obtain the trajectory information of the same object to be identified;

[0391] According to the calibration area where the position represented by the position information is located, statistics are performed on each of the position information to obtain the quantity information of the objects to be identified in different calibration areas.

[0392] Optionally, the object to be identified is a human body, and the identification module 1202 includes:

[0393] A detection unit, used for detecting the feet of a human body in the target area;

[0394] A first identification unit, configured to identify the position of the foot if the foot is detected;

[0395] The second recognition unit is used to predict the position of the foot based on the position of at least one target part of the human body if the foot is not detected; the target part is other parts of the human body except the foot.

[0396] Optionally, the second identification unit is specifically configured to perform the following steps:

[0397] Detecting the head position and the torso position of the human body in the target area;

[0398] Determining a body type of the human body based on the head position and / or the torso position;

[0399] Based on the posture category, the position of the foot is predicted.

[0400] Optionally, the step of detecting the head position and the body position of the human body in the target area includes:

[0401] Detecting a head position frame where the head is located, and a body position frame where the body is located;

[0402] Determining the body type of a human body based on the head position and / or the body position comprises:

[0403] When the head position frame and the body position frame are detected, determining the body posture category based on a size ratio between the head position frame and the body frame;

[0404] When the head position frame or the body position frame is detected, the posture category is determined as a preset category corresponding to the detected position frame.

[0405] Optionally, the step of determining the calibration areas corresponding to the plurality of image acquisition devices respectively comprises:

[0406] Determine a calibration area corresponding to the current image acquisition device from a preset calibration area library; wherein the preset calibration area library stores the area coordinates of the calibration areas corresponding to the multiple image acquisition devices respectively;

[0407] Alternatively, the calibration area is determined based on the area coordinates carried in the scene image captured by the image capture device.

[0408] Based on the same inventive concept, the present disclosure also provides an information processing system, referring to Fig.13 As shown, a schematic diagram of the framework of the information processing system is shown, including: a first module 1301, a second module 1302, and a plurality of image acquisition devices 101 located in a target scene; wherein the first module and the second module are both connected to the plurality of image acquisition devices;

[0409] Wherein, the first module is used for the area calibration method;

[0410] The second module is used to execute the information processing method based on the calibration area corresponding to each of the image acquisition devices.

[0411] Wherein, the first module is used to send the calibration areas corresponding to the plurality of image acquisition devices to the second module;

[0412] And / or, used to return the calibration areas corresponding to the multiple image acquisition devices to the image acquisition device.

[0413] The first module may be a functional module in a system, and the second module may also be a functional module in a system; or, the first module may be an electronic device, such as a first electronic device, and the second module may be an electronic device, such as a second electronic device.

[0414] Based on the same inventive concept, the present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes, the area calibration method described in Embodiment 1 or the information processing method described in Embodiment 2 is implemented.

[0415] The embodiments of the present disclosure further provide a computer-readable storage medium, which stores a computer program that enables a processor to execute the area calibration method described in the first embodiment of the present disclosure, or the information processing method described in the second embodiment.

[0416] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0417] The above is a detailed introduction to a region calibration method, information processing method and system, device, equipment and medium provided by the present disclosure. Specific examples are used in this article to illustrate the principles and implementation methods of the present disclosure. The description of the above embodiments is only used to help understand the method of the present disclosure and its core idea; at the same time, for general technical personnel in this field, according to the ideas of the present disclosure, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present disclosure.

[0418] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0419] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

[0420] References herein to "one embodiment," "embodiment," or "one or more embodiments" mean that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least one embodiment of the present disclosure. In addition, please note that examples of the term "in one embodiment" do not necessarily all refer to the same embodiment.

[0421] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present disclosure can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0422] In the claims, any reference signs placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present disclosure may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names.

[0423] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. A method for region calibration, characterized in that: The method comprises: Displaying scene images captured by at least one first image acquisition device located in the target scene, wherein the first image acquisition device is used to capture images of a part or all of the area in the target scene; In response to a region drawing operation performed on the scene image, displaying a candidate region selected by the region drawing operation in a simulation image simulating the target scene; Determine, based on the candidate areas corresponding to the respective first image devices, the calibration areas corresponding to the plurality of first image acquisition devices; The plurality of calibrated areas are respectively associated with the plurality of first image acquisition devices to indicate that the objects to be identified in the images acquired by the first image acquisition devices are to be identified based on the calibrated areas.

2. The area calibration method according to claim 1, characterized in that: There are at least two adjacent calibration areas without overlap; the determining of calibration areas corresponding to the plurality of first image acquisition devices based on the candidate areas corresponding to the first image acquisition devices comprises: In response to a region adjustment operation on some or all of the candidate regions among the plurality of candidate regions, determining calibration regions corresponding to the plurality of first image acquisition devices respectively; Or, based on the overlapping areas between the candidate areas in the simulated image, each candidate area is adaptively adjusted to obtain the calibration areas corresponding to each of the plurality of first image acquisition devices.

3. The method according to claim 2, characterized in that When the candidate region is adjusted, the method further includes: Mapping the adjusted candidate area to the source scene image for display; When a modification operation on the adjusted candidate area in the scene image is received, the area corresponding to the modification operation is displayed in the simulation image, and the step of determining the calibration areas corresponding to the plurality of first image acquisition devices respectively based on the candidate areas corresponding to the respective first image acquisition devices is repeated; The determination of the calibration area includes: When a determination operation for the adjusted candidate area is received, the adjusted candidate area is used as a calibration area of ​​the corresponding first image acquisition device.

4. The method according to claim 1, characterized in that: The step of displaying the candidate region selected by the region drawing operation in a simulated image simulating the target scene comprises: Determining, on the simulated image, an overlapping area of ​​each of the candidate areas; The candidate regions and the overlapping regions of the candidate regions are displayed.

5. The method according to claim 1, characterized in that After the step of displaying the candidate area selected by the area drawing operation in the simulated image simulating the target scene, the method further comprises: In the simulated image, an uncovered area is displayed; wherein the uncovered area includes an area not covered by the candidate area, or an area not covered by the calibrated area; Based on the uncovered area, determining the installation posture of the second image acquisition device in the target scene, and using part or all of the uncovered area as a calibration area of ​​the second image acquisition device; The second image acquisition device is used to be added to the target scene to acquire images of the uncovered area.

6. The method according to claim 5, characterized in that The step of determining the installation position of the second image acquisition device in the target scene based on the uncovered area includes: Acquire a target image obtained by capturing the target scene by the second image acquisition device at a candidate position; Mapping the scene area included in the target image into the simulation image to obtain a collection area; Outputting position adjustment information based on the acquisition area and the uncovered area; Based on the position adjustment information, the installation posture is determined.

7. The method according to claim 6, characterized in that The outputting position adjustment information based on the acquisition area and the uncovered area includes: Determine the coverage type of the acquisition area covering the uncovered area; Based on the coverage type, generating the position adjustment information; wherein the position adjustment information includes the moving direction and / or moving distance of the candidate position; The position adjustment information is sent to the user terminal.

8. The method according to claim 6, characterized in that The outputting position adjustment information based on the acquisition area and the uncovered area includes: Capturing a simulated image showing the acquisition area and the uncovered area; The captured picture is sent to the user terminal as the position adjustment information, so that the captured picture is displayed on the user terminal.

9. The method according to any one of claims 1 to 8, characterized in that: In response to the region drawing operation performed on the scene image acquired by the first image acquisition device, displaying the candidate region selected by the region drawing operation in a simulation image simulating the target scene includes: In response to the area drawing operation, determining first area coordinates of a drawing area selected by the area drawing operation on the scene image; Converting the first region coordinates into second region coordinates on a simulated image; Based on the second region coordinates, the drawing region is mapped onto the simulation image to obtain the candidate region.

10. The method according to claim 1, characterized in that In response to the region drawing operation performed on the scene image acquired by the first image acquisition device, displaying the candidate region selected by the region drawing operation in a simulation image simulating the target scene includes: In response to a plurality of position points on which a preset operation is performed on the scene image, and / or a plurality of lines drawn on the scene image, generating a closed image area on the scene image; The closed image area is mapped to the simulated image for display to obtain the candidate area.

11. The method according to any one of claims 1 to 7 or any one of claims 9 to 10, characterized in that: Before displaying the candidate area selected by the area drawing operation in the simulated image simulating the target scene, the method further includes: Acquire a marked image acquired by an image acquisition device, wherein the marked image includes images of a plurality of marked icons located in the target scene; the image acquisition device includes the first image acquisition device and the second image acquisition device; respectively determining the image coordinates of the plurality of marking icons in the marking image, and determining the world coordinates where the plurality of marking icons are located; Based on the image coordinates and the world coordinates, determining first coordinate conversion parameters corresponding to each of the image acquisition devices; Based on the world coordinate system and the coordinate system of the simulated image, a second coordinate transformation parameter is determined; wherein the first coordinate transformation parameter and the second coordinate transformation parameter are used to map the image area selected by the area drawing operation on the scene image to a candidate area in the simulated image.

12. The method according to claim 11, characterized in that The method further comprises: Determining calibration coordinates of a calibration area associated with the image acquisition device in the world coordinate system based on the first coordinate transformation parameters; When a third image acquisition device with a changed posture is detected, updating a first coordinate conversion parameter corresponding to the third image acquisition device; Based on the updated first coordinate conversion parameters and the calibration coordinates corresponding to the third image acquisition device, determining a calibration area corresponding to the third image acquisition device after the posture change; The calibration area corresponding to the position change is associated with the third image acquisition device.

13. An information processing method, characterized in that: The method comprises: Respectively acquiring scene images captured by a plurality of image acquisition devices of a target scene, and determining calibration areas corresponding to the plurality of image acquisition devices respectively; wherein the calibration areas are determined according to the area calibration method according to any one of claims 1 to 12; Identify the object to be identified in the target area of ​​the scene image to obtain target information; wherein the target area is the area framed by the calibration area; For each of the target information, information processing corresponding to a scene task is performed, wherein the scene task is a task corresponding to the target scene.

14. The method according to claim 13, characterized in that The performing of information processing corresponding to the scene task on each of the target information includes: In response to the target image display function triggered on the third drawing interface, performing information processing corresponding to the target image display function on each of the target information; wherein the third drawing interface is preset with multiple image display functions, and different image display functions correspond to different scene tasks; The processing result of the information processing is displayed on a simulation image simulating the target scene.

15. The method according to claim 13 or 14, characterized in that The target information includes the position information of the object to be identified, the scene task includes: a position display task and / or an object counting task, and the information processing corresponding to the scene task is performed on each of the position information, including: In the case where the scene task is the position display task, each of the position information is mapped to a simulation image simulating the target scene for display; In the case where the scene task is the object counting task, statistics are performed on the plurality of position information to obtain statistical information, and the statistical information is rendered in the simulation image.

16. The method according to claim 15, characterized in that The positions of the objects to be identified in the plurality of scene images are counted to obtain statistical information, including at least one of the following: The position information of the same object to be identified in multiple scene images is counted to obtain the trajectory information of the same object to be identified; Statistics are collected on the calibrated area to which the position information belongs, to obtain quantity information of the objects to be identified in different calibrated areas.

17. The method according to claim 13, characterized in that The object to be identified is a human body, and identifying the object to be identified in the target area of ​​the scene image includes: Detecting the feet of a human body in the target area; If the foot is detected, identifying the position of the foot; If the foot is not detected, the position of the foot is predicted based on the position of at least one target part of the human body; the target part is other parts of the human body except the foot.

18. The method according to claim 17, characterized in that The predicting the position of the foot based on the position of at least one target part of the human body comprises: Detecting the head position and the torso position of the human body in the target area; Determining a body type of the human body based on the head position and / or the torso position; Based on the posture category, the position of the foot is predicted.

19. The method according to claim 18, characterized in that Detecting the head position and the torso position of a human body in the target area includes: Detecting a head position frame where the head is located, and a body position frame where the body is located; Determining the body type of a human body based on the head position and / or the body position comprises: When the head position frame and the body position frame are detected, determining the body posture category based on a size ratio between the head position frame and the body frame; When the head position frame or the body position frame is detected, the posture category is determined as a preset category corresponding to the detected position frame.

20. The method according to any one of claims 13 to 19, characterized in that: Determining the calibration areas corresponding to the plurality of image acquisition devices respectively comprises: Determine a calibration area corresponding to the current image acquisition device from a preset calibration area library; wherein the preset calibration area library stores the area coordinates of the calibration areas corresponding to the multiple image acquisition devices respectively; Alternatively, the calibration area is determined based on the area coordinates carried in the scene image captured by the image capture device.

21. A region calibration device, characterized in that: include: A display module, used to display scene images captured by at least one first image acquisition device located in the target scene, wherein the first image acquisition device is used to capture images of a part or all of the area in the target scene; a drawing module, in response to a region drawing operation performed on the scene image, displaying a candidate region selected by the region drawing operation in a simulation image simulating the target scene; A determination module, based on the candidate areas corresponding to the first image acquisition devices, determines the calibration areas corresponding to the plurality of first image acquisition devices, wherein at least two adjacent calibration areas do not overlap; The associating module associates the plurality of calibrated areas with the plurality of the first image acquisition devices respectively, so as to instruct to identify the object to be identified in the image area framed by the calibrated area in the image acquired by the first image acquisition device.

22. An information processing device, characterized in that: include: An image acquisition module, used to respectively acquire scene images acquired by multiple image acquisition devices of a target scene, and calibration areas corresponding to the multiple image acquisition devices; wherein the calibration areas are determined according to the area calibration method according to any one of claims 1 to 12; A recognition module, used to recognize an object to be recognized in a target area of ​​the scene image to obtain target information; wherein the target area is the area selected by the calibration area; The information processing module is used to perform information processing corresponding to the scene task on each of the target information, wherein the scene task is a task corresponding to the target scene.

23. An information processing system, characterized in that: include: A first module, a second module, and a plurality of image acquisition devices located in a target scene; wherein the first module and the second module are both connected to the plurality of image acquisition devices; Wherein, the first module is used to execute the area calibration method according to any one of claims 1 to 12; The second module is used to execute the information processing method described in any one of claims 13-20 based on the calibration area corresponding to each of the image acquisition devices.

24. The information processing system according to claim 23, characterized in that: The first module is used to send the calibration areas corresponding to the plurality of image acquisition devices to the second module; And / or, used to return the calibration areas corresponding to the multiple image acquisition devices to the image acquisition device.

25. An electronic device, characterized in that: include: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor is executed, the method for demarcating an area as described in any one of claims 1 to 12 or the method for processing information as described in any one of claims 13 to 20 is implemented.

26. A computer-readable storage medium, characterized in that: The computer program stored therein enables the processor to execute the area calibration method described in any one of claims 1 to 12, or the information processing method described in any one of claims 13 to 20.

Citation Information

Patent Citations

  • Video monitoring system of high-speed railway station for realizing passenger traffic management based on scene judgment

    CN107483894A

  • Disaster monitoring method and device, storage medium and electronic device

    CN111212272A

  • Personnel distance detection method and device

    CN112232279A

  • Behavior recognition method and device and storage medium

    CN114202797A

  • Track measurement method and system of mobile robot based on multiple cameras

    CN114549976A